Using MITMProxy to view Fujitsu K5 HTTPS traffic

So when messing around on K5, i find sometimes really need to see the traffic between my client (eg. openstack python cli) and K5 itself.

The best way to do this is using mitmproxy (man in the middle – proxy).   It’s a compiled python script that produces an easy to use CLI GUI.

So i was getting this error:

TASK [Create the server with attached volumes] **************************************************************************************
fatal: [sap01 -> 127.0.0.1]: FAILED! => {"changed": false, "extra_data": null, "failed": true, "msg": "(404) Client Error for url: https://image.uk-1.cloud.global.fujitsu.com/images/detail ERROR:/images/detail:not supported"}
fatal: [sap02 -> 127.0.0.1]: FAILED! => {"changed": false, "extra_data": null, "failed": true, "msg": "(404) Client Error for url: https://image.uk-1.cloud.global.fujitsu.com/images/detail ERROR:/images/detail:not supported"}
fatal: [sap03 -> 127.0.0.1]: FAILED! => {"changed": false, "extra_data": null, "failed": true, "msg": "(404) Client Error for url: https://image.uk-1.cloud.global.fujitsu.com/images/detail ERROR:/images/detail:not supported"}

And i wondered if it was something i did.

So i broke out mitmproxy.  Here is the setup and a quick run through…

#
# set up mitmproxy and spoofed CA cert
#

# download, un-archive and run once
wget https://github.com/mitmproxy/mitmproxy/releases/download/v2.0.2/mitmproxy-2.0.2-linux.tar.gz
tar xvzf  mitmproxy-2.0.2-linux.tar.gz
./mitmproxy

# exit mitmproxy - we ran it to create the ca certs

# now copy the spoofed SSL certs to default folder
sudo mkdir /usr/share/ca-certificates/mitmproxy
sudo cp ~/.mitmproxy/mitmproxy-ca-cert.cer /usr/share/ca-certificates/mitmproxy/mitmproxy-ca-cert.crt
sudo dpkg-reconfigure ca-certificates

# now run mitmproxy again
./mitmproxy

 

sudo dpkg-reconfigure ca-certificates – this re-complies the certs into a single file.

At this point we have the certs in the right place and we now need to make sure python (2.7 in my case) can find them.

In a separate CLI (term / console window) i run the below.

#
# setup python to use mitmproxy and spoofed CA cert
#
export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
export https_proxy=http://localhost:8080

Environment variable REQUESTS_CA_BUNDLE points python to the newly created ca certs bundle.

And https_proxy, well that is the usual proxy environment variable used by most linux apps.

mitmproxy runs on port 8080 by default.   -p 8081  changes the port to 8081

 

So to run a test i used the openstack python CLI.

$ openstack server list
+--------------------------------------+------+--------+------------------------+--------------------------------------+--------+
| ID                                   | Name | Status | Networks               | Image                                | Flavor |
+--------------------------------------+------+--------+------------------------+--------------------------------------+--------+
| 26426f26-aec2-4c6d-bbff-bad0c3026331 | test | ACTIVE | cust-network=10.2.3.21 | Ubuntu Server 14.04 LTS (English) 02 | S-1    |
+--------------------------------------+------+--------+------------------------+--------------------------------------+--------+
$

 

And mitmproxy captured the below

>> GET https://identity.uk-1.cloud.global.fujitsu.com/v3
       ? 404 application/xml 231b 139ms
   POST https://identity.uk-1.cloud.global.fujitsu.com/v3/auth/tokens
        ? 201 application/json 9.43k 571ms
   POST https://identity.uk-1.cloud.global.fujitsu.com/v3/auth/tokens
        ? 201 application/json 9.43k 1.07s
   GET https://compute.uk-1.cloud.global.fujitsu.com/v2/69206b8239bd41c29012785db1ae2099/servers/detail
       ? 200 application/json 1.63k 1.18s
   GET https://image.uk-1.cloud.global.fujitsu.com/v2/images?limit=20
       ? 200 application/json 9.73k 1.58s
   GET https://image.uk-1.cloud.global.fujitsu.com/v2/schemas/image
       ? 200 application/json 3.55k 593ms
   GET https://compute.uk-1.cloud.global.fujitsu.com/v2/69206b8239bd41c29012785db1ae2099/flavors/detail
       ? 200 application/json 17.78k 729ms

Each of the above lines can be drilled down into, to see the full request/response bodies including headers.

Have fun!

About mohclips

Born in the UK, educated in NZ, working for a tier 1 multinational
This entry was posted in Uncategorized. Bookmark the permalink.

4 Responses to Using MITMProxy to view Fujitsu K5 HTTPS traffic

  1. I was having the same issue (weirdly) and remembered I’d seen this article. Turns out, the issue with the Shade library still isn’t resolved 🙂 That said, it was still really useful to see this in-action!

    Liked by 1 person

  2. mohclips says:

    The shade issue is actually a non-supported API call from Shade into K5. That is K5 doesn’t support that one API call. But so far it’s only one API call and it’s to do with checking the images version before creating a compute instance.

    Like

  3. I needed this page of information again today for something on a RHEL7 based docker container. With that you need to amend just a little bit:

    sudo cp ~/.mitmproxy/mitmproxy-ca-cert.cer /usr/share/ca-certificates/mitmproxy/mitmproxy-ca-cert.crt

    /\ That becomes: sudo cp ~/.mitmproxy/mitmproxy-ca-cert.cer /etc/pki/ca-trust/source/anchors/

    sudo dpkg-reconfigure ca-certificates

    /\ That becomes: sudo /bin/update-ca-trust

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.