Hacking with Charles Proxy
Today, let’s talk about Man in the Middle method that allows anyone to track any traffic sent and received by a smartphone or a browser on your computer. Sometimes, you cannot put the data you want in your application, but if you can modify content received by a browser or an application just before sending it to the smartphone, then you have a very convenient way to test with on-demand data.
This is what I will explain in this article with a practical example.
I will focus on Testing in a desktop browser, but also with Android and iOS applications. For this, I will use Charles Proxy. Later in another blog post, I will maybe write about mitmproxy which is free and open source, but a bit less easy to use. So I will detail how to use Charles Proxy with a MacBook, a desktop browser and an iPhone in this article.
Man In the Middle Proxy
What is Charles Proxy?
Charles Proxy is an interactive man-in-the-middle proxy for HTTP and HTTPS. It enables anyone to view all of the HTTP and SSL/HTTPS traffic between one of their machine and the Internet. This includes requests, responses and the HTTP headers (which contain the cookies and caching information). It is also possible to modify requests before they are sent to the server, or, and this is what I will describe here, the response received which opens lots of opportunities.
How to Install Charles
Charles is available for Windows, MacOS and Linux. Just download the one you need on Download page and follow instructions.
You will get a trial version, because Charles is not free (while mitmproxy is), but it’s worth the price. I have to admit that I first use mitmproxy, but Charles is way easier to use and if you really need a proxy, you may probably not regret the expense.
How to capture things…?
…that will be displayed in a Desktop browser
If we want to capture packets from a browser, we must configure it to use Charles. It is by default configured to use port 8888, but you can modify it for anything you want by opening Proxy Settings.
You can open Help then Local IP address to know what IP needs to be used in order to access the browser, but most of the time you will probably do it locally by using localhost (127.0.0.1).
Let’s configure our browser. Go to the proxy settings, then add 127.0.0.1 as the address of the proxy and 8888 as the port.
You will now capture all packets you request and responses but only for http servers. For example, if https://www.lyontesting.fr is captured by Charles you will see this response which is not quite very comfortable to read.
Just right click on the request, and select “Enable SSL proxying”.
Then refresh the page in browser and you will see that the result is not better. That’s because the page www.lyontesting.fr is redirected to lyontesting.fr. Go to Proxy/SSL Proxying Settings and adapt the location host in order to match all cases you want to catch.
The fun thing is that you can modify the response that will be sent to the browser. For this, just add breakpoints like this.
In Proxy/Breakpoint settings…, select “Enable breakpoints”, add a breakpoint that detects POST request with host=lyontesting.fr and select Response.
Now refresh again the page, the response will not be sent to the browser and is waiting for your approval.
In the “Edit response” tab, you can modify the html. Here I replace “Lyon Testing” with “World Testing”. And the result is displayed in the browser.
And now, you probably think that this could have been done with the dev tools of your browser quite simply, and you are right. But these easy steps were there to show you the basics of Charles and proxying. These things can also be used for smartphone applications and that’s what we will see now.
…that will be used in a smartphone App
Charles Proxy now also comes as an iOS application. With it, you can specify which hostnames to include or exclude in the recording (for example, only the requests to the server of your application), same for SSL Proxying. To install certificates, follow the instructions.
The Charles proxy app is quite a recent application, and it’s not possible to modify the responses received. Maybe that it will be added soon as a new feature. So if you want to edit responses, you’ll need to use the Desktop version of Charles proxy. Lucky you, you already installed it before. If not, better late than never.
Your smartphone must be on the same local Network (if using the same WiFi then it should be ok) in order to be able to capture the traffic from the smartphone. In Charles, go to Help then Local IP Addresses.
You will find the IP address that needs to be used as the manual proxy in your WiFi access settings on the smartphone. Here, with an iPhone, using default Port 8888 used by Charles.
That’s it, flows should be displayed in your Charles Proxy. You also need to install the Charles certificate on the Smartphone. Procedure is explained in Help/SSL Proxying menu. Let me copy/paste it for you: “Configure your device to use Charles as its HTTP proxy on 192.168.2.1:8888, then browse to chls.pro/ssl to download and install the certificate. Note that on iOS 10 and later you must then go into Settings > General > About > Certificate Trust Settings and enable the Charles certificate to be trusted.”
Just like at the beginning of this article, right click on a flow then “Enable SSL proxying” may be needed.
Voilà.
OK, so now you know how to display the http/s traffic. What could we do with this?
Filter and alter request or response
When you test, it’s not always easy to display what you want. Depending on the testability of your product, it can be hard to check all corner cases if the production back-end must be used and cannot be tricked.
With a proxy, you can change what will be displayed.
Let’s see what kind of data a weather application sends; for example, the one I may use almost every day before going out with my bike: rainToday. Hopefully one that doesn’t use certificate Spinning, otherwise our goal would be harder to reach.
Select a town when it’s raining….
…and see what is captured by Charles. Requests to weatherpro.consumer.meteogroup.com should appear in the application, then I suggest you to add a filter in order to not display other requests.
And when you select a town, you will see that 2 requests has been sent:
- a GET /weatherpro/RainService.php?method=getRainChart&areaID=UWZFR4562
- A GET /weatherpro/WeatherServiceFeed.php?format=xml&lid=1837351&mode=observation
Display the response of the first one, and you will see a json similar to this one:
{ "precAngle": 18, "domainMax": 100, "domainMin": 0, "avg": [36, 36, 35, 35, 35, 35, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 34, 34, 34, 34, 33, 33, 33, 33, 32, 32, 32, 31, 31, 31], "min": [32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 34, 34, 34, 33, 32, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 22, 21, 20, 20, 20, 19, 19, 19, 19, 18, 18, 17], "max": [37, 37, 37, 37, 37, 37, 37, 36, 36, 37, 37, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36], "startMin": 0, "endMin": -1, "intensity": 0.3477275, "uwzid": "UWZFR4364", "winddir": "18", "startdtg": "2019-02-03 21:00:00", "creationdtg": "2019-02-03 21:05:59", "interval": "60000" }I guess that rain has a value between 0 and 100, but because future cannot be 100% certain, rainRadar is giving approximate values for every minutes (interval: 60000, 60 values for avg, min and max).
What if we want to test extreme values like:
Or better, invalid values and see how the application behaves:
I just have to copy-paste the 3 lines, avg, min, max in a text editor, then replace first values with 50, 0, 100 and second values with 0, 100, 0 (repeated 3 times each in order to be more visible in the app). The result is:
"avg": [50, 50, 50, 0, 0, 0, 50, 50, 50, 36, 36, 36, 36, 36, 36, 36, 36, 36, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 34, 34, 34, 34, 33, 33, 33, 33, 32, 32, 32, 31, 31, 31], "min": [0, 0, 0, 100, 100, 100, 0, 0, 0, 35, 35, 35, 35, 35, 35, 35, 35, 35, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 34, 34, 34, 33, 32, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 22, 21, 20, 20, 20, 19, 19, 19, 19, 18, 18, 17], "max": [100, 100, 100, 0, 0, 0, 100, 100, 100, 37, 37, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36],But we need to intercept the flow, then need to add a breakpoint that detect a GET request and the following Path: /weatherpro/RainService.php* then capture response.
Select the town again, or another one and the breakpoint will stop the flow.
In order to edit the flow, select Edit Response and Json Text.
Then replace the 3 lines with the above-mentioned lines, then click on “Execute”. Alright, you were probably not quick enough if you read this at the same time. Practice and you will be fast enough in order to fake the application.
The new values will be interpreted by the application, which is not crashing or displaying values outside of the display, that’s a good point.
And you, what can you fake on the application you are testing? Are you able to make it crash when displaying unwanted data?
Warning
Under certain conditions you may have difficulties using Charles Proxy (or any other Man In The Middle tool) if the application forbids network proxies or use SSL pinning. Read for example this well explained article: How to make your iOS apps more secure with SSL pinning
For Android, beginning Android 7 (SDK v24), the SSL network is not directly visible unless you add a piece of code, but it’s probably a bad idea to activate the following trick on any production release. See this very clear article about this subject: Enable Android Nougat ‘Charles’ing SSL network
Conclusion
Charles proxy is a very powerful tool, you can modify what you want and test an application or a website even when some features are not yet available, alter data received, modify requests sent and so on.
I hope this article has been useful to you. Please comment with what you were able to do thanks to Charles Proxy, I’m really interested by your stories.
Happy Testing/Hacking!
Originally published at lyontesting.fr.
