Blog series: HTTPS tracing and debugging - A simple way

[ArielBravo]

Intro

Let's face it, tracing and debugging HTTP connections (and specially HTTPS) can be tricky. It is usually your last traccing option, generally attempted when you have exhausted all other possible alternatives.

In fact, this is a question I frequently get from colleagues and clients. Those who have a background in SAP XI/PI may be able to remember some traces using the popular "TCPGateway" (TCPGw) that even SAP has recommended in some notes (i.e. note 856597). And despite it served its purpose back in time, it is no longer an alternative due to its major limitations such as the inability of handling encrypted connections or connection outside our network (Yes, you guessed... cloud connections).

Currently, there are different ways and programs to make HTTP traces, each with its own limitations and characteristics. In general terms, they are not very friendly and they require a deeper knowledge of networking or the protocol. Some of these tools are free, however, others more user-friendly are paid tools. (e.g. Wireshark, Tcpdump, Fiddler, Npcap, SSLSplit, etc).

In this series of posts I will present a simple way to trace and debug HTTP connections, using proxies that act as "Man in the middle".

For those of you who...Wait, did you say "Man in the middle"? Isn't that dangerous? Well, if you're on an open and free network, let's say in an airport, about to pay with your credit card, of course it is!. But in this case, what we want is a controlled use of a proxy that allows us to capture connections and then re-transmit them.



For those of you who don't know it, let me introduce you to "mitmproxy" (Man-In-The-Middle-Proxy). According to their creators:

"mitmproxy is your swiss-army knife for debugging, testing, privacy measurements, and penetration testing. It can be used to intercept, inspect, modify and replay web traffic such as HTTP/1, HTTP/2, WebSockets, or any other SSL/TLS-protected protocols. You can prettify and decode a variety of message types ranging from HTML to Protobuf, intercept specific messages on-the-fly, modify them before they reach their destination, and replay them to a client or server later on. "

With this tool, not only we will be able to trace our middleware integration, but also any kind of application that uses HTTP(S) connections (Odata Services, Gateway, XI/PI, CPI, SOAMANAGER, The cloud connector, Eclipse, onwards and upwards).

This series of blog posts will cover the following topics:

1. On-prem to on-prem


2. On-prem to cloud


3. Cloud to on-prem


4. Cloud to cloud (using SAP Cloud Foundry)


5. Final words, Q&A and snippets

To begin with, a few brief requirements:

• Eagerness to learn


• A trial SAP Cloud Foundry account


• Download mitmproxy according to your platform (plus Linux binaries)


• Access to a Cloud Connector (preferably yours and running locally)


• Some HTTP-based services to try out.


• Recommended:
  
  
  
  • A SAP CPI instance to run some test
  
  
  • A SAP PI system to test
  
  
  • A local NodeJS installation (to run an HTTP echo server)
  
  
  • Postman
  
  
  • SOAP UI
  
  
  
  

That's all. I hope that this series of post can be correctly understood and enjoyed as it has a lot of practical value. As the main focus is the proxy operation, I wont be giving much details about the endopoints such as the web services I'll be using. However, shall you have any doubt, leave it in the corresponding comment section.

Let's get started!

1st scenario: On-prem to on-prem

 

Ariel Bravo Ayala