How I Came Up with the Idea to Develop ChatTCP, a TCP Analysis Tool APP
After several years of developing infrastructure, I’ve come to believe that a programmer’s technical proficiency is not determined by how beautifully they write code, but by their problem-solving abilities. Everyone can write code, but whether one can solve problems and how difficult those problems are, are the true measures of one’s technical skills.
Problem-solving basically falls into four categories. One is business problems, which test a programmer’s understanding of the business. Another is code bugs, including those in frameworks and middleware, which test a programmer’s research level of frameworks and middleware. The third is performance issues, which test a programmer’s deeper understanding of the operating system’s bottom layer, data structures, and even hardware. The last is network-related problems, which test a programmer’s ability to solve network issues.
Among these, performance and network issues are the most challenging and serve as a watershed for technical proficiency.
Network issues are difficult to pinpoint, unlike using frameworks or middleware where problems can be solved directly through stack traces, debugging, or logging. The longer the path a data packet travels from the client to the server, the harder it is to troubleshoot. With CDN, Layer 4 and Layer 7 traffic proxies, and the complexity of networks in cloud-native environments, sometimes you don’t even know where a call timeout occurs.
However, excluding more specialized scenarios like CDN and container networks, as developers, mastering packet capture and analysis is sufficient to solve 95% of the problems we encounter, and for the remaining 5%, it can at least help narrow down the problem area.
After several years of middleware development, I’ve found that there’s almost no problem that can’t be pinpointed through packet capture, except for some security issues where packet capture is not allowed.
Packet capture itself is not difficult; the challenge lies in analyzing the captured packets. Some Layer 7 protocol analysis tools can be hard to pinpoint certain issues. For example, why does the client’s request to the server get disconnected immediately upon connection? It could be due to a Layer 7 protocol decoding failure, causing the server to disconnect directly. Why is an HTTP packet captured using a Layer 7 protocol incomplete? It could be because there are two newline characters in the request header, causing issues with Layer 7 protocol decoding. Why does the HTTP event get stuck until the end before returning once, failing to achieve the typing effect of ChatGPT? Because the framework automatically enables gzip compression. Of course, there are many more such issues that are hard to detect directly using Layer 7 protocol tools and need to be analyzed from the Layer 4 perspective. Moreover, Layer 7 supports only a limited number of protocols.
The reason for choosing the TCP protocol is that, currently, TCP is still the mainstream Layer 4 protocol, unless UDP-based Layer 4 protocols like XQUIC become popular. Of course, UDP itself is not hard to understand; the challenge lies in UDP-based Layer 7 protocols.
Actually, during my years of middleware development, many problems raised by business colleagues could be solved through packet capture. Often, I would guide these colleagues to pinpoint problems through packet capture themselves, which not only reduces the burden on others but also enhances their abilities. However, many people are resistant to packet capture, probably due to unfamiliarity.
As someone who has come a long way from being a beginner, I understand the difficulty of learning the TCP network protocol and getting started with Wireshark. Moreover, for me, I’ve applied quite a bit of theory in practical situations. Many developers only review theory during interviews and forget it when it’s time to use it. I couldn’t even remember some flag bits before.
As a blogger who has been sharing technology for several years, I want to lower the barrier and make more people not afraid to learn the TCP protocol. Packet capture and analysis are actually fun and can solve many problems.
I think about how to write about a technical point in a way that readers who don’t understand it can understand, rather than writing for those who already do. I found that even though there are many articles online about the TCP protocol, honestly, no matter how many GIF animations you watch, nothing beats hands-on practice! There’s a difference between theory and hands-on practice, just like why we do experiments in physics and chemistry. That’s how the idea of ChatTCP came about: to make analyzing TCP packets as simple as reading chat records! This not only reduces the difficulty of understanding the TCP protocol but also the learning curve of Wireshark.
Of course, it should also support viewing the details of individual data packets.
Initially, I planned to create an online tool because I’m familiar with website front-end development and can get started quickly, and it doesn’t require installation, making it simpler and shareable. So, the initial online version, Easy TCP Analysis, was born.
After promoting the online tool for a while, I wondered if I could make it even simpler. Hence, the commentary mode was born, which explains how to understand each packet step-by-step. Isn’t that simple enough?
Later, for classic concepts like the three-way handshake and four-way handshake, understanding what happens if the three-way handshake is not completed, and why some handshakes seem to be only “three-way” while others are four-way, I realized that it’s hard for beginners to simulate packet captures on their own. So, I thought of a case library to share these cases, allowing learners to study without having to capture packets themselves. Isn’t that even better?
Furthermore, it would be great if it could assist in discovering some problems. So, a diagnostic function was added, although its capabilities are still limited and need further development and refinement.
Later, I found that looking at the WebSocket protocol wasn’t intuitive enough, and I had forgotten what the data structure of the WebSocket protocol looked like. As I reviewed the WebSocket protocol, I noticed that many articles used a table to describe the protocol’s data structure, which was very intuitive and easy to understand. That’s when I had the idea: Why not directly display the decoding of application layer protocols in a tabular format?
In addition to decoding the application layer protocol for individual data packets, support for flow stream is also essential.
So, why consider an offline version?
After the online version was launched for some time, I received feedback asking if there was an offline version. Indeed, many scenarios are not suitable for the online version because packets may contain sensitive data. Companies pay special attention to data security in this regard. I couldn’t ignore the many requests for an offline version any longer. So, I started rethinking the product’s positioning.
Therefore, I decided to create ChatTCP.
The reason for the name ChatTCP is that “Chat” means conversation, and the name fits the product positioning well, making TCP packet analysis as simple as reading chat records!
Why only macOS? Because I prefer using the Apple system, which is closer to the Linux system. I’ve been using macOS for nine years, starting from when I was in school.
Moreover, I want the product experience to be better, and native development is preferable. Using cross-platform frameworks has some limitations. After all, I started my career with Android and iOS development, and I have a clear understanding of the differences in experience between native and H5.
My energy is limited, so I can only start with the macOS version first.
I liked this idea early on, so I applied for a patent for this human-computer interaction method of analyzing TCP packets through chat sessions.
This is why I did it, from an idea to the online tool Easy TCP Analysis and then to the offline tool ChatTCP.
If you find Wireshark challenging, then give ChatTCP a try!
