IO Ninja — programmable terminal emulator/sniffer
Greetings, fellow geeks!
Today I would like to tell you about an interesting tool that can be used by IT-professionals, system administrators, network security experts, penetration testers, and software developers working on device interaction and other low-level IO tasks.
The tool I am referring to is a programmable terminal emulator and sniffer called IO Ninja (from now on I will omit the word “emulator” and simply say “terminal”). I suspect that the definition itself — “terminal/sniffer” — may sound unusual, if not weird. So I’m going to start with the story of its creation.
Bits of history
The very first product made by our company, Tibbo Technology, was the EM100 serial-to-Ethernet module.
A large portion of the development process revolved around designing and implementing network communication protocols for interaction between the module and PC software such as virtual serial ports, configuration and upgrade utilities and so on.
This was arduous, hard work. In our cruel world nothing works as intended — well, at least not immediately. So we spent long months of designing and debugging our communications. Here is the list of software tools that we employed:
- Serial terminal;
- TCP terminal;
- UDP terminal;
- Serial monitor;
- Ethernet sniffer.
When designing and debugging our product we needed to:
- Listen and accept incoming TCP connection;
- Send broadcast UDP packets;
- Receive replies to UDP packets from multiple nodes;
- Send binary data over TCP, UDP, and serial links;
- Display the HEX representation of sent and received binary data.
We didn’t have any problems with Ethernet and serial data monitors — even free products available at that time fully satisfied our needs. Things weren’t as smooth with the terminals.
We found a plethora of free terminal software, but they all had their limitations. First, most of them were not that great at working with binary data. Some did not implement the binary mode at all, and those who did weren’t very convenient — the terminal software we tried did not have a good HEX editor/viewer supporting select, copy, paste, undo, and redo operations. We also found that almost no terminal utilities allowed incoming TCP connections, and none offered support for broadcast UDP packets.
It was after our unsuccessful quest for the required toolset that we’ve got the idea to create a universal all-in-one software package that would incorporate all of the IO facilities we so desperately needed in our work. And then we thought: Wouldn’t it be simply awesome if this new software was also scriptable and allowed adding custom protocol analyzers, syntax highlighters, and testing scripts (for example, wait for a packet, analyze it, generate and send a reply, wait again, and so on)?
Well, the scripting part remained just a dream for a very long time. The first version of our software was “hardcoded”, and it still proved to be immensely useful. Initially, it was a small utility for sending and receiving binary data, with a decent HEX editor, HEX viewer, the ability to accept TCP connections, as well as work with broadcast UDP packets. This utility was uncreatively named “SockTerm”. Here is how it looked:
Besides supporting “missing” socket modes, this utility demonstrated the great convenience of its innovative logging approach — the log was presented as a seamless HEX sheet with packets of the same kind stitched together.
We kept working on this utility in spare time for a year or so. The logging module was improved and re-organized as a fully functional logging engine. When it was ready we recognized that it was suitable for use not only in terminals but in sniffers and monitors as well. Convinced that we were on the right track, we kept working on the project and, after several more months of development we released our first official version of the product, now called IO Ninja:
At this stage, IO Ninja already allowed us to work with all the transports we needed (serial, TCP, UDP), and it did so in two capacities: as a terminal, and as a non-intrusive sniffer/monitor. Different kinds of IO session were implemented as plugins, which were written, like the software itself, in C++. In theory, this plugin architecture allowed us to extend IO Ninja’s functionality as much as we needed. We even through about publishing a plugin SDK and thus allow “advanced” users to create their own plugins. In reality, we never got around to doing this. Why? Because we wanted to aim higher and implement real programmability.
What we wanted to do was to make IO Ninja scriptable. Question was, what scripting language should we choose? Should it be Python? Lua? JavaScript or some of its twins?
Well, just like we had a dream of having a scriptable terminal, we had a vision of what a perfect language for this terminal would look like. And two of the biggest “must haves” on the wish list for this perfect scripting language were:
- High level of compatibility with C (both on the source and binary level), and…
- Safe pointer arithmetic.
It’s easy to see why these two features are so desirable. C is de-facto the standard language for system, network, and other low-level programming tasks. You can find the C implementation of any protocol, any checksum calculation algorithm, any cryptographic algorithm. On top of that, C-structs and pointer arithmetic offer the best way of dealing with binary buffers!
Now add to this the ability to run that code safely and not be bugged by pointer-induced problems — and you’ve got yourself a perfect language for writing IO-related scripts.
We looked around in hopes of finding such a language but to our surprise, nothing was available. The scripting language we needed simply did not exist! I know, right? There are almost 700 languages on the Wikipedia page listing all notable programming languages, and there is no ready-to-use scripting dialect of C with safe pointers!
To fix this ridiculous situation we designed and created the Jancy scripting language. While featuring a high level of source and binary compatibility with C, Jancy provides safe pointers and safe pointer arithmetic. It also has plenty of other delicious IO-related features such as built-in regular expression-based Lexer generator, scheduled function pointers, reactive programming, and much much more.
Once we’ve got the first prototype of Jancy up and running, we immediately started using it as a scripting engine for the new generation of our IO Ninja software, which was released in late 2014.
Conclusion
The biggest advantage of IO Ninja is, by far, its programmability — all of its sessions are implemented as open-source scripts, giving you the opportunity to fine-tune them to your liking (or roll out your own sessions for your own transports and protocols). However, even without writing a single line of code, IO Ninja is an extremely useful tool offering you:
- All-in-one terminal/sniffer/monitor;
- Binary-oriented logging engine;
- Packet builder with convenient HEX editor and C-compatible packet templates;
- Universal redirector allowing you to “redirect anything to anything”.
IO Ninja was born as an answer to our own needs, and I am sure it will help you, too. Download the latest version of IO Ninja from our website (http://tibbo.com/ninja/downloads.html) and take it for a spin — it is completely free for any non-commercial use.
