Building a High Performance, Asynchronous, Port Scanner with… Ruby?

7 min readJun 8, 2018

Asynchronous, high performance, and Ruby — you might not think that makes a lot of sense. However, the async Ruby future is here; and it’s fast.

Sadly couldn’t find the original link to this piece of original art to credit the artist directly.

To see just how fast, let’s try building an asynchronous port scanner; meaning a port scanner that can look for many open ports at a time. To qualify as “fast”, at least for the sake of this blog post, means it should be able to scan hundreds of ports a second.

Just how do we do that?

🔧 Single-Threaded Design

The first option you might jump to in order to increase performance in languages like Ruby or Python is usually multi-threading. This isn’t the absolute worst choice but quickly becomes a bottle-neck in and of itself due to a global lock these languages enforce on their runtime ensuring only one thread runs at a time, even if you have many.

Effectively, this means your “multi-threaded” programs are really more like a confusing single-threaded program — sometimes actually performing worse than their truly single-threaded variants. If this is the first time you’ve heard this, it can be kind’of tricky to really understand.

The main advantage of using multi-threading is providing a means to deal with real world situations like network or disk I/O (input/output) operational latency. So, programmers have come up with abstractions to deal with this. With multi-threading APIs in Ruby or Python, we have an abstraction as a programmer to splice up our work into time segments. On I/O blocking operations, the language will decide that another thread can run, usually providing a speed boost for things like varying network connection speeds. This doesn’t come cheap though.

Threads will often end up “fighting” each-other for control of the runtime causing CPU context-switching which eats up your performance. This can be a really annoying thing to wrap your head around and provide a rather erratic runtime behavior that is a pain to debug or code complex interactions.

To avoid the headache of orchestrating multiple threads, we will be building a single-threaded port scanner. But, like in a multi-threaded program, we still want to be able to weave together tasks asynchronously.

If a program can be broken up into a series of tasks that can be executed independently, it can be considered asynchronous. It is a property of how a program can be executed, not what it does. — Samuel Williams

⚡️ Modern Async Support for Ruby

For sometime now, Samuel Williams and friends have been working away on bringing modern, asynchronous and timeout-capable I/O for Ruby to the masses under the Socketry project.

The creator of Ruby has even tweeted about it before!

Side Note: You can read more about it here.

Using async-await will provide us with a clean API to describe asynchronous tasks. To ensure our network connections are asynchronous, we will use async-io. Using async Gems within the Socketry project provides a plethora of options to build non-blocking I/O programs with great performance. The underlying design of async is based on a Reactor Pattern for concurrency backed by Fibers.

Side Note: Another project you might want to check out is Falcon, a high performance asynchronous multi-process web-server built on async libraries.

👩🏽‍💻 Implementing the Port Scanner with Async

To get started, let’s write the first part of the script to require the async libraries we want to use.

require "async/await"
require "async/io"
require "async/semaphore"# more code ...

Now, let’s create a PortScanner class, including the libraries within it to provide clean async semantics and create our #initialize method:

# previous code ...class PortScanner
  include Async::Await
  include Async::IO  def initialize(host: "127.0.0.1", ports: (1..1024))
    @host      = host
    @ports     = ports
    @semaphore = Async::Semaphore.new(`ulimit -n`.to_i)
  end  # more code...
end

By default, if no host: argument is given when creating a new instance of our scanner, our localhost will be the target to scan. If not given any Range/Array/Enumerable for ports: , then ports 1...1024 will be used, very similar to nmap.

The @semaphore will be used to lock down the number of scans going on at a time to whatever the max number of available file descriptors are on the system. For every open connection, there’s an open file on the system, and we want to limit this to what ulimit tells us that is, at least specifically on unix-like operating systems including macOS or Linux.

Note: ulimit is a builtin shell command used to manage various resource restrictions. The -n flag will reveal the maximum number of open files allowed; #to_i on the output from that command in the code will turn it into a Integer. We cache this result and use it to set the maximum number of open connections we’ll try at a time, since there’s no reason to push past that threshold.

Now, let’s implement a method which can scan a single port in our PortScanner class:

⚠️ This is the code to scan a single port. Moreover, this is will be the most complicated part of the code because of all the things we need to juggle, which I’ll explain in a bit more detail in just a moment.

# inside of PortScanner, under initialize ...def scan_port(port, timeout: 0.5)
  timeout(timeout) do
    Async::IO::Endpoint.tcp(@host, port).connect do |peer|
      peer.close
      puts "#{port} open"
    end
  end
rescue Errno::ECONNREFUSED, Async::TimeoutError
  puts "#{port} closed"
rescue Errno::EMFILE
  sleep timeout
  retry
end# more code...

To break down what’s happening, the method #scan_port will attempt to connect to the target on the given port — I know, that’s the obvious part.

If any error occurs — and that’s going to happen— we can rescue that method and do something based on the cause of the error, right inside of the method itself.

🤔 Why do we do this again?

During a port scan, especially for very large ranges, say thousands, or tens of thousands, our scanner may try to open up a connection, but — due the the limitations of our operating system — we may not be able to. We could be browsing Twitter or GitHub while our scanner runs super large batches after-all. Remember ulimit and the @semaphore from earlier inside of #initialize?

🤹🏻‍ Three Essential Errors to Juggle:

🛑Errno::ECONNREFUSED is an error that will occur when a port is not open to connect to. This is is a pretty typical response for most ports, and this response that is enforced with most REJECT firewall rules.
⏱Async::TimeoutError is an error produced by timeout(timeout) when an operation didn’t complete within the allotted timeout: value ( default of 0.5 ). This may happen for various reasons including a DROP firewall rule where the target doesn’t explicitly tell you a port isn’t open from the scanner’ perspective. We really don’t want to wait around forever to get nothing.
📑Errno::EMFILE is an error that will occur when we try to connect to a port, but we have way too many connections (files, really) already open and need to schedule the execution to happen again, but just a little later, continuously delaying its execution until it’s able to be processed.

Keeping track of the appropriate errors helps prevent missing open ports on larger scan ranges and allows us to handle things according to the error. In certain situations, multi-threaded port scanners can end up missing open ports simply because the scanner is counting any and all errors as a “closed port” when it probably should be retrying the ports that error’d out because of the operating system’s limitations. This goes for basically every port scanning implementation I’ve created and/or used.

In order to scan all the ports the PortScanner class is initialized with, let’s implement a #start method which will iterate over each port and calls the #scan_port method we just created. But, this time we’ll use the async keyword in-front of the method definition.

# inside of PortScanner, under scan_port ...async def start(timeout: 0.5)
  @ports.map do |port|
    @semaphore.async do
      scan_port(port, timeout: timeout)
    end
  end.collect(&:result)
end

Within the #start method, each port in the range that the PortScanner is initialized with will be scanned, running asynchronously — and resource bound to the operating system’s limitations with the @semaphore. The #collect(&:result) near the very end of the method will allow us to collect the result of each port scan attempt.

⚙️ Using the Scanner

To use the port scanning API we just created could look something just like the following:

# previous code ...PortScanner.new(host: "127.0.0.1", ports: (1..1024)).start

There aren’t any open ports to test with? Using nc, it’s pretty easy to create some open ports. The following will setup a TCP listener on your localhost on port 1024:

$ nc -v -l -p 1024

Now, let’s run our port scanner, using grep to filter for the open ports:

$ ruby async_port_scanner.rb | grep "open"
1024 open

💥Bam! Just like that, we have a port scanner! How fast does it go? Using time on Unix-like operating systems can help us get an idea:

$ time ruby async_port_scanner.rb | grep "open"
1024 openreal 0m2.345s
user 0m0.531s
sys  0m0.130s

That’s roughly 2–3 seconds to scan all those ports (1024 of them ) on a single thread! What if we wanted to scan all the ports (1…65535), how long would that take?

real 1m12.693s
user 1m5.933s
sys  0m5.157s

So, if I can do the calculation right: 65535 ports/(1min + 12secs = 72secs) would be roughly ~910 ports a second. That’s pretty fast! 🚀

📃 Source Code

Below is the full source code for the scanner (copy+paste friendly for testing):

🤘
40 lines of asynchronous port scanning awesomeness!

👋 Conclusion

Building a high performance, asynchronous port scanner with Ruby is actually pretty awesome, and very possible! By embracing single threaded performance backed by Fibers provides really interesting opportunities, and great performance for I/O-bound operations ( like a port scanner ) that should only get better over time.

If you like what you see, be sure to give the Async/Socketry gem(s) a star 🌟on GitHub — or, make a pull request to make them better! If you see where I did something wrong, want to improve the implementation, or have any questions, feel to chat on Gitter!

Until next time, that’s all folks!