Journey Into Rails Routing
So you can be pretty productive without thinking too much what’s actually going on. It super comfortable, and you know it very well, and this seems to hold true, except, of course, in the few instances, or many instances, where you just veer a little bit off the happy path. For me, veering off the happy path usually means I’m trying to figure out how something works, or I’m stuck debugging an error, or for some wild reason. Even though I’ve just started learning rails and how it functions, there is some abstraction where I find them, I’m like, “I thought I understood this, but it turns out I actually understood only one layer”. For me, a great example of that is the Rails router.
Rails have a lot of magic that we often take for granted. A lot is going on the behind the clever, elegant abstractions that Rails provides us as users of the framework. And at a certain point, I find it’s useful to peek behind the curtain and see how things really work.
With this in mind, let’s take some time to explore how routing works in Rails. How does a web request accepted by Rack make it all the way to your Rails controller?
ROUTING BASICS
The router basically allows us to recognize URLs. It lets us navigate throughout our application. It’s the go-between of our application and the external world because it’s what handles incoming requests. The router, you can kind of think it as the post office of your application which means you can think of the incoming requests as all of the incoming mail that has to go to the correct places, and correspondingly, responses are outgoing mail. The router is responsible for making sure that a request ends up at the right controller action.
When rails receive an incoming request, the router is the one that has to decide which controller to send you to, and then the controller is gonna be the one to make it from there. The router is kind of like a middle person because the controller can only get what the router sends it.
Rails Middleware
The first step to investigating how the router works is, of course, by identifying exactly where it is. If you look at the Rails guides, it turns out there is a command called rails middleware. This output the entire middleware stack, which is in use within your rails app. If you run this command, you’ll see that a bunch of things is being outputted, but all they really are just the middleware in the order that it’s executed in your app.
What’s middleware again?
It’s a rack app that takes another rack app as an argument. This is an important distinction because you can have rack apps that are standalone, which means that they aren’t initialized with other rack apps. A good way to differentiate these two things is, rack apps that are not standalone are often referred to as rack endpoints.
What’s rack again?
A rack app is an app that responds to call. A proc is often used in this example.
All it does is return a status, body, and headers. All the middleware which you have seen in that middleware stack, well, those are just rack apps.
Now if you run ‘rails middleware’ we see something called Application.routes which means it’s the last piece of the middleware stack. In fact, it is actually very closely related to the router. But we need to figure out what .routes thing actually return. If we do some more investigation and just search in Rails, which is always fun, you’ll find that there’s actually a file called the Rails engine.
lib/rails/engine.rb
It has a definition for routes method. This routes method returns an instance of all of our routes.
Now we know that this is how the request actually gets through the middleware stack into the router.
How does the router route it?
So now the important question is how does the request that comes into a router get sent to the right one of these?
Well, the naive solution that, maybe even on a first glance, it makes sense just to work our way through all of the routes in our app until we find the right one. We could maybe even just iterate through, and write a loop, and we could use a regular expression, and just check the request, and see, if it matches the route or not.
That is an option, of course, right? But maybe it’s not the best because, as you’ all notice you might have a lot of routes and your if statement’s going to get very long. There are already regular expressions in there, and it can be generally confusing and very hard to read. The fact there are a lot of if statements is one problem, but another issue is that, as your routes file grows, this is not gonna scale well. In fact, let's say we have n number of routes and N could be anything depends on the size of your application. This is actually going to run in linear time because in the worst case you could be looking for a route that doesn’t even exist, and you’re going to check every single route in your route file. There’s a definitely room for improvement in our strategy here.
Journey::Routes
Once upon a time, Journey was a standalone gem, before it was merged into ActionPack. It focuses on routes and figuring out how to route an incoming request. It doesn’t know about Rails at all, nor does it care — give it a set of routes, then pass it a request, and it will route that request to the first route that matches.
So where does Journey fit in?
So when we call Application. routes. This routes method returns a route instance,
which we can see, it creates something called routes set. But now I have one more question which is, what is a routes set?
If we look at the definition of the route_set, and now we’re inside of action_dispatch, actually, we’ll see that set has a Journey.Routes within it.
So now let’s see how the Journey engine works. Journey uses a Data Structure concept called a Graph in order to make life much easier for our routes. All you need to create a graph is one node, but usually, you’re gonna see a graph with multiple nodes. Those nodes are connected to each other with something called edges or links. Now edges can be either undirected or directed, and for the context of Journey today, we’re really only going to deal with directed graphs.
We can imagine when journey looks a request, the first thing it has to do is figure out where on earth this request actually needs to go in the routes file.
That’s how it needs to start its search. For example, when we have a request that comes in, “/posts/:id”, we have to start looking for a specific post. What we eventually want to do is send it to the controller where we can look for a specific post by ID. But before even we can find the controller, it seems odd to consider the route or sessions, or any of the moderators routes, because we know for a fact, it’s not even going to be there at all. We’re actually narrowing down a huge number of routes that we don’t have to search through.
What if, instead of looking at every single route, we have to narrow down the addresses but think how we are going to read incoming request? or how does the router read?
That seems like a very big skill for a router to have. Well, we have to teach it. But it turns out that Journey reads similarly to how you and I would read. In fact, the way that Journey reads is exactly how compilers will read code too. So what Journey does is with every single request that comes in, the URL string, except, of course, In order for Journey to read “/posts/:id”, it also has to know what a letter is. It needs to know what grammar is, and then it needs to be able to group them together. If you have worked on NLP before you know what I’m talking about. It’s kind of follow the same pattern.
So Journey has to perform a process called Tokenization. All it really is is the work of breaking one expression down to its minimally significant parts. So the individual parts are called Tokens. There’s one program within Journey that handles the work of tokenizing. It’s a class actually, it’s called the Journey Scanner. So the JourneyScanner actually inherits from Ruby’s StringScanner class. This scanner can take any string, and it’s going to follow a set of defined rules in order to derive tokens.
As you can see if you call the next_token on the scanner instance, you can see it actually splitting apart our request URL.
So next step we are going to figure out what’s going on with those words and make some sense of them. Well, Journey is just like us. They have to follow grammar too. What you and I are intuitively able to do, Journey needs to learn how to do. The way that Journey solves this problem is with some help from another class called the parser. The parser’s job is to take those tokenized pieces, the tokenized string and make some sense of it. The parser does this by utilizing yet another computer science concept called a Syntax Tree. Where you took a sentence and you turned it into a tree of the words in the sentence in order to figure out the parts of the sentence and how to structure it.
The parser creates a syntax tree for Journey and to help it make sense of the grammar of its language.
Here’s an example syntax tree that Journey would generate for a route that corresponds to the show action for a recipes controller.
If you’re curious, you can actually see this in action in the Rails console. You can use Graphviz to visualize it.
What do we do with all these syntax trees?
Well, you might remember our graph. This is where that comes into play. Journey uses a graph to route your request and that graph is made up of that syntax tree, Interesting, right? But let me show you.
So, in the end, it will look like this. Journey builds a graph of all your possible routes by combining this syntax trees. Journey’s code actually refers to this as something called a Generalized Transition Graph. In order for Journey to do what we can do, it has to implement data structures under the hood.
NFA(Nondeterministic Finite Automaton) is a type of state machine, and it handles a little bit of input at a time and decides, “How am I gonna proceed and transition forward?”. This is the current input that I have, this is the current state. I’m going to make my choices based on what I have now, and where I should go forward in the graph next. For computer science students it will be easy because they are exposed to what NFA and DFA are and how it works.
Two Possible Outcomes
There are only two things that can ever happen. Either we’ll find something that matches the state, the string request URL that we took in as our input, or we don’t, It’s really a binary situation here. When we get to the end of our string, and we're at the point in the graph that actually works, that corresponds to real route, then we know that there’s a route that matches our request URL. If we’re able to find a route that matches, and that means we found an acceptable place, an acceptable controller and action, to send the request on forward to. This is also referred to as an Accepted State. If there is an acceptable route to be found, then all we have to do, or all Journey, the router has to do, rather, is just dispatch the request to the corresponding controller and action.
Conclusion
If you made it this far, congratulations! 🎉 As you can see, there’s a lot going on behind the scenes, but hopefully, this has helped to demystify some of the magic.
