How I Built a Custom Framework and App with Rack: Part 1

I was introduced to Rack while working with Sinatra and was only vaguely knowledgeable about what it meant for Sinatra to be “built on top of Rack,” not to mention how to work with Rack, itself.

While learning how to work with Sinatra, I continually encountered scenarios that made me stop and ask myself: “Well, how does Sinatra work “under the hood,” anyway?” Whenever I tried to take a peek behind the curtain, I would come upon another world entirely: The world of Rack. It was evident to me that if I wanted to understand Sinatra, I ought to understand Rack, too!

The Rack Specification and the Problem it Solves

Rack is, at its heart, a specification. It puts forth rules that both Ruby-web servers (aka ‘application servers’) and Ruby applications should follow to send and receive information between each other.

In the same way that HTTP introduces one protocol, thereby allowing a myriad of machines and software to communicate, Rack is a protocol (for web apps and web servers). It acts as a standard interface between web servers and ruby applications.

Therefore, the Rack specification solves the many-frameworks/many-web-servers problem. First of all, rather than expecting web-server developers to take sole responsibility for developing framework-specific APIs (or vice versa) Rack splits the responsibility between developers on both ends. Everyone just follows the rules of Rack, sharing the task of maintaining a single interface.


Getting more technical about the Rack specification

As I mentioned above, the Rack specification solves the “many frameworks/many web servers” problem by detailing a generalized interface that applications and ruby web servers can use to communicate.
A ruby web server needs to send information to the application of a particular format: one that the application is written to handle. Similarly, the application’s logic must return the information to the ruby web server that the ruby web server’s code is written to handle. Otherwise, errors will result.

Whatever a ruby web server does after it receives an HTTP request doesn’t matter regarding Rack-compliance; it’s the last step the server takes — starting the ruby application itself and passing in the right values — that makes it Rack-compliant. As long as it completes this final step, it will work.

A “Rack-compliant” ruby web server is simply a Ruby web server that parses the client’s HTTP request into a hash with specified key-value pairs (as described in the Rack specification) and then calls the Rack application via example_app.call(env) where env is the hash containing the parsed HTTP -request information.

If this discussion of web servers, ruby web servers, and applications have you a bit confused, consider reading my article on the differences between them and how they interact.

From the above description of ruby web servers, it is clear that the ruby web server will be loading an instance of the application and calling it via app.call(env). Therefore, it expects the application to respond to app.call because that is what the Rack-specification requires of rack-applications. The ruby web server also expects that the call method will take the env hash as an argument.

There are a few more requirements of Rack-applications. First, the application code not only has to respond to call(env), but the call method must return an array with three elements. The first element is an HTTP status code, the second is a set of key-value pairs which will be used to create the HTTP headers, and the third is an array of Strings:
[‘200’, {‘Content-Type’: ‘text/html’}, [“Hello World!”]]`

See this great blog post for a more in depth discussion of Rack-based Ruby web servers.

With the preliminary information about Rack out of the way, it’s time to dive into Rack-app development.


What should the app do, and how should it do it?

What features should the app have, and how should those functions all fit together into one coherent experience for the user? These are questions to ask oneself at the beginning of development.
I wanted to build an application that could:
• Serve custom content based on the client’s requested route
• Redirect clients to a different route at the developer’s discretion
• Output dynamic content
• Render templates and layouts
• Maintain state between each request/response cycle.

I, therefore, needed the following:
• An easy way to render templates with a templating engine (I chose ERB due to familiarity).
• A way to store routes with associated executable code.
• A way to integrate sessions.
• A way to add redirects to stored routes.

These are all non-specific features that any framework would need to offer for web application development. It made sense, then, to group these into a single class that I would later use as a framework. Whichever application I decided to build would then simply inherit from this framework.


My General Workflow

After developing a high-level list of features from the perspective of the hypothetical user, I zoomed in on one feature at a time, breaking it down into a sequence of steps. Each step required its logic and implementation.

So, I worked on four levels of abstraction:

Highest-level: List of features for the application, described from the user perspective. Void of implementation details. E.g., routing

Mid-level: The sequence of steps that detail the subprocesses that comprise a feature from start to finish (represented by a flow chart). E.g., routing steps:

store routes -> get client’s request -> match client’s request to a stored route -> execute code associated with stored route

Low-level: The logic each of the steps developed above should take (represented by pseudo code). E.g., Writing the pseudo code for a method that stores out in a certain data structure.

Really-low-level: Implementation of the logic for each of those steps. E.g., implementing the pseudocode developed in the preceding level into the code itself.

It is better to test your pseudo code as you develop it by combining the ‘low-level’ and ‘really-low-level’ steps into one alternating step. Develop some logic for the sub function, test it, develop more logic, test it, etc.

Below is a general outline of the workflow I followed.

Software development is a continuous loop of planning, implementing, researching, and learning.

Designing the Routing Functionality

I had my list of features; now I just had to pick one to implement, first. The first feature I worked on was routing. Routing in an application is a combination of many sub-features. I chose to represent the sequence of steps that routing should take as a flow chart (shown below).

Since the application was going to be subclassing from the framework’s class, I thought it appropriate to store all of the routes, and their associated executable code, in an instance variable that the child-class could access. The instance variable and the data it referenced (i.e., the routes) would persist as long as the application object persisted.

So, to integrate routing functionality into the framework, the framework needed:

• An instance variable that stored route information, coupled with executable code.
• A mechanism that added custom routes to the route storage.
• A mechanism that read the route requested by the client and matched it with a route stored in the routes instance variable.
• A mechanism that executed the code associated with the stored route information if, and only if the requested route matched the stored route.

The above requirements contain further implicit requirements:

• The routes need to be added before the client’s request was read and compared to the route information stored in the instance variable.
• The route-relevant information from the client’s request had to be extracted from the request.

The steps that comprise the routing feature

A few crucial points about web applications

There are some implicit constraints in web-application development (without a database) by nature of the statelessness of HTTP.
First, when the starts up, it initializes an instance of the Ruby application and keeps that instance in its process. Then, every time the Ruby web server receives a request, it will invoke the call method on that instance. However, a lot of information is still lost: once the call executes and returns its response, the information in the env hash passed to that object is lost. Therefore, each invocation of the call method is passed a completely new env hash. This exemplifies the statelessness of HTTP. To maintain state, the application will have to store information about the client with the client in the form of a session, and then extract the session information out of the subsequent request sent from the client to the server.

Second, this means that the application code executes every single time a request is “sent” from the ruby web server to the application.
The application will, therefore, have to set each route over again every time a new request is received. *

*Edit on December 4, 2017: The routes would not have needed to be set each time a request was received from the client if the routes were set up within an initialize method rather than within the call method. In fact, logic that only needs to execute once-per-process should not be placed within the call method of a Rack application.


Implementing the Routing Functionality

I decided to initialize an empty hash to an instance variable @routes every time the ruby web server spawned a new instance of the application (which, again, would happen in response to every HTTP request).

class Framework
def initialize
@routes = {}
end
end

The goal was to build an easy-to-use interface to add routes to the @routes hash:

add_route(“get”, “/hello”) do
“Hello World”
end

From this example, I knew that I needed an add_route method that took an HTTP method, a request path, and a block as arguments. I also needed the block’s code to be stored with the route, and only be executed if the client’s request matched the stored route.

The easiest way to accomplish this was to store the block in the @routes hash as a Proc object. The @routes hash would not only store the HTTP request methods and request paths: it would store the code to execute when a client’s request matched the request method (e.g. ‘GET’) and the request path (e.g. ‘/hello’).

Here is my add_route method:


def add_route(http_method, path, redirects={}, &block)
response = Rack::Response.new
response.body = block
 if redirects[:location]
response.headers[“Location”] = redirects[:location]
response.status = ‘302’
end
 @routes[[http_method, path]] = [response.status, response.headers,  [response.body]]
end

Remember that a Rack-compliant application responds to call(env) and that the env hash contains a parsed HTTP request with lots of handy key-value pairs, like env[‘REQUEST_PATH’]. Further, the application’s call method must return an array with three elements: an HTTP status, response headers, and an HTTP response body (which needs to be an array of strings). Keep that in mind as we go through the explanation of the add_route method…

As you can see, the add_route method takes four arguments: an HTTP method, a request path, a hash called redirects, and a block. The redirects hash is optional and is initialized to an empty hash if nothing is passed in.
The first line of the add_route method simply wraps the response array (in a convenience object. Now, all I need to do to add headers, a status, or a body, to the response array is to type response.body = something. It makes things much easier. And that is exactly what happens in the next line: response.body is set equal to a Proc object.

Response array:

[‘200’, {‘Content-Type’ => ‘text/html’}, [“This is HTTP body”]]

Response object:

response.body = [“This is HTTP body”]
response.status = ‘200’
response.headers[‘Content-Type’] = ‘text/html’

After that, the method checks whether redirects[:location] contains a value. If it does, then ‘Location’: redirects[:location] is inserted into the response header, which will tell the client which path to automatically send a new request to when the response is received. However, on its own, the location header will not make the browser redirect. For the redirect to work, the application’s response should contain a redirect status code in conjunction with the location header. Therefore, the method sets the response.status = ‘302’.

The last part of the add_route method inserts the route information into the @routeshash. As you can see, keys in the @routeshash are arrays containing an HTTP method (e.g. ‘get’ or ‘post’) and a request path (e.g. ‘/home’). The values in the @routes hash are arrays with three elements, where the first element is a status, the second is a hash with headers, and the third is an array containing a Proc object (more on procs later).


Rationale of the #add_routes method

Since this method stores triplet arrays as keys in the @routes hash, it is reasonable to assume that these keys are what the Rack app will return to the ruby web server and, ultimately, the client.
There is a caveat here: The third element in the triplet array is an array that contains a Proc object according to the add_route method. The Rack-specification requires that applications return a triplet array whose third element is an array that contains String objects.

But I can’t just store String objects in place of the Proc objects because then I won’t be able to store executable code: I will merely be storing Strings. Further, I can’t just store the return value of the_proc_object.call, even though the return value will be a String object, because the executable code will be running at route-store-time rather than response-time.


A few notes on Procs

Proc objects are useful because they store executable code. To execute the code stored in a Proc object, you simply type name_of_proc_object.call. The Proc object will then return whatever the last line of code in the Proc returns. All blocks passed into our add_route method ought to return a String object on their last line, but the application ought not to execute the code in that Proc unless the client’s request matches the key in @routes associated with that response array that contains that Proc.

The reason for this is clearer with an example. Imagine if the app had a route for logging in and a route for deleting a user’s account:

add_route(‘post’,’/account/create’) do
# create account
# return string
end
add_route(‘post’,’/account/delete’) do
# delete user’s account
# return string
end

The example blocks above contain code that should obviously not be executed at the start of the application when add_route method calls execute. Rather, the executable code passed to the method via the do..end blocks needs to be stored and only executed if the user sends a POST request to ‘/account/create’ or ‘/account/delete.’

With adding routes out of the way, the next step to implement routing functionality is to code the logic required for accessing the client’s requested route and then to compare that to the routes stored in the hash, referenced by the @routes instance variable. After that, there are plenty of more features to implement.

This article covered a lot of material, but much of it was preliminary and related to Rack-app development in general rather than the construction of a particular framework or application. The next article in this series will be more coding-oriented than concept-oriented. It will detail a large part of the framework implementation, as well as the reasoning process behind it.