Nginx for surviving in microservices era

The typical nginx configuration

In the microservices ecosystem the network is the base upon you build your application. Managing it with the right tool could help you.

nginx is a great tool but, sometimes, its configuration can be tricky: the copy-paste approach is an evil friend making the refactoring not simple. In this article we will introduce some concepts to help you with your network configuration. We’ll make an overview how your network can be managed to dispatch the requests to the right backend.

Nginx is a great tool

From nginx website:

nginx [engine x] is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server,

In other words: nginx receives all your requests and forwards them to the right backend. But why should we have to bring an application behind another network layer?

There’re lots of benefits. Only for making some examples:

• resolves SSL
• makes HTTP compression as gzip or deflate
• adds/removes headers
• is single entry point to dispatch requests to the right place
• hides your real application backend

nginx is a very good tool, but it has a lot of feature: we are going to cover a little piece of the whole its world to see how the growing complexity can be managed easier. So, let’s start from the beginning!

Set up

In this article we’ll use docker containers for our examples to keep the setup independent from the underlying operative system. In particular, docker-compose is very useful to simulate a production environment containing more than one application backends.

We are going to write two files: docker-compose.yml and my-server.conf. The former contains the docker compose services definition; the latter will be used to study and play with nginx configuration. For simplicity, keep those files in the same directory.

First of all, install docker and docker-compose. Then we create a new file docker-compose.yml with the following content:

version: '3'
services:
  reverse-proxy:
    image: nginx
    ports:
      - "8888:80"
    volumes:
      - ./my-server.conf:/etc/nginx/conf.d/default.conf:ro
  be1:
    image: jmalloc/echo-server
    environment:
    - PORT=80
  be2:
    image: tutum/hello-world
  be3:
    image: breerly/hello-server
    environment:
    - HELLO_PORT=80
    - HELLO_RESPONSE_DELAY=1

This file describes four services: a reverse proxy implemented by nginx and three application backends for simulating real backends. The reverse proxy configuration describe that we’ll use the 8888 port on localhost to make requests to it. We are mounting the volumes to change the nginx configuration easily.

The configuration for the application backends is not important: be1, be2, be3 services listen on 80 port. In this way we can “forget the ports” in nginx configuration during the service definitions because the 80 is the HTTP standard port.

Now we are ready to play with nginx configuration! First of all, our configuration is kept as simple as possible to add complexity little by little.

So, let’s start creating the simplest possible configuration: in this first iteration, nginx will always respond directly with “200 OK” to every request, without forwarding the request to an application backend.

Create another file in the same directory called my-server.conf with the following content:

server {
  listen 80;
  
  default_type text/plain;
  
  location / {
    return 200 'OK!';
  }
}

To use our configuration with nginx, we’ll use the following command in a terminal:

docker-compose up

Our configuration servers “OK!” at http://localhost:8888. In the terminal, the previous command has an output like:

reverse-proxy_1  | 172.21.0.1 - - [19/Dec/2017:22:41:10 +0000] "GET / HTTP/1.1" 200 3 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36" "-"

This output proves that the service called reverse-proxy numbered 1 has received and served the request returning 200 status code. Well done!

To stop docker compose, please send it a SIGINT signal using the ctrl+C.

All requests have the same response for all HTTP methods. This is not a big deal but is a good baseline to explore and to understand nginx better.

Your first proxy to an application backend

In the microservices era your application is made by one or more backends. For starting easier, we’ll consider only one service.

We simulate it using the docker image jmalloc/echo-server. In our docker compose we have already declared that the be1 uses that image for running.

So, let’s overwrite the previous nginx configuration with the following:

upstream service1 { server be1; }
server {
  listen 80;
  location /my-service1 {
    proxy_pass http://service1;
  }
}

This configuration tells to nginx that all requests that have /my-service1 as path prefix will be proxied to the upstream service1. For our purpose we can consider upstreams as aliases for the given hostname. Thanks to docker compose the be1 hostname is resolved pointing to the “jmalloc/echo-service” container.

So, CURLing http://localhost:8888/my-service1, the request reaches nginx that sees the path matches the location block and, though the service1 upstream, proxies it to the be1 “aka” jmalloc/echo-service image.

In the output in your terminal there’re lines like those:

be1_1 | 172.21.0.5:48330 | GET /my-service1
reverse-proxy_1 | 172.21.0.1 — — [28/Dec/2017:19:29:02 +0000] “GET /my-service1 HTTP/1.1” 200 538 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36” “-”

As you can see, the request reached the be1 service numbered 1 through the reverse-proxy numbered 1.

Add a lot of services

Your webserver may be built using a lot of application backends. So, we suppose three application backends (be1, be2, be3). We are already defined them in our docker-compose-yml. nginx proxies the requests based on the first part of the url.

upstream service1 { server be1; }
upstream service2 { server be2; }
upstream service3 { server be3; }
server {
  listen 80;
  default_type text/plain;
  location /my-service1 {
    proxy_pass http://service1;
  }
  location /my-service2 {
    proxy_pass http://service2;
  }
  location /my-service3 {
    proxy_pass http://service3;
  }
}

In this configuration, we have described three locations to handle the three different urls, one for each application backend.

Now you can request http://localhost:8888/my-service1, http://localhost:8888/my-service2 and http://localhost:8888/my-service3 and retrieve right responses.

But… how can we manage a lot of microservices? Or how can we write the configuration better avoiding listing all locations?

Simplify the configuration

Our goal is to keep the network configuration as simple as possible and in only one place. In the previous configuration we have three location blocks with three different proxy_pass definitions. We can do better!

nginx defines some default variable like $uri, $request_method and so on. We can use them to map the request to the upstream name.

We are going to use map. From nginx documentation:

Creates a new variable whose value depends on values of one or more of the source variables specified in the first parameter.

The following configuration defines a map called $upstreamName. For each request, the map defines the $upstreamName variable using the url path mapping the incoming request url to upstream name.

upstream service1 { server be1; }
upstream service2 { server be2; }
upstream service3 { server be3; }
map $uri $upstreamName {
  "~^/my-service1" service1;
  "~^/my-service2" service2;
  "~^/my-service3" service3;
}
server {
  listen 80;
  server_name localhost;
  default_type text/plain;
  location / {
    proxy_pass http://$upstreamName;
  }
}

$upstreamName map is valorized according to the map:

• GET /my-service1: $upstreamName is set to service1
• DELETE /my-service2: $upstreamName is set to service2
• GET /my-service3/foo/bar: $upstreamName is set to service3
• GET /my-service15: $upstreamName is set to service1

When a request arrives, the nginx finds the right location, set the $upstreamName variable according to the regexes above and proxies the request to the right upstream.

Now we have only one place to write out configuration! With this configuration the output is the same.

We can complicate the $upstreamName definition using other predefined variables to send the requests to the right backend.

map $request_method-$uri $upstreamName {
  "~^(GET|POST|PUT|DELETE)-/my-service1" service1;
  "~^POST-/my-service2" service2;
  "~^(GET|DELETE)-/my-service3" service3;
}

The above example builds a map using the HTTP method and the request url.

• The GET, POST, PUT and DELETE requests that have the url path that starts with /myservice1 will be proxied to service1 aka be1.
• Only the POST requests that have the url path that starts with /my-service2 will be proxies to service2 aka be2.
• The GET and DELETE requests that have the url path that start with /myservice3 will be proxied to service3 aka be3

The following CULRs work well:

curl -X GET http://localhost:8888/my-service1
curl -X POST http://localhost:8888/my-service1
curl -X PUT http://localhost:8888/my-service1
curl -X DELETE http://localhost:8888/my-service1
curl -X POST http://localhost:8888/my-service2
curl -X GET http://localhost:8888/my-service3
curl -X DELETE http://localhost:8888/my-service3

Instead, the following ones don’t:

curl -X CUSTOM_METHOD http://localhost:8888/my-service1
curl -X GET http://localhost:8888/my-service2
curl -X PUT http://localhost:8888/my-service2
curl -X DELETE http://localhost:8888/my-service2
curl -X POST http://localhost:8888/my-service3
curl -X PUT http://localhost:8888/my-service3
curl -X GET http://localhost:8888/unknown-service

This last block of CURLs returns always 500 response code. It is not nice: another HTTP status can be used. The following chapter explains how the different status code can be managed.

Add HTTP server utilities

nginx allows us to define multiple servers in the same configuration file. Previously our server listens on 80 TCP port. Another way to create an HTTP server is to bind a unix socket. This allows us to speed the TCP connection up but with the drawback that the connections can be made only from localhost.

Let’s define a simple nginx server that responses always 404 Not found page.

server {
  listen unix:/var/run/not-found.sock;
  return 404;
}

We can use it to proxy all requests that not matching any map rules. Update the nginx configuration as following:

server {
  listen unix:/var/run/not-found.sock;
  return 404;
}
upstream not_found { server unix:/var/run/not-found.sock; }
upstream service1 { server be1; }
upstream service2 { server be2; }
upstream service3 { server be3; }
map $request_method-$uri $upstreamName {
  default not_found;
  "~^(GET|POST|PUT|DELETE)-/my-service1" service1;
  "~^POST-/my-service2" service2;
  "~^(GET|DELETE)-/my-service3" service3;
}
server {
  listen 80;
  server_name localhost;
  default_type text/plain;
  location / {
    proxy_pass http://$upstreamName;
  }
}

The above configuration defines the not_found upstream used as default value of $upstreamName map using the default keyword. If an incoming request doesn’t match any rules, the map will assign the default value not_found to $upstreamName variable.

TIP: the not_found server can be customized serving your HTML page.

With the previous configuration, a GET to /my-service2 path returns a 404 Not found. In REST the right status code is 405 Method not allowed. So, let’s add another server that responses always that status code and use it in the $upstreamName map definition:

server {
  listen unix:/var/run/not-found.sock;
  return 404;
}
server {
  listen unix:/var/run/not-allowed.sock;
  return 405;
}
upstream not_found { server unix:/var/run/not-found.sock; }
upstream not_allowed { server unix:/var/run/not-allowed.sock; }
upstream service1 { server be1; }
upstream service2 { server be2; }
upstream service3 { server be3; }
map $request_method-$uri $upstreamName {
  default not_found;
  "~^(GET|POST|PUT|DELETE)-/my-service1" service1;
  "~^\w+-/my-service1" not_allowed;
  "~^POST-/my-service2" service2;
  "~^\w+-/my-service2" not_allowed;
  "~^(GET|DELETE)-/my-service3" service3;
  "~^\w+-/my-service3" not_allowed;
}
server {
  listen 80;
  server_name localhost;
  default_type text/plain;
  location / {
    proxy_pass http://$upstreamName;
  }
}

The not_found and not_allowed upstreams are used for all not matching requests or for not allowed methods respectively.

Handle secreted API

Continuing with the map usage, we are able to manage the API secrets automatically.

As we have defined not-found and not-allowed servers, we can define an unauthorized HTTP server too.

server {
  listen unix:/var/run/unauthorized.sock;
  return 401;
}

The above server always responses 401 Unauthorized status code.

Assuming that our APIs are under authorization: a secret has to be provided to access to the backend. For instance, my-secret and another-secret are the only right value of the secret HTTP header to access the backend.

To check the secret correctness, we can build a map called $secretResolution defined as below:

map $http_secret $secretResolution {
  default n;
  "my-secret" y;
  "another-secret" y;
}

nginx defines the $http_* variables with the right HTTP header. In our case the $http_secret variable is valorized with the value of the “secret” HTTP header. For each request, $secretResolution is y if the request has the right value in the HTTP secret header, n otherwise.

We change the $upstreamName map to include the new $secretResolution variable. Adding the right upstreams, we are able to list our network dispatcher. Use the following configuration:

server {
  listen unix:/var/run/not-found.sock;
  return 404;
}
server {
  listen unix:/var/run/not-allowed.sock;
  return 405;
}
server {
  listen unix:/var/run/unauthorized.sock;
  return 401;
}
upstream not_found { server unix:/var/run/not-found.sock; }
upstream not_allowed { server unix:/var/run/not-allowed.sock; }
upstream unauthorized { server unix:/var/run/unauthorized.sock; }
upstream service1 { server be1; }
upstream service2 { server be2; }
upstream service3 { server be3; }
map $http_secret $secretResolution {
  default n;
  my-secret y;
  another-secret y;
}
map $secretResolution-$request_method-$uri $upstreamName {
  default not_found;
  "~^n" unauthorized;
  "~^y-(GET|POST|PUT|DELETE)-/my-service1" service1;
  "~^y-\w+-/my-service1" not_allowed;
  "~^y-POST-/my-service2" service2;
  "~^y-\w+-/my-service2" not_allowed;
  "~^y-(GET|DELETE)-/my-service3" service3;
  "~^y-\w+-/my-service3" not_allowed;
}
server {
  listen 80;
  server_name localhost;
  default_type text/plain;
  location / {
    proxy_pass http://$upstreamName;
  }
}

The $upstreamName map now includes the new $secretResolution variable too. Only if the secret, HTTP method and the first path of url match, the request is proxied to the right backend.

The following CURLs work well:

curl -X GET -H 'secret: my-secret' http://localhost:8888/my-service1
curl -X GET -H 'secret: another-secret' http://localhost:8888/my-service1
curl -X GET -H 'secret: my-secret' http://localhost:8888/my-service1/foo/bar

Instead the following ones return 401:

curl -X GET http://localhost:8888/my-service1
curl -X GET -H 'secret: ' http://localhost:8888/my-service1
curl -X GET -H 'secret: unknown-secret' http://localhost:8888/my-service1

Conclusion

nginx is a great tool. Writing nginx configuration is not easy. map is a good feature. Be wise!

If you liked this article, please clap it!

I will share other nginx tips, so stay tuned!

For more infos:

• http://nginx.org/en/docs/http/ngx_http_core_module.html
• http://nginx.org/en/docs/http/ngx_http_upstream_module.html
• http://nginx.org/en/docs/http/ngx_http_map_module.html

A special thanks go to Luciano Mammino for helping me to write this article better.