EMQ X Message Server Nginx Reverse Proxy

Foreword

Many newcomers who use the EMQ X message server will consider the expansion of the EMQ X node as the business grows. When you have multiple nodes and there is no proxy before the node, you need to specify which EMQ X node to be connected by the client. If one of the nodes is upgraded, you need to reset on the client. However, if the reverse proxy is loaded on the cluster node, you can assign the connection to the other node through the reverse proxy, thus avoiding changes and redeployments on the client. Mr. EMQ is going to about Nginx reverse proxy today.

Nginx Installation

Dependent installation

· Install pcre、zlib、openssl

$ wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.42.tar.gz
$ tar -zxf pcre-8.42.tar.gz
$ cd pcre-8.42
$ ./configure
$ make
$ sudo make install

$ wget http://zlib.net/zlib-1.2.11.tar.gz
$ tar -zxf zlib-1.2.11.tar.gz
$ cd zlib-1.2.11
$ ./configure
$ make
$ sudo make install

wget -O openssl.tar.gz -c https://github.com/openssl/openssl/archive/OpenSSL_1_0_2l.tar.gz
tar zxf openssl.tar.gz
mv openssl-OpenSSL_1_0_2l/ openssl

Download and install Nginx

· Source code compile and install Nginx

$ wget https://nginx.org/download/nginx-1.14.0.tar.gz

$ tar zxf nginx-1.14.0.tar.gz
$ cd nginx-1.14.0

./configure — sbin-path=/usr/local/nginx/nginx — conf-path=/usr/local/nginx/nginx.conf — pid-path=/usr/local/nginx/nginx.pid — with-pcre=../pcre-8.42 — with-zlib=../zlib-1.2.11 — with-http_ssl_module — with-stream — with-stream_ssl_module — with-openssl=/opt/openssl

· Compile and install

make && make install

EMQ X node cluster

The node cluster of EMQ X can be found in the official documentation: https://developer.emqx.io/docs/emq/v3/en/cluster.html

Nginx TCP/SSL reverse proxy settings

TCP reverse proxy settings

· reverse proxy settings:

$ mkdir -p /usr/local/nginx/tcp.d/

$ cat <<- ‘EOF’ >> /usr/local/nginx/nginx.conf
include /usr/local/nginx/tcp.d/*.conf;
EOF

$ vim emqx_tcp_nginx.conf

stream
{
log_format proxy ‘$remote_addr [$time_local] ‘
 ‘$protocol $status $bytes_sent $bytes_received ‘
 ‘$session_time “$upstream_addr” ‘
 ‘“$upstream_bytes_sent” “$upstream_bytes_received” “$upstream_connect_time”’;

access_log /var/log/nginx/tcp-access.log proxy ;
 open_log_file_cache off;
upstream mqtt1883 {
 #zone tcp_servers 64k;
 #hash $remote_addr;
 server 192.168.1.10:1883 weight=1;
 server 192.168.1.13:1883 weight=1;
}
server {
 listen 1883;
 proxy_send_timeout 2h;
 proxy_read_timeout 2h;
 proxy_connect_timeout 150s;
 proxy_timeout 150s;
 proxy_pass mqtt1883;
 proxy_buffer_size 3M;
 tcp_nodelay on;
}
}

The client connects the address<Nginx IP:1883> , and Nginx distribute the connect to the EMQ X node. Mr. EMQ test 200 client connections, and the reader can see that 200 connections have been distributed to the 2 EMQ X nodes.

SSL reverse proxy settings

In the following configuration file, the EMQ X comes with a certificate directly used by the certificate, with the certificate directory emqx/etc/certs.

cat emqx_ssl_nginx.conf

stream{
 upstream backend{
 # hash $remote_addr consistent;
 least_conn;
 server 192.168.1.10:1883 weight=1;
 server 192.168.1.13:1883 weight=1;
 }

server {
 listen 8883 ssl;
 proxy_send_timeout 2h;
 proxy_read_timeout 2h;
 proxy_connect_timeout 150s;
 proxy_timeout 150s;
 proxy_pass backend;
 proxy_buffer_size 3M;
 ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 ssl_ciphers AES128-SHA:AES256-SHA:RC4-SHA:DES-CBC3-SHA:RC4-MD5;
 ssl_certificate /root/nginx-certs/certs/cert.pem;
 ssl_certificate_key /root/nginx-certs/certs/key.pem;
 ssl_session_cache shared:SSL:10m;
 ssl_session_timeout 10m;
 }
}

The client connects to the address <Nginx IP:8883> via SSL, and Nginx will distribute the connection to the EMQ X node in TCP mode.

Summary

The above configuration briefly introduces the configuration of Nginx in the TCP/SSL reverse proxy. In the actual working environment, the Nginx configuration parameters can be adjusted. Readers can do the test by reference to this article and the documentation from Nginx official website, enjoy it.