NSQ service discovery with DNS records

Nsqd is typically launched with a list of IP addresses pointing to each of the nsqlookupd nodes.

docker run --rm --name nsqd \ 
-p 4150:4150 -p 4151:4151 \
nsqio/nsq /nsqd \
--broadcast-address=$HOST \
--lookupd-tcp-address=0.0.0.1:4150 \
--lookupd-tcp-address=0.0.0.2:4150

This technique works well in a traditional hosting infrastructure but poses some issues when using a cluster manager such as AWS ECS, Kubernetes or Fleet. In these highly dynamic environments we rarely know which IP address or machine new services will launch on.

Luckily the NSQ team recently added a /config endpoint which allows us to configure the nsqd node via HTTP PUT request. Now we can set the nsqlookupd IP's on the node at runtime instead of having to hardcode the IP's at launch.

There are several service discovery tools out there like Consul and etcd. However, a DNS entry can act as a light weight alternative. To leverage DNS discovery we can use the sidecar container nsqd-discovery which runs on the same server as the nsqd node.

docker run --rm --name harlow/nsqd-discovery \ 
harlow/nsqd-discovery \
--lookupd-dns-address=$NSQLOOKUPD_DNS_ADDRESS \
--config-http-address=$CONFIG_HTTP_HOST

The sidecar continually pings the DNS entry and updates nsqd when the IP addresses of the A-record change. This enables us to easily run and dynamically configure nsqd in container cluster managed environment.