Tic Tac Consul — Multi-Cloud, Serverless Networking
Tic Tac Consul pits serverless functions from GCP, AWS, and Azure in an epic match of Tic Tac Toe! No, but seriously, Tic Tac Consul is focused on exploring the challenges of networking across environments and integrating serverless functions with the rest of your application stack.
As more and more workloads move to the cloud, many organizations are faced with the challenges of mapping their traditional skill sets and on-premise data center resources to the constructs of one cloud provider. But the reality is that most organizations already are or will leverage multiple cloud providers to take advantage of best-of-breed solutions.
So how does this all map back to serverless functions playing Tic Tac Toe with each other? Well, I pose a question for you, how would you handle network connectivity between multiple environments securely while allowing any application, whether it’s deployed on a VM, in K8s, on a mainframe, or even a serverless function, to find and connect to each other?
It turns out it’s rather challenging, and that’s what we’re going to explore today.
Consul Overview
I want to start with a quick overview of what Consul is and a general idea of why you would utilize it. At a fundamental level, Consul is an agent-based solution for registering services and routing requests between services. As services come online, they register themselves with Consul, and agents perform localized health checks to track healthy and unhealthy applications.
These agents allow Consul to load balance requests across services, define which applications are allowed to talk to each other with intentions, and securely proxy requests between environments.
Connecting Environments
So before we deep dive into Tic-Tac-Consul, let’s go through a brief overview of the common ways that Cloud Providers currently connect environments between each other and traditional on-premise environments.
VPN
VPNs are probably the simplest way of connecting two different environments across the public internet. At their most basic level, VPNs are responsible for encrypting your traffic as it travels over the public internet to a connected environment. You’ve likely heard ads for VPNs that provide an exit node in another country and encrypt your traffic so that it can’t be “immediately” tracked.
All the major cloud providers have a VPN offering for connecting your local environment with their VPCs and Virtual Networks in a secure way. VPNs provide a cheap and easy way to begin securing your traffic between two environments. Still, they don’t offer any guarantees on latency as you’re at the mercy of your ISP and how quickly your requests can get routed to the cloud provider’s network.
Examples: GCP Cloud VPN, AWS VPN, and Azure VPN Gateway
Peering
Peering isn’t entirely different from a VPN but rather a way to provide a shorter route for your requests to go from a local environment to a remote environment. With two networks peered together, you might still be communicating over the public internet, but at least you have a defined route from your Datacenter to the cloud provider.
A VPN would still be responsible for encrypting your traffic, but peering provides higher availability and lower latency via the dedicated route mentioned above. The tradeoff, of course, happens to be the cost in comparison to a traditional VPN.
Examples: GCP Direct/Carrier Peering, Azure Peering Service
Interconnect
Aside from running all of your workloads directly in a cloud provider’s environment, interconnects are the lowest latency and highest availability that you’re going to achieve. Interconnects provide a direct, physical route from your environment to the cloud provider’s network. Interconnects remove unnecessary network hops for your traffic and send traffic over your private interconnect instead of the public internet.
It’s been interesting to watch this industry pop up, and we’re even starting to see interconnects between cloud providers from companies like Equinix. All that being said, interconnects are far from the cheapest option available. However, if you’re relying on this connection for mission-critical services or need the absolute lowest latency possible, interconnects are necessary.
Examples: GCP Direct Connect, AWS Direct Connect, Azure ExpressRoute, Equinix
Serverless Function Challenges
All right, so we have plenty of options for handling network connectivity. The question becomes, how do we find services, load balance across them, handle multi-dc failover, and quite frankly integrate more complex environments like serverless functions, K8s, and the OG serverless functions, mainframes.
For this discussion, I will focus on the challenges that Serverless Functions face. The solutions we explore will provide us with a framework that can be used across any environment.
Dynamic IP Addresses
Dynamic IP Addresses aren’t unique to serverless functions; in fact, Kubernetes embraces this concept by dynamically assigning IP addresses to pods every time they come up. And, I’m sure some of you who have used serverless functions are probably wondering why I’m even talking about IP Addresses. Most serverless functions are exposed with a DNS entry that can be utilized across the public internet.
What if we wanted to do this only over private networks? Easy enough, we can peer two networks together and be done with it. What about connecting that function across a private network to another cloud and making it accessible via DNS?
Authorization
Authorization is typically accomplished utilizing the cloud provider’s native IAM permissions. Once again, this is great if you are operating solely in a single environment, but using a GCP Service Account in an AWS Lambda function or managing API Keys so that your on-premise applications can talk to an Azure function can quickly become a bit cumbersome.
Firewalls
Firewalls aren’t likely to apply to everyone, but high-security environments are likely to have firewalls surrounding their perimeter that control not only ingress traffic but also egress traffic. Now you have these serverless functions that are liable to be running on any number of machines needing to be called and talk outbound across environments. This results in firewall rules that need to be kept up to date continually.
Architecture
Mesh Gateways
At this point, you’re probably nodding in agreement that multi-cloud and multi-environment connectivity is challenging. This is not exactly a new idea. Consul combines two critical components that help us solve these challenges.
Mesh Gateways are a relatively new feature of Consul that encrypts traffic and proxy information between environments. Combining this with Consul’s built-in DNS server, ability to perform distributed health checks, and load balancing, we have something that is a bit like VPN+. The best part is that all of this works across the public internet, with or without a VPN, Peering, or even a dedicated Interconnect.
Here is where things get even more interesting. As I mentioned above, Consul is an agent-based architecture, so how can we integrate a Service Mesh with serverless functions and other managed services that we don’t control?
Terminating and Ingress Gateways
Terminating and Ingress Gateways are endpoints deployed in various environments that allow traffic to exit and enter the Consul service mesh. In the architecture above, you can see how Tic Tac Consul deploys a terminating and ingress gateway within AWS, Azure, and GCP networks peered with a serverless function deployed in each environment.
Combining this with the Mesh Gateways that we talked about above, we’re able to securely transit requests across environments while also using Consul’s global DNS to call these functions from any environment.
We gain the other benefit that mesh, terminating, and ingress gateways all provide relatively static ingress/egress points for our traffic. This means that traditional firewalls only have to worry about our gateways. Also, leveraging Consul Terraform Sync would allow you to easily automate those firewalls and load balancers based on services that get registered with Consul.
Consul Intentions
If you remember, we have one final challenge: authorization between functions and, more specifically, across environments. Intentions allow us to easily create “allow” and “deny” policies between services. In Tic Tac Consul’s case, we could easily allow traffic from our GCP function to our AWS function, but not in return.
Compared to managing API keys or keeping IAM credentials synchronized across Cloud providers, intentions provide a much simpler mode of authorizing traffic.
Conclusion
We covered many different topics in this post, including connecting multiple environments securely, some of the challenges with serverless functions, and ultimately what Consul does. Multi-cloud is not easy, and frankly, something that most organizations might never find valuable. However, in the same vein, most organizations have some existing infrastructure, presenting all the same challenges if not more as they start to connect multiple cloud environments.
I hope this was valuable, and you can find a link to the GitHub repo for Tic Tac Consul below if you’d like to try deploying this yourself. Drop a comment below or send me a message on Twitter, and I’ll gladly offer any help that I can.
