Building resilient Amazon OpenSearch cluster with AWS CDK (part 3)

Configure Access to OpenSearch dashboards from outside of a VPC

If we deploy our stack now, create a user in the Cognito user pool we’ve created, let’s say “admin”, put it to the admin group, and try open Kibana, we will find out that it does not work. And it is expected because our cluster belongs to VPC and private subnet, and is not accessible from outside. Above, we already briefly explained the idea of how we want to access our cluster, so let’s jump into implementation.

We will create an EC2 instance with Session Manager Agent installed on it, then we will install NGINX proxy on this machine and configure it to listen to a specific port and redirect all requests coming to this port to our cluster (we already mentioned before that we have two clusters, so depending on the port where the request comes to, we will redirect to a specific cluster, but in the example, we will provide configuration for one cluster only).

Let’s look at what our NGINX config looks like:

A few things should be mentioned here. We will use SSL to connect to our NGINX instance. For that, it is recommended to obtain an SSL certificate from a certificate authority (CA) to configure SSL for NGINX. But for non-production environments, you can generate a self-signed certificate instead. Note that it’s a best practice to use only SSL certificates signed by a third-party certificate authority in your production environment. You can generate a self-signed certificate with this command:

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /<path where to store>/cert.key -out /<path where to store>/cert.crt

Now we can jump to the code for EC2 instance:

A small note regarding the Cognito endpoint here — you may notice that we are constructing it by ourselves. Another way you can get it is to use the “userPoolDomain.domainName” property, but it returns an endpoint name with protocol “http”. If you use it, do not forget to update the NGINX config and remove the protocol from the rules.

So now our proxy instance is ready for use. All we need is to establish an SSM session from our local machine to the EC2 instance. For that, we have to know the instance identifier, but it is not a problem because we know the machine name, so we can easily get it through AWS CLI. Here is an example of a batch file you can use for that:

Let’s assume we have the AWS CLI default profile properly set up and the batch file name is connect.bat, so the command will look like this:

connect.bat default 2000 us-east-1

And voila, we are on the Cognito login page…

A few things for consideration

You may notice that we must pass a region identifier for connection. As we discussed at the beginning, we are going to deploy a passive cluster to another region to establish cross-cluster replication. Besides the cluster, we also will deploy all other parts of our infrastructure. Because of that, you should know the region of the EC2 instance you want to connect to. And at that point, you can have some problems. The first problem relates to the resource identifiers we are using in our code. As you know, not all AWS resources are regional, some of them are global, so if you try to deploy the provided code in the second region, you will get an error that a resource with the same identifier already exists. To overcome this problem, we include the region name as a suffix to our identifiers, but with a bit of transforming (removing dashes and making it pascal case) to make it more consistent with the naming convention we follow. Depending on the convention you follow for resource identifiers, you can use one or another helper function to get the region name.

Cross-regional resource identifier now may look like this:

The second thing for consideration is closely titled with the first one — it is roles. Because AWS IAM is also a global resource, you should decide how you are going to deal with roles for different regions. You can either use the same roles between regions for stuff related to cross-cluster replication or create different roles for each region. In our solution, we decided to follow the second way. If you go the same way, you should remember that Cognito service is a regional resource. It means that users, user groups, and roles associated with them in one region are not the same as ones in another region. So, you should add corresponding mappings for OpenSearch service backend roles in another region.

Also, you can’t use the same custom endpoint name for the OpenSearch cluster in another region, so you should create a new one and adjust the NGINX configuration on the EC2 instance in the secondary region with the endpoint name you chose.

Now we can move to the next part.

We are ACTUM Digital and this piece was written by Mikhail Chumakov, Senior .NET Developer of Apollo Division. Feel free to get in touch.