Selenium Grid Infrastructure by using Rancher, Packer and Docker on AWS

Hi all, in this article, I will explain to you how to setup Selenium Grid with Rancher on AWS. First we need to create an AWS account, for now, we may use t2.micro instances in US Regions which are the free-tier option enabled for new users in the first year of membership. These light resources can handle our demonstration load.

Objectives on AWS Cloud

• Create AWS EC2 instance running Ubuntu 16.04.
• Ensure EC2 security group has port 8080, 22(in case if you need ssh), 2376, 450, 4500 open.
• Install Docker.
• Add Ubuntu user to docker group sudo usermod –append –groups docker ubuntu.
• Pull Rancher Server container image from Docker Store and run at startup.

Identity Access Management (IAM)

We need to create Group to define policies like users to have EC2FullAccess rights. So Rancher can manage EC2 resources.

We need to create User for programmatic access: Management Console > IAM > Create User & choose access type programmatic access and add a user to Group that we created above.

Programmatic Access

Enables an access key ID and secret access key for the AWS API, CLI, SDK, and other development tools. For us, it’s used for Packer and Rancher.

Elastic Computing (EC2)

We need to create a security group and open ports below for inbound access to EC2 machines.

UDP 450, TCP 2376, UDP 4500, TCP 8080

Amazon Machine Images (AMI) Creation for Rancher Server

I’d like to use Packer for creation desired ubuntu machine image in AWS. You may want to check more detailed Packer Article from here. You should install Packer first and add packer.exe to Environment Variable PATH.

https://releases.hashicorp.com/packer/1.2.5/packer_1.2.5_windows_amd64.zip

Objectives

• Provision Ubuntu image from AWS.
• Installs compatible docker version on Ubuntu image.
• Pull rancher server container image and run.
• Pack AMI image and ready on the created region.

Packer JSON File:

provision.sh file

Packer validate packer.json

Packer example run command;

packer build -var ‘aws_access_key=AKIAJOLN6HYMWIDFNZOA’ -var ‘aws_secret_key=WJBen0fMGxmiKlks2q9CrQQ4C/Nhacy55YlAC4nh’ packer.json

Output

Now head over to EC2 → AMIs tab, find your rancher-master 1529683817 image and launch. Select Security Group that we created before and click Review and Launch.

Go to Instances tab and wait for the instance to be ready. The EC2 machine will boot up and run rancher server container on port 8080. This can approximately take 3–5 minutes.

Click instance id to review status.

Let’s check docker container on the host;

Copy IPv4 public IP Address and hit url http://< ${Ipv4 public IP Address}> :8080

Add a Host

Host machines will cover your infrastructure resources; containers will be running on these hosts as automatically distributed.

I selected host quantity as 3 and it’s going to provision 3 Ubuntu t2.micro 16GB disk mounted AMIs on EC2.

We are ready to create Environment and Stack;

Then we can add our services for creating SeleniumGrid Stack;

• 1 container for Selenium Hub
• 1 container for Chrome Node
• 1 container for Firefox Node

Wait for the stack to be ready and active.

Selenium Grid Container Information;

Go to Selenium Grid Dashboard;

As you can see Selenium Hub is attached with 1 Chrome and 1 Firefox WebDriver instances.

If you want to scale out services of WebDrivers increase Scale count; I did for both nodes.

Updated Dashboard view after scaleout:

Also, you can upload below .yml files to create Stack directly in Add Stack page. So you won’t need these manual steps anymore.

Docker-compose.yml

This is the end of the article and I hope you enjoyed to read and create a Selenium Grid architecture by using Rancher on AWS. If you have any questions, please write a comment and I will help you when I see them.