Unleashing the Power of Portainer: Comprehensive Guide to Monitoring AWS ECS EC2 Cluster with Ease
Monitoring AWS ECS EC2 Cluster with Portainer
I have been working on AWS ECS to manage Docker containers for production. Sometimes, I need to check the container logs when there is an issue. AWS has a service called CloudWatch to monitor logs of ECS containers but it comes with price.
Since AWS ECS is built on top of the Docker engine, I have an idea to use Portainer as a monitoring tool. Portainer is my favorite management and monitoring tool for Docker. Portainer has many excellent features and I much prefer to use it over AWS CloudWatch while saving money.
The infrastructure of the ECS cluster
All ECS EC2 instances are placed on the private subnet. Each ECS EC2 instance has a private IP and only can be accessed from the Bastion instance on the public subnet.
Deploy Portainer Agent as daemon services
First, we need to create a task definition for Portainer Agent. Create a JSON file named portainer-agent-td.json and copy-paste these codes.
We will use AWS CLI to create the task definition.
aws ecs register-task-definition --cli-input-json file:/portainer-agent-td.jsonOpen your ECS cluster on the AWS console and create a new service.
Set the launch type to EC2.
Set the service type as a daemon service and use the task definition that we created.
Hit the create button to finish.
Now Portainer Agent is deployed as a daemon. Every EC2 instance in the ECS cluster will have a Portainer Agent container.
Install Portainer on Bastion Server
Now we need to have Portainer on the bastion server to monitor all Portainer Agents in the ECS cluster. We will install Portainer via Docker. Let’s SSH into the bastion server and run this command.
docker volume create portainer_data
docker run -d -p 9000:9000 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:latestLogin to Portainer on the port 9000 via a web browser. Go to Environment > Add environment.
Choose Docker Standalone and click the Start Wizard button.
Choose Agent as Docker Standalone Environment.
- You can ignore the Portainer Agent installation instruction because we already deploy the Portainer Agent as daemon services previously.
- You can set the Name of your new environment to whatever you want. I prefer to include the private IP of the EC2 instance in this format :
ec2-x.x.x.50. - You need to set the Environment addresses with the private IP or the hostname of the EC instance followed by port
:9001.
Add the environment for every EC2 instance of the ECS cluster.
Congratulation! Now we can monitor all containers in the ECS cluster. 👏
