How to send Linux & Windows logs to CloudWatch Logs --EC2 Mastery

7 min readJul 19, 2024

In today’s world, centralized logging is crucial for monitoring, troubleshooting, and maintaining the health of your systems.
AWS CloudWatch Logs provides a great solution to managing and analyzing logs from various sources. This article will teach you how to send Linux & Windows logs to CWL.

Scenario

In order to get the “green light” to test run an application for a client, the security team needed to be aware of anything fishy with our environment. We devised the plan to send all of the security, application & system logs to CWL in order to give them full insight into what’s going on.

Aside from the situation above, there are many reasons to send your OS Logs to a centralized authority, these may include:

Security Monitoring & Incidence Response.
Compliance & Auditing.
Troubleshooting & Debugging.
Performance Monitoring & Optimization.
Operational Monitoring.
Development & Testing.

Prerequisites

Before we dive into the setup, ensure you have the following:

An AWS account.
Necessary IAM roles and permissions for CloudWatch Logs.
AWS CLI installed and configured on your systems.

Configure IAM Role for EC2 Instance

The first thing we have to do is create an IAM Role which will allow us to send our OS logs to CloudWatch Logs.

To create the IAM role necessary for each server to run the CloudWatch agent

Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
In the navigation pane, choose Roles and then choose Create role.
Under Select type of trusted entity, choose AWS service.
Immediately under Common use cases, choose EC2,and then choose Next: Permissions.
In the list of policies, select the check box next to CloudWatchAgentServerPolicy. If necessary, use the search box to find the policy.
For Role name, enter a name for your new role, such as CloudWatchAgentServerRole or another name that you prefer.
(Optional) For Role description, enter a description.
Confirm that CloudWatchAgentServerPolicy and optionally AmazonSSMManagedInstanceCore appear next to Policies.
(Optional) Add one or more tag-key value pairs to organize, track, or control access for this role.
Choose Create role.

The role is now created.

Add the role to EC2 Instances

Head to EC2 Console https://us-east-1.console.aws.amazon.com/ec2/
Click on the Instance you want to add this role to.
Actions…Security…Modify IAM Role
Choose the IAM Role we just created.
Click Update IAM Role.

Sending Linux Logs to CloudWatch Logs

Step 1: Install the CloudWatch Logs Agent

check out this page for a more detailed look at installing CWL agent to different flavors of linux/windows. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-cloudwatch-agent-commandline.html

For Amazon Linux:

sudo yum install -y amazon-cloudwatch-agent

Step 2: Run the CloudWatch Agent Wizard

Execute this command & the wizard will ask you a series of questions.

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

NOTE: For the sake of keeping this tutorial simple, I’m adding the /var/log/dnf.log file to the wizard.

I’m adding the actual text from the Linux CLI so you’re aware of what the wizard will ask you.

[ec2-user@ip-172-31-84-185 aws]$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard 
================================================================
= Welcome to the Amazon CloudWatch Agent Configuration Manager =
=                                                              =
= CloudWatch Agent allows you to collect metrics and logs from =
= your host and send them to CloudWatch. Additional CloudWatch =
= charges may apply.                                           =
================================================================
On which OS are you planning to use the agent?
1. linux
2. windows
3. darwin
default choice: [1]:
1
Trying to fetch the default region based on ec2 metadata...
I! imds retry client will retry 1 timesAre you using EC2 or On-Premises hosts?
1. EC2
2. On-Premises
default choice: [1]:
1
Which user are you planning to run the agent?
1. cwagent
2. root
3. others
default choice: [1]:
1
Do you want to turn on StatsD daemon?
1. yes
2. no
default choice: [1]:
2
Do you want to monitor metrics from CollectD? WARNING: CollectD must be installed or the Agent will fail to start
1. yes
2. no
default choice: [1]:
2
Do you want to monitor any host metrics? e.g. CPU, memory, etc.
1. yes
2. no
default choice: [1]:
2
Do you have any existing CloudWatch Log Agent (http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html) configuration file to import for migration?
1. yes
2. no
default choice: [2]:
2
Do you want to monitor any log files?
1. yes
2. no
default choice: [1]:
1
Log file path:
/var/log/dnf.log
Log group name:
default choice: [dnf.log]

Log group class:
1. STANDARD
2. INFREQUENT_ACCESS
default choice: [1]:
1
Log stream name:
default choice: [{instance_id}]

Log Group Retention in days
1. -1
2. 1
3. 3
4. 5
5. 7
6. 14
7. 30
8. 60
9. 90
10. 120
11. 150
12. 180
13. 365
14. 400
15. 545
16. 731
17. 1096
18. 1827
19. 2192
20. 2557
21. 2922
22. 3288
23. 3653
default choice: [1]:
5
Do you want to specify any additional log files to monitor?
1. yes
2. no
default choice: [1]:
2
Do you want the CloudWatch agent to also retrieve X-ray traces?
1. yes
2. no
default choice: [1]:
2
Existing config JSON identified and copied to:  /opt/aws/amazon-cloudwatch-agent/etc/backup-configs
Saved config file to /opt/aws/amazon-cloudwatch-agent/bin/config.json successfully.
Current config as follows:
{
        "agent": {
                "run_as_user": "cwagent"
        },
        "logs": {
                "logs_collected": {
                        "files": {
                                "collect_list": [
                                        {
                                                "file_path": "/var/log/dnf.log",
                                                "log_group_class": "STANDARD",
                                                "log_group_name": "dnf.log",
                                                "log_stream_name": "{instance_id}",
                                                "retention_in_days": 7
                                        }
                                ]
                        }
                }
        }
}
Please check the above content of the config.
The config file is also located at /opt/aws/amazon-cloudwatch-agent/bin/config.json.
Edit it manually if needed.
Do you want to store the config in the SSM parameter store?
1. yes
2. no
default choice: [1]:
2
Program exits now.

Example of Linux Logs:

System Logs:

/var/log/syslog: General system log.
/var/log/messages: General message and system-related information.
/var/log/secure: Security-related messages.
/var/log/auth.log: Authentication logs.
/var/log/boot.log: Boot-related messages.
/var/log/kern.log: Kernel logs.
/var/log/cron: Cron job logs.
/var/log/dmesg: Boot and hardware-related logs.

Application Logs:

Logs generated by specific applications, such as Apache (/var/log/apache2/access.log, /var/log/apache2/error.log), Nginx (/var/log/nginx/access.log, /var/log/nginx/error.log), MySQL (/var/log/mysql.log), etc.
Custom application logs located in various directories, often under /var/log/.

Service Logs:

Logs for specific services managed by systemd, available via journalctl.
Example: journalctl -u sshd for SSH service logs.

Custom Logs:

Any custom logs created by scripts or other processes.
Located anywhere in the file system, typically /var/log/custom/ or specified directories.

If you want more information on customizing the config.json manually, here’s a link: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html

Step 3: Start and Enable the CloudWatch Logs Agent

Begin sending your logs to CloudWatch Logs with this command:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

Congratulations, you’re now

Sending Windows Logs to CloudWatch Logs

Step 1: Create a Directory on your Desktop (or anywhere else)

I created a new folder on my Desktop called ‘AWSCWLAgent’

This is simply a place to download the package to.

Step 2: Download & Install the CloudWatch Logs Agent

To down load the pack we will run the command:

Invoke-WebRequest https://s3.amazonaws.com/amazoncloudwatch-agent/windows/amd64/latest/amazon-cloudwatch-agent.msi -OutFile ‘C:\Users\Administrator\Desktop\AWSCWLAgent\amazon-cloudwatch-agent.msi’

For the -OutFile part, you need to use the path to where you created the folder in the previous step. In my case I created it on the Administrator’s Desktop.

Next Install the program in the file we created.

Simply open the file & double click the program & hit install.

This will install the CloudWatchAgent on your system.

You’ll be able to find various different files here:

C:\Program Files\Amazon\AmazonCloudWatchAgent\

Step 3: Configure the CloudWatch Logs Agent Wizard

Next we will begin the wizard, which will take you through a series of questions like the one up above in the Linux section.

Open your file explorer & go to the C:\Program Files\Amazon\AmazonCloudWatchAgent

Double click the amazon-cloudwatch-agent-config-wizard

I tracked these logs:

C:\Windows\System32\winevt\Logs\System.evtx

C:\Windows\System32\winevt\Logs\Security.evtx

Once have entered in the logs you want your CloudWatch Agent to log, hit enter to exit.

This will create a config.json file which you can further edit to your liking.

Example of Windows Logs:

Windows Event Logs:

Application Log: Events logged by applications or programs.
Security Log: Security-related events, such as login attempts and resource access.
System Log: Events logged by Windows system components.
Setup Log: Events related to application setup.
Forwarded Events: Events collected from remote computers.

IIS Logs:

Web server logs for Internet Information Services.
Default location: C:\inetpub\logs\LogFiles.

Custom Application Logs:

Logs created by custom applications or services.
Could be in various locations, often under the application’s installation directory.

Service Logs:

Logs related to specific Windows services.
Example: SQL Server logs, often found in C:\Program Files\Microsoft SQL Server\....

Step 4: Start the CloudWatch Logs Agent

Apply the configuration and start the agent:

Open a PowerShell tab & move to the following directory:

cd 'C:\Program Files\Amazon\AmazonCloudWatchAgent'

We will then run the script to begin sending the logs to your CloudWatch Logs:

.\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -c file:config.json -s

Step 5: Verify Log Data in CloudWatch Logs

Navigate to the CloudWatch Logs section in the AWS Management Console to verify that the logs are being captured:

Open the CloudWatch console.
In the navigation pane, choose Logs.

& confirm your logs are being sent over!

Conclusion

By following these steps, you can effectively centralize your Linux and Windows logs in AWS CloudWatch Logs. This setup provides you with powerful tools for monitoring, troubleshooting, and analyzing your system logs, ensuring better system health and security.

Additional Tips

Automation: These steps can be automated using scripts or AWS CloudFormation.
Advanced Configuration: For more advanced log configuration options, refer to the AWS CloudWatch Logs documentation.