Why It’s Time to Migrate to Azure Monitoring Agent

Log Analytics Agent isn’t dead yet, but here’s why it’s time to migrate to Azure Monitoring Agent

Published in

Contino Engineering

9 min readSep 18, 2023

If you’re working in Azure, it hopefully isn’t coming as a shock to you that Microsoft will be retiring Log Analytics Agent on 31st August 2024.

In this blog we explain why you should consider migrating as soon as possible, rather than taking it down to the wire. We’ll take you through the new Azure Monitoring Agent, its benefits and how it compares to legacy agents, as well as migration best practices and how to prepare for changes.

Even without the 2024 deadline, we’d recommend migrating well ahead of the cutover date to reap the benefits of the Azure Monitoring Agent.

1. What is Azure Monitoring Agent (AMA)?

The Azure Monitoring Agent (AMA) is a data collection tool that collects monitoring data from a Virtual Machine and delivers it to Azure Monitor.

Azure Monitor is a service in Azure that provides performance and availability monitoring for applications and services in Azure, other cloud environments, or on-premises. Azure Monitor collects data from multiple sources into a common data platform where it can be analysed for trends and anomalies.

Effectively, AMA will collect monitoring data from different operating systems and send it to Azure Monitor for further analysis and monitoring use.

Why migrate?

Excluding the fact that the Log Analytics agent will be retired in less than a year, there are many reasons why you should consider migrating to the new Azure Monitor Agent.

Benefits of migrating to Azure Monitor Agent

i/ Better security and performance

AMA uses Managed Identity or Azure Active Directory (Azure AD) tokens (for clients), which are much more secure than the legacy authentication methods.
AMA provides a higher events per second (EPS) upload rate compared to legacy agents.

ii/ Cost saving

AMA relies on Data Collection Rules (DCRs) as the control plane, which lets you target data collection from groups of machines connected to the same or different workspaces, as compared to the “all or nothing” approach of the legacy agents.
Using DCRs you can define which data to ingest and which data to filter out to reduce workspace clutter and save on costs.
Support for additional destinations, like Azure Storage accounts, will be available in the future, which will enable you to optimise for longer storage periods at cheaper costs.

iii/ Centralised and simpler management

Easy multihoming on Windows and Linux, across regions and tenants.
Centralised, ‘in the cloud’ agent configuration makes every action across the data collection lifecycle, simpler and scalable, from onboarding to deployment to updates and all related changes over time.
Greater transparency and control over services and related updates, including Sentinel, Defender for Cloud, and VM Insights.

2. Cost Optimisation With Data Collection Rules

Data collection rule (DCR) in Azure Monitoring Agent allows sharing the same rule across single or multiple VMs as mentioned above.

To understand how a DCR can save costs, see this following graph

From Data collection rule to transformation to destination

Using transformation that is available within Data Collection Rule before sending to destination, you can reduce the amount of data ingestion while creating more granular and useful data that can be viewed in Log Analytics Workspace.

Transformations are defined in a data collection rule (DCR) and use a Kusto Query Language (KQL) statement that’s applied individually to each entry in the incoming data. It must understand the format of the incoming data and create output in the structure expected by the destination.

Microsoft has a list of cost optimisation strategies for different log collections using DCR. Topics covered are featuring

Migrate from Log Analytics agent to Azure Monitor agent for granular data filtering.
Filter data that you don’t require from agents.
Determine whether you’ll use VM insights and what data to collect.
Reduce polling frequency of performance counters.
Ensure that VMs aren’t sending duplicate data.
Use Log Analytics workspace insights to analyse billable costs and identify cost saving opportunities.
Migrate your SCOM environment to Azure Monitor SCOM Managed Instance.

And current tables that support transformations in Azure Monitor Logs can be found here.

For example, you create a DCR rule custom filter using XPath queries. You can further filter the collected data by editing the DCR to add a transformation.

An example of XPath data collection rule filtering Windows Event using XPath.

For more detail on transformations, go to this tutorial: Add a transformation in a workspace data collection rule by using the Azure portal.

ARM sample DCR templates

Terraform sample DCR templates

3. Comparison Between New and Legacy Agent

Microsoft has already stated that Log Analytics is fully consolidated into Azure Monitor Agent.

However, InfluxData Telegraf agent and diagnostic extension are still under consolidation, and unfortunately Microsoft hasn’t given any more details on the exact timeline. Watch this space for latest extension updates.

Telegraf agent: Sends data to Azure Monitor Metrics (Linux only). Only basic Telegraf plugins are supported today in Azure Monitor Agent, and bear in mind that InfluxData Telegraf is an open source agent and not officially supported by Azure Monitor.

Diagnostics extension: Sends data to Azure Monitor Metrics (Windows only), Azure Event Hubs, and Azure Storage.

If you have the requirement to collect additional logs that are not yet consolidated, coexisting AMA with diagnostic extension or telegraf agent alongside would still be the way to go. You would need to consider and try to avoid monitoring data duplication. There are a number of good articles highlighting different combinations or ways to achieve that. (Further links at the end of this post!) However, that is not our focus today.

The following two tables summarise the comparison of Azure Monitor Agent with the legacy Azure Monitor telemetry agents for Windows and Linux.

Windows agent

Linux agent

Comparison summary

Azure Monitor Metric is in public review
Azure Stack HCI is supported by Azure Monitor Agent over legacy
Automation Update management is no longer required. Update management center (Preview) is the v2 version of Automation Update management and the future of Update management in Azure.
Bear in mind that again there are limitations on which OS are being supported (custom images, marketplace images and Azure Arc-enabled servers) and unsupported (Windows client, Virtual machine scale sets and Azure Kubernetes Nodes)

We can conclude that for log analytics agent replacement, AMA can be enough to suit your needs if you only send data to Azure Monitor.

However, if you want to get events from Windows (ETW), crash dumps, application logs and agent logs or send data to destinations like Azure Storage or Event Hub. Then we need to combine AMA with diagnostic extensions

4. Operating Systems Compatibility

It is worth mentioning Azure Monitor Agent compatibility before even considering migration.

Architecture: All operating systems are assumed to be x64. x86 isn’t supported.
Physical and on premise servers: View supported operating systems for Azure Arc Connected Machine agent, which is a prerequisite to run Azure Monitor agent.
ARM-based: Azure Arc connected agent is not supported.

As AMA is still constantly evolving, I would suggest checking out official Microsoft for the operating system compatibility chart.

For Windows OS support:

Please visit https://learn.microsoft.com/en-us/azure/azure-monitor/agents/agents-overview#windows

For Linux OS support:

Please visit: https://learn.microsoft.com/en-us/azure/azure-monitor/agents/agents-overview#linux

5. AMA Migration Best Practices and Recommendations

Assess current risks, requirements and standards

In all migration, you want to first review if you can migrate successfully. Microsoft has provided a checklist and tools to help as follows

AMA Migration Helper to view virtual machine status
This is a public Azure workbook template to see connected agents and discover solutions enabled on your Log Analytics workspaces that use legacy agents, including per-solution migration recommendations.
You can find it on the Azure portal under Monitor > Workbooks > Public Templates > Azure Monitor essentials > AMA Migration Helper
DCR generator to parse Log Analytics Agent configuration from your workspaces and generate/deploy corresponding data collection rules automatically.
This is a powershell script for creating ready to deploy ARM templates to deploy data collection rules based on previous legacy agents.
Review prerequisites for installing Azure Monitor Agent
Built-in roles, permission, managed identities, azure arc agent (for on-premise machines), networking/firewalls. For more detail, visit here
Existing Azure policy in enterprise scale
Existing legacy policies may be in place that deploys legacy agents. That would require proper migration planning and testing before rolling out to all other machines.
Review machines that require upgrade
As some of you might face the problem where your OS is non-compatible by AMA which we have briefly mentioned in the “Operating systems compatibility” section. Maybe it is an opportunity to re platform from older versions of Windows and Linux.

Validate and test migration steps

As Microsoft has already well documented migration steps that we can follow, troubleshoot and test along.

6. Prepare for Change

Following considerations that might be useful which has been categorised below

a) Pre migration review

Determining test subject machines with AMA migration helper and DCR generator
Finding suitable test machines depends on multiple factors that may vary across teams, business needs and ownerships. Alignment with different projects/teams in a phased approach would be recommended.
Conducting Proof of Concept (PoC) prior to full scale migration
We all want some assurances before having a full scale migration that could involve 100+ virtual machines or more for large organisations. By doing time boxed PoC, we can quickly understand and estimate time and resources required.

b) Automation

New Azure policies to enable Azure monitoring agents are still under preview.
Contino and our clients are actively tracking this issue.
https://github.com/Azure/Enterprise-Scale/issues/1055
Develop or upgrade existing infrastructure as code (IaC) templates to deploy AMA and their data collection rules
There are powershell, Azure cli and ARM templates. I would recommend to further read our article if you are interested in IaC here

c) Security

Enable network isolation for AMA. By default, Azure Monitor Agent connects to a public endpoint to connect to your Azure Monitor environment. To enable network isolation for your agents, create data collection endpoints and add them to your Azure Monitor Private Link Scopes (AMPLS).
Or by using a firewall with correct ports and endpoints. See firewall requirements.
Disable and block old agent from installation or modification
There are different angles to this solution. Both legacy and AMA are installed from VM extensions which can be prevented if proper drift management is implemented.
We can further monitor or remove non compliances if we find an old agent policy exists above an AMA one. For active protection within the operating system, we can define definitions or look at the installation path to prevent or protect old/new agents from being installed or modified. Additional antivirus or security scanning tools would be the way to go.
Good to note that currently, Microsoft Defender for Cloud and Microsoft Sentinel support are still in public review.
See further here for their detailed implementation:
Deploy the Azure Monitor Agent to protect your servers with Microsoft Defender for Cloud
Click here for the Migration process for Microsoft Sentinel

d) Field expert troubleshoot and support

Make use of Azure Monitor Agent troubleshooter
This is a command line executable designed to help diagnose issues with the agent, and general agent health checks. It can run checks to verify agent installation, connection, general heartbeat, and collect AMA-related logs automatically from the affected Windows or Linux VM.
Microsoft and Contino can help! We have all kinds of Azure devops, architects, development, security and migration experts and client engagement experiences. You can get in touch here with the folk at Contino who will be only too happy to support you!

Summary

With the current log analytics agent expiring within the year, now is the right time to be thinking about migration.

We have highlighted some of the potential risks, migration best practices, cost optimisation using DCR with filters and transformations, and preparations for changes to a successful migration journey. Start small, fail fast and repeat.

Let us know your thoughts and feel free to reach out! We are always here to help.