Lessons in Azure Security, Part 2— Logging Made Simple
In the second part of the series we will be focusing on getting the right security information out of your Azure Environment.
Public cloud environments can (and should) be incredibly fast paced, where users and applications will be authenticating to provision, iterate and tear down resources on a regular basis.
This is why it is useful to have strong visibility over your cloud environment, it allows for spotting anomalous usage patterns through use of a SIEM tool (we will cover Azure Sentinel as one example later on in the series).
This article will cover the various ins and outs of logging in Azure, how to set it up, what types of information you should care about retaining and what supported destinations are in place for logging.
Azure Logging in a Nutshell
In the diagram above i’ve outlined some of the key areas to consider when monitoring your Azure ecosystem and the supported destinations. This may not be an exhaustive list but hopefully a solid starting point.
In the rest of this article we will cover:
- Setting Things Up- how to effectively set up log exports for various types of Azure resources using Diagnostic Settings. Plus tips to ensure you don’t miss out on critical platform logs.
- What to monitor?— what Azure resources should you aim to monitor and why gathering logs from them is useful.
- Logging Destinations- suitable destinations to store logs in Azure and the various pros and cons of each (Azure Storage, Event Hubs and Log Analytics Workspaces).
1) Setting Things Up
When it comes to deciding which resources to gather log information from, it’s worth firstly working out what types of information you may need to retain beyond the Azure standard retention period. It’s careful you weigh this up against the cost of storing logs for long periods of time.
The key logging service provided is Azure Monitor Logs and these entries can be aggregated from various resources to provide actionable insights as shown below.
Don’t miss out!- Resources will not capture logging information unless you set up the correct diagnostic settings, these logs are really useful when an investigation needs carrying out.
Introducing Diagnostic Settings… for the majority of resources in Azure, logging will be configured by an entity called Diagnostic Settings. Diagnostic settings hook together the following types of information:
- What categories of information you want to collect — this differs per resource and you can chose which information to capture.
- Where you want to send the logs — the supported options are Azure Storage, Event Hubs and Log Analytics Workspace, we will cover these destinations in more detail later in the article.
Here are some top tips for setting up diagnostic settings, and most importantly ensuring these are set up for your most important resources, you could consider one or multiple methods to ensure full coverage:
- Get these into your infrastructure as code modules — for example Terraform, we include diagnostic settings by default in our infrastructure modules, so we know that any infra we provision will configure compliant logging to our desired destination. Heres an example resource below:
- Use Azure Policy to check for missing diagnostic settings- you can set up Azure Policy to call out missing diagnostic settings, and even do a deploy if not exists (this will require some config based on your use case).
- Run Automation to ensure your resources are covered — a talented member of my engineering squad has used the Azure Python SDK and Azure DevOps pipelines to automatically onboard new resources to capture logs, this means that we can spin up resources at scale without worrying about missing logging — Bravo!
2) What to Monitor?
In this section i’ll highlight some key resources and why you may find it useful to gather logs from a security standpoint — note this is not exhaustive and you should review against your own needs.
Azure Subscriptions
An Azure Subscription is a cost object linked to a billing profile and a grouping that contains cloud resources within it. Often it will contain the cloud resources for a particular project or environment.
Subscription level activity is recorded to the Activity Log, from a security perspective you should be looking to retain the following items from the schema:
- Azure Policy Actions — any policy actions that have happened within the subscription — analyzing the affect of policies and potentially flagging suspicious behavior.
- Security — any security centre alerts generated for the subscription — if configured correctly this can spot intrusions on Azure Resources using Azure Defender.
- Administration — these cover all resource manager calls in the subscription, such as creating, deleting and updating resources — combined with tools such as sentinel and log tooling this can build up a profile of administrative events in your subscription.
These logs are retained by Azure for 90 days but I recommend retaining these for at least a year using Diagnostic Settings.
Azure Active Directory
We discussed Azure Active Directory in detail in my last article, it is essentially the gatekeeper of your cloud ecosystem, responsible for authenticating and authorizing your users and everything in-between. I recommend retaining the following logs:
- Audit Logs — tracking any tenant level configuration changes to your Azure Active directory — again this can spot unusual and sometimes unwanted configuration changes.
- Sign in Logs — for P1/P2 licensed users this schema will track information around user sign ins, including conditional access — this helps track and quantify any anomalous or risky sign ins to your cloud environment.
- Service Principle / Managed Identity logs- it is likely that you will be running Service Principles within your Azure environment and potentially harnessing the benefits of managed identities — these logs help track activity associated to services and applications you have authorized to access Azure, this could help spot if these credentials were compromised or being used for nefarious activities.
Azure retains these for 30 days by default, again i’d recommend retaining these for up to year using Diagnostic Settings.
Azure Resources
For all Azure Resources, there are two groupings of data that can be ingested Logs and Metrics, for the purposes of this article you should be focusing on logs which tend to have a more audit / security focus. Again our friend Diagnostic Settings come into play here if you wish to export and retain the logs in your chosen destination.
Each resource will have a different Log schema that can be researched using the Azure Documentation — this covers the types of operations recorded and the data fields.
Giving a real life example, I tend to use Azure Key Vault for secrets management, in this case we want to keep a track of who is creating / accessing secrets — I set this diagnostic setting using Terraform on creation of the resource into a central log analytics workspace, achieving this with ease!
It’s worth noting here that some resources have separate audit settings that can not be configured via diagnostics such as Azure SQL. Instead this is a resource level configuration that will stream SQL Server logs to Azure Storage.
OS Level Audit
When it comes to OS Level Audit on your Virtual Machines, Azure offers two agents that can be used in parallel or exclusively depending on your use case… there are also a plethora of other tools out there.
- Windows / Linux Diagnostics Agent — This agent is perfect at shipping logs to Azure Storage for archiving and Event Hubs for external tool integration.
- Log Analytics Agent — As you can probably tell within the name, this ships OS logs through to a log analytics workspace, for user friendly analysis.
Tip: Get these into your infrastructure as code modules using VM extensions or deploy them as part of a baseline machine image.
Azure Security Centre
Security Centre alerts on suspicious and potentially malicious activity within your Azure environment. Using the recently rebranded Azure Defender this can offer advanced threat detection capabilities for various supported resources.
You may want to export these alerts elsewhere, such as a SIEM tool or log analytics workspace — for this you can configure Security Centre Continuous Export. (I’ve provided an article on this in the further reading section!).
Azure DevOps
I will almost certainly write a post around Azure DevOps in the near future, but for now if you want to capture and retain security related events head on over to Auditing > Streams.
This functionality will capture and retain a series of useful information such as config changes to your repos, pipelines and artifact streams. As well as pipeline executions.
Log streaming functionality supports the usual destinations, Log Analytics Workspaces, Event Grid and even native Splunk integration.
3) Log Destinations
Last but not least let’s round off this article with Azure supported logging destinations and their merits / drawbacks, when setting up Diagnostic Settings you can choose one or multiple destinations dependent on your use case!
Azure Storage
A scalable, platform as a service Blob, File and Table storage service that can be used to store large volumes of log files for long periods of time.
Pros — Perfect for cost affective archiving and contractual retention of log files.
Cons- Not as easy to read / query as other options.
Azure Log Analytics Workspace
A dedicated logical storage unit that makes it easy to query data sets over a large variety of different log sources using KQL (Kusto Query Language), this is my tool of choice when it comes to Azure Logging as it’s breathtakingly easy to craft queries, save workbooks and even alert off custom query rules!
Pros- Easy to query / manipulate data from different sources, can use workbooks to visualize / summarize data.
Cons- Retaining large amounts of data can be more costly than the Storage option, only use for data ranges that would need to be queried actively.
Azure Event Hub
A scalable way to stream and process data from various event sources, perfect to consolidate data to be ingested by a third party toolset such as DataDog, Splunk and other third party SIEM or log management solutions.
Pros Great for third party integrations.
Cons Not suited for querying / retention.
Wrap Up
That’s it, we’ve covered everything from how to set up logging in Azure using Diagnostic Settings, what types of resources you should look to gather information from / how to use the information and lastly how to find a suitable home for your logs using supported destinations.
Next up in the series we will discuss how to use Azure Sentinel to get some proactive insights from your consolidated logs.
Look out for Part 3— Getting Started with Azure Sentinel
Please also check out some selected articles around the topics discussed below.
