Audit Logs in MuleSoft Anypoint Platform

Retrieve them, centralyze them, and analyze them

As in any activity (sports, music, retail, government, etc), auditory is a crucial part to take into account to avoid or to identify unusual actions.

I remember famous artists like The Rolling Stones being in problems because an auditory detected that they were not paying taxes (https://www.dailymail.co.uk/tvshowbiz/article-10722959/Rolling-Stones-money-spinning-tour-UK-Europe-makes-36m-pay-just-300-000-tax.html).

Taken from here https://www.sopitas.com/musica/the-rolling-stones-tocaban-por-primera-vez-con-este-nombre/

Anypoint Platform offers a set of foundational services that also include an Audit Log where any event that occurs within the platform will be registered. For example:

1. Someone creates a user or deletes it
2. Someone creates a new environment
3. A new Business Group is created
4. A new role has been assigned to a group of people
5. A new application has been deployed into the Production environment
6. A new API has been published or deprecated
7. A new API version has been approved
8. An application has been stopped
9. A user has failed to log in but is trying to get access with multiple attempts
10. An API policy has been removed
11. An API policy has been modified
12. A new user team has been created
13. More vCores have been added to a specific environment
14. An API has been added to a public API Portal
15. An API contract has been revoked and activated
16. A new application has been created
17. An API definition was changed
18. A new asset has been pushed to Exchange
19. etc

The list can continue and it can be quite large.

All that is available within the Audit logs options under the Access Management menu in Anypoint Platform:

Audit Logs menu option.

If you click it, then we can see the audit activity:

As we can see there are different filters we can use to retrieve the information:

• The organization and sub-organizations. If your tenant is divided you can filter to the specific organization you would like to retrieve the information
• The time window from which you want to retrieve the audit logs
• The products and or Anypoint Platform modules from which you would like to retrieve the audit logs:

1. API Designer
2. API Management
3. Access Management
4. Anypoint MQ
5. Exchange
6. Object Store
7. Runtime Manager
8. Trusted Domains
9. etc

Not only that, but you can also filter out the specific objects that you are interested in, for example:

1. APIs
2. Alerts
3. Contacts
4. Clusters
5. Flex Gateway
6. Queue
7. VPC
8. Agents
9. etc

And then if you are looking for a specific action, you also have that alternative:

1. Activate
2. Add API
3. Delete Contract
4. Delete Partner
5. Deploy
6. Login
7. Logout
8. Modify queue
9. Rename
10. Redeploy
11. Updating API

And the obvious filter is to select who executed the action.

For example, this query:

We’ve used:

and also:

With this action:

And this user:

The result:

All good at this point. This is useful enough for an operator responsible for getting that type of information to audit reports, compliance, and any other needs. The operator also can export the query results to a CSV file:

And again, it’s all good at this point. And since all that information is stored for 06 years there is no reason to be worried about if an external (or internal) auditor asks you about what a specific user did 05 years ago (believe me, some auditors have that type of ability -just kidding).

But if we just use it to react to a specific inquiry or to generate audit reports, we may be losing the opportunity to promptly detect anomaly changes that could affect our organization, The fastest you get notified, the fastest you react will directly affect the avoidance of a disaster.

And we are not just talking about sending an email every time an abnormal activity occurs, that is OK and is recommendable but we can do it a little bit better. We may be interested in sending the audit logs to a centralyze Syslog that captures not only Anypoint Platform events but also other applications events. We can then correlate actions that occurred across different systems. Or we may be interested in sending the audit logs to a SIEM (Security information and event management), or a centralyze logs system such as Elastic.

We can continue giving ideas, but let’s nail it. Let’s create something that may be useful and practical enough that serve you to easily send the audit information wherever you want.

The first part of the useful and practical method we will use is the Audit API. This is the base resource:

https://anypoint.mulesoft.com/audit/v2/organizations/{{org_id}}/query

The documentation can be found here:

Audit Log Query API

If you take a look at the developer tools (in your browser) while executing the sample query we described in previous paragraphs you will notice that the Audit API was used, look:

It was a post to the audit API, to my specific organization. The body that was used is something like this:

Full JSON object:

{
  "startDate": "2023-10-14T14:40:26.052Z",
  "endDate": "2023-10-14T15:40:26.052Z",
  "platforms": [
    "CoreServices"
  ],
  "objectTypes": [
    "User"
  ],
  "actions": [
    "Login"
  ],
  "environmentIds": [],
  "objects": [],
  "user-ids": [
    "b449eaf7-631"
  ],
  "offset": 0,
  "limit": 50,
  "ascending": false
}

If we translate that into a cURL:

curl --location 'https://anypoint.mulesoft.com/audit/v2/organizations/p-1bf7-4c08-96a3-c9013a01d871/query?cursorPagination=true&doIncludeTotal=capped' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 68e9741c-4323-b6ba-c8b2006d6302' \
--header 'Cookie: XSRF-TOKEN=dtvQpQpR-hjDvcPPVKki33KxISwuZmxTsTPQ; _csrf=_hQmGrVDZqul3mcmsPnEZZPq' \
--data '{
  "startDate": "2023-10-14T14:40:26.052Z",
  "endDate": "2023-10-14T15:40:26.052Z",
  "platforms": [
    "CoreServices"
  ],
  "objectTypes": [
    "User"
  ],
  "actions": [
    "Login"
  ],
  "environmentIds": [],
  "objectIds": [],
  "userIds": [
    "b449eaf7-6312-42ad-9b40a68542"
  ],
  "offset": 0,
  "limit": 50,
  "ascending": false
}'

If you are wondering about the bearer token that was used in the previous call, that is a good observation. When the query is performed within the Anypoint Platform console, a person (user) has already logged into the platform, and therefore a bearer token has been served. But if we want to externalize that to create a certain level of automation to retrieve the audit logs and send them to any system (such as a syslog or SIEM), then we need to create a connected app in Anypoint Platform and set the right scopes to allow the app to use the Audit capabilities. To learn more about connected apps, please read this:

https://docs.mulesoft.com/access-management/connected-apps-overview

Alright, what we’ve learned so far is:

1. Anypoint Platform can store for 06 years the audit logs of the system
2. Audit logs can be queried from the Anypoint Platform console. A lot of filters can be used to retrieve the specific information
3. The queries can also be performed using the Audit API
4. We can use the Audit API to automate the action to retrieve the audit logs and send them to another platform

Let’s review some queries using the API:

1. If we want to query which applications were deployed by a given user:

curl --location 'https://anypoint.mulesoft.com/audit/v2/organizations/-1bf7-4c08-96a3/query' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer -741c-4323-b6ba-c8b2006d6302' \
--header 'Cookie: XSRF-TOKEN=dtvQpQpR-hjDvcPPVKki33KxISwuZmxTsTPQ; _csrf=_hQmGrVDZqul3mcmsPnEZZPq' \
--data '{
    "startDate": "2022-10-14T16:32:00.142Z",
    "endDate": "2023-10-14T16:32:00.142Z",
    "platforms": [
        "Runtime Manager"
    ],
    "objectTypes": [
        "application",
        "Application"
    ],
    "actions": [
        "Deploy"
    ],
    "environmentIds": [],
    "objectIds": [],
    "userIds": [
        "b44af7-6312-42ad--027d40a68542"
    ],
    "offset": 0,
    "limit": 50,
    "ascending": false
}'

2. If we would like to know which policies have been applied to APIs:

curl --location 'https://anypoint.mulesoft.com/audit/v2/organizations/1bf7-4c08-96a3-c9013a01d871/query' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 68e9005e-741c-4323-b6ba-c8b2006d6302' \
--header 'Cookie: XSRF-TOKEN=dtvQpQpR-hjDvcPPVKki33KxISwuZmxTsTPQ; _csrf=_hQmGrVDZqul3mcmsPnEZZPq' \
--data '{
    "startDate": "2023-03-19T04:27:14.614Z",
    "endDate": "2023-12-18T04:27:14.614Z",
    "platforms": [
        "api-platform-api"
    ],
    "objectTypes": [
        "policy"
    ],
    "actions": [],
    "environmentIds": [],
    "objectIds": [],
    "userIds": [],
    "offset": 0,
    "limit": 50,
    "ascending": false
}'

Or what about if we want to know which users have been edited in a specific environment:

curl --location 'https://anypoint.mulesoft.com/audit/v2/organizations/1bf7-4c08-96a3-c9013a01d871/query' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 68e9005-4323-b6ba-c8b2006d6302' \
--header 'Cookie: XSRF-TOKEN=dtvQpQpR-hjDvcPPVKki33KxISwuZmxTsTPQ; _csrf=_hQmGrVDZqul3mcmsPnEZZPq' \
--data '{
    "startDate": "2023-02-21T08:08:58.521Z",
    "endDate": "2023-12-21T09:08:58.521Z",
    "platforms": [
        "CoreServices"
    ],
    "objectTypes": [
        "User"
    ],
    "environmentIds": [
        "c1d2045d-5ba9--a9b94336beeb"
    ],
    "actions": [
        "edit",
        "Edit"
    ],
    "subactions": [
        "Enable user"
    ],
    "objectIds": [],
    "userIds": [],
    "ascending": false,
    "organizationId": "ac584c40-1bf7-",
    "offset": 0
}'

We’ve created a starter project that can be found here:

GitHub - rcarrascosps/mulesoftAuditLogs: Contains a sample application to constantly query the…

The idea is to use the APIs and then send them to a Syslog, in my case papertrailapp.com.

The code is very basic:

#!/bin/bash

#muleLogin="curl --silent --location --request POST 'https://anypoint.mulesoft.com/accounts/login' --header 'Content-Type: application/json' -d @auth.json"

muleLogin = "curl --location 'https://anypoint.mulesoft.com/accounts/api/v2/oauth2/token' --header 'Content-Type: application/x-www-form-urlencoded' --header 'Authorization: Basic $creds' --data-urlencode 'grant_type=client_credentials' --data-urlencode 'scope=full'"

muleLoginJSON=$(eval $muleLogin)
accessToken=$(echo $muleLoginJSON | jq -r '.access_token')
echo $accessToken

environmentIds="<your_anypointplatform_environment_id>"
organizationId="<your_anypointplatform_organization_id>"

total=0
currentDate=`date +"%Y-%m-%d %T"`
echo Current date is: ${currentDate}
startDate="2023-01-30 14:59:43"
echo $startDate
while :
do
        queryJSON='{"startDate": "'$startDate'","endDate": "'$currentDate'","platforms": ["Runtime Manager"],"environmentIds": ["'$environmentIds'"],"objectTypes":["Application"],"ascending": false,"organizationId":"'$organizationId'","offset": 0,"limit": 25}'
        #echo "[INFO]$queryJSON"
        audit="curl --silent --location --request POST 'https://anypoint.mulesoft.com/audit/v2/organizations/$organizationId/query' --header 'Authorization: Bearer $accessToken' --header 'Content-Type: application/json' -d '$queryJSON'"
        auditJSON=$(eval $audit)
        echo "[INFO] #########################################"
        echo $auditJSON
        totalJSON=$(echo $auditJSON | jq -r '.total')
        echo "Total of new events: $totalJSON"
        if [ $totalJSON -gt $total ]
          then
              echo "[INFO] Event(s) send to syslog"
              echo $auditJSON  > /dev/udp/yourUDP_server/yourUDP_port
          else
              echo "[INFO] There are no new events."
        fi
        sleep 5
        startDate=$currentDate
        currentDate=`date +"%Y-%m-%d %T"`
        echo "[INFO] StartDate $startDate"
        echo "[INFO] EndDate $currentDate"
done

In the code, we are just using a single call to the Audit API, and the results are retrieved and then pushed to Papertrail. We are querying if an application is modified and or deployed

If you clone the repo and execute it:

You will see something similar to what is in the image, then undeploy (for example) an application, and the script will notice it:

0

And at Papertrail:

The repo will continue its way to be modified and extended to make it even more useful and support more queries. Keep an eye on the repo if you are interested in this topic.