Audit The Security Groups Across All Your AWS Accounts
If the scale of your organization allows you to operate on a single AWS region within a single account, it may not be a big deal to keep your security groups safe. But if your organization has multiple accounts and operations in various regions, It’s hard to feel safe when there is a possibility that someone may leave an unwanted security group open. Moreover sometimes this, open security group, may include the whole world. So this article is about how to audit and even revoke your security group ingresses in a secure, efficient, and affordable manner.
Options to Follow
You have few options for feeling safe and secure. You can audit and even revoke some security groups in all the regions throughout your accounts. The first option to consider is the AWS Firewall Manager. But you can find it a bit costly, because of the number of regions being multiplied by the number of accounts for calculating the cost. So if you find this price too high you can also find another way which is provisioning an EC2 instance configured with a cronjob script that utilizes the AWS CLI to list all the security groups periodically and revoke the unwanted security groups. But you had better implement this approach for all of your accounts separately and also you should be aware of the network cost for all these periodic request — response traffic. Also, it would be a bit overburden and not so efficient.
What We Did — The Short Story
At Insider, we have figured out a third way to overcome the problem:
First of all, the picture above explains our approach. Let me help you read the picture if it seems a bit complicated. The flow in the diagram is as follows: When an unwanted security group ingress is created, an event is registered in the CloudTrail Service, then this event triggers EventBridge Service. The EventBridge Service triggers an Event Rule through its default Event Bus. The rule triggers another event rule residing in the management account via an event bus open to the organization. At the end of this setup, a lambda function is triggered and it revokes the unwanted security group ingress.
What We Did — The Long Story
Before starting, the first possible pitfall is not having a CloudTrail trail that collects all the events coming out of all your active regions. It is necessary because when you don’t have a trail in your account, the CloudTrail Service cannot trigger the EventBridge Service. If one of your AWS account is recently activated and not yet configured properly, you could possibly encounter such a situation. So, be sure to have a trail in your account, and also that trail should be a multi-region-trail. Since you want to audit and clean all the regions, this is something crucial to keep in mind. For a detailed consideration, you can visit this page.
Secondly, AWS has allowed the Amazon EventBridge Service to operate on cross accounts for some time. The Amazon EventBridge is a serverless event-bus service that runs in an event-driven architecture. The words “Event-driven architecture” are important here because it’s not necessary to create a system that checks your resources periodically or monitors them all the time. The setup you configure should start running after a special CloudTrail event is being triggered. This means that you need to catch this event in the EventBridge.
We thought that we can get benefit from this feature and trigger a lambda function when a black listed ingress rule is created in any single account’s region.
I keep telling about the CloudTrail event. But what is this event? The name of this special event is “AuthorizeSecurityGroupIngress”. When any CloudTrail event occurs, the event falls into the default Event Bus. Then these events are applied to the event rules of the default Event Bus, and they are filtered according to the patterns of the rules. The pattern of the rule defines the filtering action. For a detailed look at how to write your own filtering event pattern, you should visit this page. I have to inform you that It’s a bit tricky to consume the information there. You should give some time to this pattern issue. By that time, you can target an SNS and bind your email as an SQS target to examine the way it works. You can use my event pattern as an example below. Don’t forget to modify the pattern according to your own situation.
If the pattern configured in the rule catches an event, you have the option to trigger various targets. One of those targets is an event bus (either in the same account or in a different account). If the target lambda is located in a different account, you should target a different account’s event bus. In our example, we have configured an organization-level event bus and all other events were addressed to this event bus. Definitely, you should also configure the resource-based policy of that event bus to permit the events of other accounts in your organization. Below is an example resource-based policy. Don’t forget to put your own data into the policy.
If you have managed to come to that point, the rest is easy. You will arrange an identical event rule in that account also. So that the event rule will have the event to trigger another target. This time the target will be a lambda function. The lambda function will leverage the AWS SDK to take effect according to the event’s account id, region, security group rule, and the ingress id. The NodeJS code snippet below shows the critical parts of the function :
The above code handles the cross-account operation by assuming a role configured in that cross-account. All the accounts should have the same role with the proper permissions. You can visit this page to learn more about this role assuming in a different account.
End Of The Story
When you have completed the setup above, you’re good to go. You can directly revoke the unwanted ingress or you can even send a slack message to inform your security team. We chose to do both. If you have figured out a different approach, please feel free to send a comment below.
