How to set up your own Threat Intel infrastructure (II)
Introduction
This is the second post in the series “How to set up your own Threat Intelligence infrastructure”. The aim of this series is to describe how to set up your own Threat Intel infrastructure using MISP and deploy it on AWS. This article describes how to deploy MISP on AWS without exposing the platform on the Internet.
AWS
AWS is the cloud platform offered by Amazon and it is made up of a large number of cloud computing products and services. AWS provides many different kinds of services, such as servers, storage, networking, remote computing, email, mobile development or security. Like other cloud providers, AWS offers a pay-as-you-go model for its cloud services.
AWS, and cloud providers in general, offer flexibility and scalability on demand. This helps organisations to plan their infrastructure roadmap with a subscription plan without making a big commitment.
AMI
An Amazon Machine Image(AMI) is a special type of virtual appliance that is used to create a virtual machine within the AWS EC2. Different types of instances can be launched from a single AMI to support the hardware of the host computer used for the instance. With AMIs, it is faster and easier to set up an instance than with traditional software deployments as there is no manual set-up, no configuration and no additional hardware.
Once an AMI has been created and registered, it can be reused to launch multiple instances with the same configuration. Tags can be used to categorize and manage AMIs. When an AMI is no longer needed, it can be deregistered. A deregistered AMI cannot be used to launch new instances, but existing instances will remain active.
MISP provides an AMI ready to be deployed and it has been helpful for setting up the MISP environment on AWS in an easier way.
Architecture
In this case, the MISP instance has been deployed on the private subnet of an AWS VPC that has been created for this purpose, which makes the MISP instance unreachable from the Internet. The connection to the MISP instance will be done through a bastion host, that will be hosted on the public subnet at it will have only the SSH port opened. The SSH authentication is using a public key and there will be also a security group for controlling the IP that is trying to reach the bastion host.
If you are not familiar with these concepts, I encourage you to take a look at AWS documentation, which explains how to perform this configuration step by step. This post by CloudAcademy might be also useful.
The infrastructure includes the following components:
1. VPC: A VPC with a CIDR range of 10.0.0.0/16 with the following sub-components:
- A Public subnet that hosts a NAT instance and a bastion host.
- A Private subnet that hosts the MISP instance.
2. Route tables:
- One Route Table for controlling the network traffic in the private subnet.
- One Route Table for controlling the network traffic in the public subnet and forwarding traffic to the Internet gateway.
3. 1 Internet gateway: It connects the VPC with the Internet.
4. 3 EC2 Instances
- MISP instance: Hosted on the private subnetwork.
- Bastion host: A server that provides access to the private network from the Internet.
- NAT instance: This instance allows instances hosted on the private subnetwork (e.g., MISP instance) to call external services on the Internet (e.g., external server for updates) while at the same time it blocks inbound traffic from the internet.
5. Security Groups: A security group acts as a virtual firewall for AWS instances to control inbound and outbound traffic. It has been created the following security groups:
- NAT-SG: Only allows HTTPS and HTTP traffic from the Internet.
- Bastion-host-access: Only allows SSH traffic from a specific IP address.
- MISP-private-subnet: Security group that allows HTTPS and SSH traffic from the Bastion-host access security group. This means that only inbound traffic that comes from this machine can reach the private network.
6. AWS SES (Simple Email Service): AWS email service for sending notifications via email regarding MISP events. This service is optional, the MISP instance can be configured to send emails.
The bastion host-access security group contains the list of IP addresses that are allowed to connect to the bastion-host via SSH. Using an SSH tunnel the user can access the MISP instance via SSH for administrative tasks or via web using a proxy SOCKS5.
MISP configuration
To deploy the MISP instance, we will use a special AMI provided by the MISP project. The AMI can be found on the AWS AMI repository (Figure below) and it is quite straightforward to run the AMI following a few additional steps.
After selecting the AMI, the user will have to configure the type of instance. Since this is only a PoC, I’ve chosen a micro instance, but it’s up to the users to choose the type of instance that fits better with their requirements.
After downloading the private keys of the host instance and the bastion instance, we will need some configuration in order to access the MISP instance via the bastion host. We can also create a config file within our .ssh folder with the following information:
With this configuration, we access our machine to the MISP instance using the bastion host as a bridge. This article by RedHat explains quite well bastion host and ProxyCommand.
Once that we will have to add the keys to the authentication agent by typing:
ssh-add -k misp_key.pem
ssh-add -k bastion_key.pem
We can test the connection with the MISP instance by typing:
ssh misp_instance
Finally, for being able of reaching the MISP instance by HTTP access, we will have to create a dynamic port forwarding by typing:
ssh -C -N -D 5555 bastion
I chose this port, but you can configure any other available port. The -C option compress the network traffic for better performance, the -N avoids the shell to be executed on the bastion host and the -D opens the port.
This article by AWS explains how to do this in more detail.
After the port forwarding configuration, we are almost ready to access our MISP instance using the browser. We will have to configure a SOCKS5 proxy in the browser and type the IP of our MISP instance in the browser. That’s all.
The next step is to login into the MISP instance and start to configure your threat feeds and sources. This will be explained in the next article: How to set up your own Threat Intel infrastructure: Part 3.
