Part 1: An easy way to automatically manage the AWS SG with Terraform and Python…

What if you have 500+ servers on your AWS account, managing Security group become a tedious job especially when you have multiple VPC, DevOps who changes the SG [Security Group] manually or your office has multiple IP’s which may lead to the public access to some of the confidential / vulnerable applications.

You can easily manage each and every security group with terraform with proper descriptions so that any normal engineer can understand.

Assuming if servers are Linux machines. Firstly, you need to find which all ports are listening to the world which I mean port are public.

1. You can manually login to each and every server and check the listen port by below command

sudo netstat -putan | grep LISTEN | awk '{print $1"\t"$4"\t"$7}'| grep "0.0.0.0" | grep -v 'rpc'

2. You can automate the complete process and takeout output in the text file.

import socket
import base64
import paramiko
import os
from subprocess import check_output
#Downoading all the private ip of your AWS Account to get SSH. If you have not setup your awscli then configure it by below 
#aws configure
os.system('echo " " > ip.txt')
print "Now Downloading  the ec2-hosted in AWS ......................"
os.system("aws ec2 describe-instances --query 'Reservations[*].Instances[*].[PrivateIpAddress]'  --output text > ip.txt")
with open('ip.txt') as f:
  content = f.readlines()
content = [x.strip() for x in content]
#Enter the path of your PEM file to SSH on the server
key= paramiko.RSAKey.from_private_key_file("<Path_to_Server_Pem_file>")
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
#main logic by login using EC2-USER or UBUNTU
for ip in content:
  print "\n"
  print ip
  try:
    client.connect(ip, username='ec2-user', pkey=key, timeout=10)
    cmd1='curl -s wgetip.com'
    publicip=check_output(cmd1, shell=True).strip() #on request of Mr.Arora
    stdin, stdout, stderr = client.exec_command("sudo netstat -putan | grep LISTEN | awk '{print $1\"\t\"$4\"\t\"$7}'| grep -e \"0.0.0.0\" -e $publicip | grep -v 'rpc'")
    for line in stdout:
      cmdline = line.strip('\n')
      print(cmdline)
  except socket.error, e:
    if 'Connection refused' in e:
      print('*** Connection refused ***')
except Exception as e:
    try:
      client.connect(ip, username='ubuntu', pkey=key)
      cmd1='curl -s wgetip.com'
      publicip=check_output(cmd1, shell=True).strip() #on request of Mr.Arora
      stdin, stdout, stderr = client.exec_command("sudo netstat -putan | grep LISTEN | awk '{print $1\"\t\"$4\"\t\"$7}'| grep \"0.0.0.0\" -e $publicip | grep -v 'rpc'")
      for line in stdout:
        cmdline = line.strip('\n')
        print(cmdline)
    except:
      print "unable to SSH on this"
      pass
client.close()

Output will be something like this :

192.1.6.23
tcp 0.0.0.0:80 11629/nginx
tcp 0.0.0.0:22 5077/sshd
192.1.23.51
tcp 0.0.0.0:80 11380/nginx
tcp 0.0.0.0:22 2810/sshd
tcp 0.0.0.0:443 11380/nginx
192.1.62.80
tcp 0.0.0.0:22 6092/sshd
tcp 0.0.0.0:5665 3422/icinga2
192.1.0.68
tcp 0.0.0.0:4532 1392/redis-server
tcp 0.0.0.0:21 795/vsftpd
tcp 0.0.0.0:22 1083/sshd
192.1.93.74
unable to SSH on this

From the above output, we can justify that port which needs to be allowed in the security group. For eg: 192.1.6.23 should have ports 80 and 22 open only. It can be either publicly, internally or with a certain IPs.

To understand it better let's take 192.1.6.23 SG. For me, port 22 should only be open from bastion/jump_host server or office_Ip for protection and port 80 should be open publicly if it hosts some website.

We will see how to manage these security groups with terraform in part 2 of this. Stay tuned……