Ansible Deployment Of Splunk
Automating An AWS Splunk Install With Ansible
I know, we’ve been very quiet lately, but we’ve been working hard to get our third book released and out to the wild. But we’re back now and you should be hearing a lot more from us in the coming weeks and months. We thought we would get back into it by showing you how simple it is to get a Splunk server running on AWS using Ansible.
What You Need To Get Started
- A Linux host with Ansible installed
- An AWS account with administrator access to set up EC2 instances, etc
- Credentials for this AWS account on your Linux host
- AWSCLI installed on the Linux host
- A valid Key Pair, added to AWS, so you can SSH to your new server
Get Our Ansible On
1.Start with a new directory to store everything in, with the first file you need to make is you host file. It will simply be called host, and add the following details.
1 [local]
2 localhost2.Create the new role for our server. By using the ansible-galaxy command, we can save time setting up the skeleton of the role, so run the following command:
ansible-galaxy init splunk_server
- splunk_server was created successfullyIf you haven’t used roles before, that’s all good. We are not going to do anything too complex here. In brief roles allow you to package up your tasks in a dedicated structure that Ansible already understands. It also allows you to the reuse and share you roles with others, if you like.
3.As we are creating a specific role, lets clean up a little first, let’s make a dedicated roles directory to hold our new role and move the newly created splunk_server role there:
mkdir roles
mv splunk_server roles/If you run the tree command over the directory, you will see the folder structure already set up for you.
tree roles/splunk_server/splunk_server/
├── defaults
│ └── main.yml
├── files
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── README.md
├── tasks
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.yml8 directories, 8 files
This is not a post of roles, but for now just keep in mind that the structure above allows us to seperate our code and place everything in a neat order compared to all our tasks in one playbook.
4.We can start by creating our tasks, that need to be run. This will be in the roles/splunk_server/tasks/main.yml file. So let’s start by opening that up and adding in the security group details for our new instance:
1 ---
2 - name: create the host security group
3 ec2_group:
4 name: ansible security group
5 description: security group for new host
6 region: "{{ ec2_region }}"
7 rules:
8 - proto: tcp
9 from_port: 22
10 to_port: 22
11 cidr_ip: 0.0.0.0/0
12 - proto: tcp
13 from_port: 8000
14 to_port: 8000
15 cidr_ip: 0.0.0.0/0
16 - proto: tcp
17 from_port: 443
18 to_port: 443
19 cidr_ip: 0.0.0.0/0
20 rules_egress:
21 - proto: all
22 cidr_ip: 0.0.0.0/0
23 register: basic_firewall
24If you have worked with AWS before, you will see the above code is simply adding in access to ports 22 for SSH, 443 for https access and 8000 for the Splunk web port.
5.We can now work further on our tasks file, now add the following details to launch the EC2 instance:
25 - name: launch the new ec2 instance
26 ec2:
27 group: "{{ ec2_sg_name }}"
28 instance_type: "{{ ec2_instance_type }}"
29 image: "{{ ec2_image }}"
30 wait: true
31 region: "{{ ec2_region }}"
32 keypair: "{{ ec2_keypair }}"
33 count: 1
34 user_data: "{{ lookup('file', 'user_data.sh') }}"
35 register: ec2We first refer to the security group we created in line 27, then the values in lines 28, 29, 31 and 32 are variables we will be setting up shortly. They are all properties we need to specify to set up our instance. Line 34 then lets AWS know to use the user_data file we will create shortly to install our Splunk server.
6.The tasks that we have performed so far will take a little while, so we will now add in a wait, to allow everything to install and make sure we can SSH to the host:
36 - name: wait for SSH to come up
37 wait_for:
38 host: '{{ item.public_ip }}'
39 port: 22
40 state: started
41 with_items: '{{ ec2.instances }}'
427.Finally the following tasks give the EC2 instance a tag so we can identify it, when it is set up:
43 - name: add tag to instance
44 ec2_tag:
45 resource: '{{ item.id }}'
46 region: "{{ ec2_region }}"
47 state: present
48 with_items: '{{ ec2.instances }}'
49 args:
50 tags:
51 Name: splunkserverThis is the end of all our tasks, so make sure you save the file before closing.
8.We can now create the user_data file we referred to in our tasks. Start by creating the following file:
touch roles/splunk_server/files/user_data.sh9.Open the file with your text editor and add in the following 6 lines.
1 #!/bin/bash
2 set -e -x
3 wget -O splunk-7.1.1-8f0ead9ec3db-linux-2.6-x86_64.rpm 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.1.1&product=splunk&filename=splunk-7.1.1- 8f0ead9ec3db-linux-2.6-x86_64.rpm&wget=true'
4 rpm -i splunk-7.1.1-8f0ead9ec3db-linux-2.6-x86_64.rpm
5 sleep 30
6 sudo -u splunk /opt/splunk/bin/splunk start --answer-yes --no-prompt --accept-license --seed-passwd newpasswordIt’s basically a script that will download the version of Splunk you need(line 3), install if using RPM(line 4), then at the end it will start Splunk accepting user licenses and creating your admin password as newpassword. Feel free to change anything in this, for example Splunk versions, if you prefer a Deb package and if you want to use a more complex password.
10.We can now populate our variables file. So open the roles/splunk_server/vars/main.yml file with your text editor and add in the following details:
1 ---
2 ec2_sg_name: "AnsibleSecurityGroup"
3 ec2_region: "ap-southeast-2"
4 ec2_instance_type: "t2.micro"
5 ec2_image: "ami-423bec20"
6 ec2_keypair: "key-pair-name”This is all pretty self explanatory, but our tasks need to have variables listed, all the region’s, instance type, and image type will work with the code I have provided in this post. Make sure, you have you key pair added in line 6.
10.We can now create the playbook to set up our new server. Start by creating our playbook in the base directory:
touch splunk_server.yml11.Now open the playbook and add in the following details to run our newly created role:
1 ---
2 - hosts: localhost
3 connection: local
4 gather_facts: false
5 user: root
6 roles:
7 - splunk_server12.It’s now time to run the playbook. From the command line, you can now use the ansible-playbook command to run the new role, you have created:
ansible-playbook -i hosts splunk_server.ymlMaybe go get a coffee, as Splunk does take a little while to install and start up, but if all goes well, in a few moments you should have a new server installed, running on an EC2 instance on your AWS account. You will now be able to use the Public IP address provided by AWS to then access the web interface. Simply go to your web browser and add in <PublicIP>:8000 and hopefully you will now be able to log in as the admin user with the password you specified in the user_data file.
This has been a pretty quick and dirty deployment but hopefully gives you a good idea of what you can do with Ansible. Remember, if you would like more details on working with Ansible, feel free to check out our new book:
Found this post useful? Kindly tap the clap button below! :)
About The Author
