Automation with Ansible: Install ELK Stack on Centos7
Introduction
In the world of modern IT infrastructure, automation plays a crucial role in simplifying and streamlining complex tasks. Ansible, a popular open-source automation tool, provides a simple and efficient way to automate various processes. One such task is installing the ELK (Elasticsearch, Logstash, and Kibana) Stack on CentOS 7, which empowers organizations to effectively manage and analyze their logs and data. In this topic block, we will explore the power of automation with Ansible and demonstrate how to automate the installation of the ELK Stack on CentOS 7.
Prerequisites
Before we begin, you should have the following:
- System requirements for running the ELK Stack
- A CentOS 7 server with Ansible installed
- Basic knowledge of ELK Stack, Ansible and YAML syntax
Architecture
Ansible Playbooks for ELK Stack Installation
Step 1: Create an Ansible inventory file
Create an inventory file with the list of CentOS 7 servers that will be part of the Elasticsearch cluster. You can use the IP address or hostname of the servers in the inventory file.
For example:
[elk-server]
10.0.2.214 ansible_ssh_user=root ansible_ssh_pass=nkt&uqrC
10.0.2.215 ansible_ssh_user=root ansible_ssh_pass=qKCuLf3(
10.0.2.216 ansible_ssh_user=root ansible_ssh_pass=Y2A%8NBs
[master-elk-server]
10.0.2.214 ansible_ssh_user=root ansible_ssh_pass=nkt&uqrC
[join-elk-server]
10.0.2.215 ansible_ssh_user=root ansible_ssh_pass=qKCuLf3(
10.0.2.216 ansible_ssh_user=root ansible_ssh_pass=p@ssw0rd
[kibana-server]
10.0.2.214 ansible_ssh_user=root ansible_ssh_pass=nkt&uqrC
10.0.2.215 ansible_ssh_user=root ansible_ssh_pass=qKCuLf3(
[logstash-server]
10.0.2.214 ansible_ssh_user=root ansible_ssh_pass=nkt&uqrC
10.0.2.215 ansible_ssh_user=root ansible_ssh_pass=qKCuLf3(
# If server have keygen can remove variable ansible_ssh_pass.Step 2: Create an Ansible playbook
Create a file named elk.yml and add the following content.
- name: Start Provision ELK Stack
hosts: elk-server
become: yes
vars:
elasticsearch_config: |
network.host: {{ ansible_host }}
discovery.seed_hosts: [ "10.0.2.214", "10.0.2.215", "10.0.2.216" ] # Enter your Elasticsearch server's IP address or hostname.
kibana_config: |
elasticsearch.hosts: [ "https://10.0.2.214:9200", "https://10.0.2.215:9200", "https://10.0.2.216:9200" ] # Enter your Elasticsearch server's IP address or hostname.
server.host: "{{ ansible_host }}"
server.port: 5601
elasticsearch.ssl.verificationMode: none
kibana_ssl_config: |
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/kibana-cert.crt
server.ssl.key: /etc/kibana/certs/kibana-key.key
# If you do not use Logstash, you can skip the variable logstash_config.
logstash_config: |
xpack.management.enabled: true
xpack.management.pipeline.id: ["main"]
xpack.management.elasticsearch.username: elastic
xpack.management.elasticsearch.password: {{ elastic_password }}
xpack.management.elasticsearch.hosts: [ "https://10.0.2.214:9200", "https://10.0.2.215:9200", "https://10.0.2.216:9200" ] # Enter your Elasticsearch server's IP address or hostname.
xpack.management.elasticsearch.ssl.certificate_authority: "/etc/logstash/certs/http_ca.crt"
xpack.management.elasticsearch.ssl.verification_mode: none
# Can change password if desired.
elastic_password: passw0rd
# Can change version ELK stack if desired.
elk_version: 8.7.1
tasks:
- name: Download Elasticsearch RPM
get_url:
url: https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-{{ elk_version }}-x86_64.rpm
dest: /tmp/elasticsearch.rpm
- name: Install Elasticsearch RPM
yum:
name: /tmp/elasticsearch.rpm
state: present
- name: Config Elasticsearch
blockinfile:
path: /etc/elasticsearch/elasticsearch.yml
block: "{{ elasticsearch_config }}"
marker: "# {mark} Elasticsearch configuration"
backup: yes
when: inventory_hostname in groups["master-elk-server"]
- name: Start Elasticsearch service group master-elk-server
service:
name: elasticsearch
state: started
when: inventory_hostname in groups["master-elk-server"]
- name: Run password reset
command: "/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i"
args:
stdin: "y\n{{ elastic_password }}\n{{ elastic_password }}\n"
when: inventory_hostname in groups["master-elk-server"]
- name: Display the elastic cluster password.
debug:
msg: "New password: {{ elastic_password }}"
when: inventory_hostname in groups["master-elk-server"]
- name: Generate enrollment token for Elasticsearch nodes.
shell: /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node
register: elastic_token_node
changed_when: elastic_token_node.stdout_lines[0] != 'password changed'
when: inventory_hostname in groups["master-elk-server"]
- name: Keep Elasticsearch tokens on different servers.
copy:
content: "{{ elastic_token_node.stdout }}"
dest: /etc/elasticsearch/elastic_token_node.txt
loop: "{{ groups['join-elk-server'] }}"
when:
- inventory_hostname in groups["master-elk-server"]
- inventory_hostname != item
delegate_to: "{{ item }}"
- name: set a variable using a file.
shell: cat /etc/elasticsearch/elastic_token_node.txt
register: elastic_token
when: inventory_hostname in groups["join-elk-server"]
- name: Register/Enrolling elasticsearch
shell: echo "y" | /usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token {{ elastic_token.stdout }}
when: inventory_hostname in groups["join-elk-server"]
- name: Delete line discovery.seed_hosts
lineinfile:
path: /etc/elasticsearch/elasticsearch.yml
regexp: '^discovery.seed_hosts:'
line: '# delete line discovery.seed_hosts'
when: inventory_hostname in groups["join-elk-server"]
- name: Config Elasticsearch
blockinfile:
path: /etc/elasticsearch/elasticsearch.yml
block: "{{ elasticsearch_config }}"
marker: "# {mark} Elasticsearch configuration"
backup: yes
when: inventory_hostname in groups["join-elk-server"]
- name: Reload systemd daemon on Elasticsearch group join-elk-server
command: systemctl daemon-reload
when: inventory_hostname in groups["join-elk-server"]
- name: Start Elasticsearch service group join-elk-server
service:
name: elasticsearch
state: started
when: inventory_hostname in groups["join-elk-server"]
########################################################
# (optional) Activate if required.
# - name: Run curl Enable license elasticsearch cluster
# shell: "curl -kv -XPOST -u elastic:{{ elastic_password }} 'https://{{ ansible_host }}:9200/_license/start_trial?acknowledge=true&pretty'"
# register: curl_license_output
# when: inventory_hostname in groups["master-elk-server"]
#
# - name: Display license output
# debug:
# var: curl_license_output.stdout_lines
# when: inventory_hostname in groups["master-elk-server"]
########################################################
- name: Download Kibana RPM
get_url:
url: https://artifacts.elastic.co/downloads/kibana/kibana-{{ elk_version }}-x86_64.rpm
dest: /tmp/kibana.rpm
when: inventory_hostname in groups["kibana-server"]
- name: Install kibana RPM
yum:
name: /tmp/kibana.rpm
state: present
when: inventory_hostname in groups["kibana-server"]
- name: Generate enrollment token for Kibana nodes.
shell: /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
register: kibana_token_node
when: inventory_hostname in groups["master-elk-server"]
- name: Display the Kibana token
debug:
msg: "Kibana Token: {{ kibana_token_node.stdout }}"
when: inventory_hostname in groups["master-elk-server"]
########################################################
# (optional) Enable If Kibana only has 1 node.
#
# - name: Register/Enrolling kibana
# shell: /usr/share/kibana/bin/kibana-setup --enrollment-token {{ kibana_token_node.stdout }}
# when: inventory_hostname in groups["kibana-server"]
########################################################
########################################################
# Enable If kibana have more than 1 nodes
- name: Keep Kibana tokens on different servers.
copy:
content: "{{ hostvars[groups['kibana-server'][0]]['kibana_token_node']['stdout'] }}"
dest: /etc/kibana/kibana_token_node.txt
loop: "{{ groups['kibana-server'] }}"
when:
- inventory_hostname in groups["kibana-server"]
- inventory_hostname != item
delegate_to: "{{ item }}"
- name: set a variable using a file.
shell: cat /etc/kibana/kibana_token_node.txt
register: kibana_token
when: inventory_hostname in groups["kibana-server"]
- name: Register/Enrolling kibana
shell: /usr/share/kibana/bin/kibana-setup --enrollment-token {{ kibana_token.stdout }}
when: inventory_hostname in groups["kibana-server"]
########################################################
- name: Delete line elasticsearch.hosts
lineinfile:
path: /etc/kibana/kibana.yml
regexp: '^elasticsearch.hosts:'
line: '#'
when: inventory_hostname in groups["kibana-server"]
- name: Config Kibana
blockinfile:
path: /etc/kibana/kibana.yml
block: "{{ kibana_config }}"
marker: "# {mark} Kibana configuration"
backup: yes
when: inventory_hostname in groups["kibana-server"]
########################################################
# (optional) Enable SSL if required.
# - name: Generate self-signed certificate
# command: "/usr/share/elasticsearch/bin/elasticsearch-certutil cert -name kibana-server --self-signed"
# args:
# stdin: "kibana-server.p12\n\n"
# when: inventory_hostname in groups["master-elk-server"]
#
# - name: Create directory /etc/kibana/certs
# file:
# path: /etc/kibana/certs
# state: directory
# when: inventory_hostname in groups["kibana-server"]
#
# - name: Copy file to Ansible control machine
# fetch:
# src: /usr/share/elasticsearch/kibana-server.p12
# dest: /tmp/kibana-server.p12
# flat: yes
# fail_on_missing: yes
# when: inventory_hostname in groups["master-elk-server"]
#
# - name: Copy file from Ansible control machine to logstash server
# copy:
# src: /tmp/kibana-server.p12
# dest: /etc/kibana/certs/kibana-server.p12
# when: inventory_hostname in groups["kibana-server"]
#
# - name: Extract certificate from PKCS12 file
# command: "openssl pkcs12 -in /etc/kibana/certs/kibana-server.p12 -out /etc/kibana/certs/kibana-cert.crt -nokeys -passin pass:"
# when: inventory_hostname in groups["kibana-server"]
#
# - name: Extract private key from PKCS12 file
# command: "openssl pkcs12 -in /etc/kibana/certs/kibana-server.p12 -out /etc/kibana/certs/kibana-key.key -nodes -nocerts -passin pass:"
# when: inventory_hostname in groups["kibana-server"]
#
# - name: Config Kibana
# blockinfile:
# path: /etc/kibana/kibana.yml
# block: "{{ kibana_ssl_config }}"
# marker: "# {mark} Kibana ssl configuration"
# backup: yes
# when: inventory_hostname in groups["kibana-server"]
########################################################
- name: Start kibana service
service:
name: kibana
state: started
when: inventory_hostname in groups["kibana-server"]
- name: Download Logstash RPM
get_url:
url: https://artifacts.elastic.co/downloads/logstash/logstash-{{ elk_version }}-x86_64.rpm
dest: /tmp/logstash.rpm
when: inventory_hostname in groups["logstash-server"]
- name: Install logstash RPM
yum:
name: /tmp/logstash.rpm
state: present
when: inventory_hostname in groups["logstash-server"]
########################################################
# (optional) Enable If Elasticsearch cluster Activate License.
# - name: Config Logstash
# blockinfile:
# path: /etc/logstash/logstash.yml
# block: "{{ logstash_config }}"
# marker: "# {mark} Logstash configuration"
# backup: yes
# when: inventory_hostname in groups["logstash-server"]
#
# - name: Create directory /etc/logstash/certs
# file:
# path: /etc/logstash/certs
# state: directory
# when: inventory_hostname in groups["logstash-server"]
#
# - name: Copy file to Ansible control machine
# fetch:
# src: /etc/elasticsearch/certs/http_ca.crt
# dest: /tmp/http_ca.crt
# flat: yes
# fail_on_missing: yes
# when: inventory_hostname in groups["master-elk-server"]
#
# - name: Copy file from Ansible control machine to logstash server
# copy:
# src: /tmp/http_ca.crt
# dest: /etc/logstash/certs/http_ca.crt
# when: inventory_hostname in groups["logstash-server"]
#
# - name: Disable lines in file
# replace:
# path: /etc/logstash/pipelines.yml
# regexp: '^(.*)$'
# replace: '#\1'
# backup: yes
# when: inventory_hostname in groups["logstash-server"]
#
# - name: Start logstash service
# service:
# name: logstash
# state: started
# when: inventory_hostname in groups["logstash-server"]
########################################################Step 3: Run the Ansible
Run the following command to execute the Ansible playbook:
ansible-playbook elk.ymlStep 4: Verify Elasticsearch cluster on Kibana
To access Kibana, you typically need to open a web browser and enter the URL where Kibana is hosted. The default URL for Kibana is https://localhost:5601, but you might need to adjust it based on your specific configuration.
A. Open a web browser on your device.
B. Enter the URL for Kibana in the address bar. For example, https://localhost:5601.
C. Press Enter to load the Kibana login page.
D. Press Enter your <username> and <password>.
E. Once the Kibana interface loads, locate the navigation menu on the left side of the page. It typically contains various options such as “Discover,” “Visualize,” “Dashboard,” etc.
F. Scroll down the menu and find the “Dev Tools” option. It is usually located under the “Management” section.
G. Click on “Dev Tools” to access the Kibana Dev Tools page.
H. Run the “GET _cat/nodes?v” command in devtools and review the output.
Summary
By following this guide, you will be able to easily install the ELK Stack on CentOS 7. Using Ansible helps save time and enables you to test the installation of the ELK Stack on multiple servers. If you have any questions or need further clarification on this topic, please feel free to comment below.
We’re hiring, Anyone interested in getting in touch with us can do so at https://lin.ee/ms8vit4.
👉🏻 Follow other company news at the channel below.
Instagram: https://www.instagram.com/sirisoft_official
Tiktok: https://www.tiktok.com/@sirisoft
Youtube: https://www.youtube.com/@sirisoftofficial1698
FB: https://www.facebook.com/sirisoft
Website: https://www.sirisoft.co.th
