Building an internally hosted Repo Server — Part 2

First of all, thanks to the contributors of this role Ansible Role: Nexus 3 OSS. This is the base role used to help us deploy nexus. At the moment our Nexus deployment is split into two Ansible Roles.

The first one is named nexus3-oss named “ansible-role-nexus3-oss” and can be found at the link above.

The second Ansible role (ansible-role-yumrepo) we use to create our internal repo server. The two roles install exactly the same Nexus Package Server, however our version of the role adds the necessary configuration on top of the initial build.

There are a number of external repositories we want to synchronise. The first is a repo with all the RHEL rpm packages (nearly 80 gb), the others include ClamAV, and a number of Open Source software packages used in other build pipelines.

The YAML file our Packer pipeline uses to build the AMI is install.yml. This file can be found in yumrepo/install/packer. An example file configuration has been provided below:

You can use this YAML to configure Nexus password, LDAP, Nexus privileges, Nexus roles and any local user configurations etc. You can also specify the number of repositories you would like to create. In our example below, you will see we create three repositores YumRepo, ClamAV and Triage.

The last part of our YAML calls the main nexus role ansible-role-nexus3-oss which points to the playbook that builds the whole lot!

The main.yml in the task directory is used to drive the role. The css_repo_upload.rml file in the tasks directory is called by main.yml and this playbook is responsible for the upload to the repos we have specified in packer install.yml.

The css_repo_upload.yml calls the manifest.yml file in vars directory which holds any external links and variables required.

If you don’t want to use the default 8081 port to access your instance of Nexus, you’ll need to amend the following YML file and update the regexp with your chosen port. This will modify the iptables configuration to allow traffic in on the desired port (iptables is enabled by default on CIS hardened RHEL images).

Finally you want to update the manifest file that uses the external link to synchronise with the internal link on your Nexus image. The link to this role can be found here.

The content of the css_repo_upload.yml is provided below:

- name : inmport variables file
 include_vars: manifest.yml
- name: install pip
 easy_install:
 name: pip
 state: latest
- name: install boto
 pip:
 name:
 — boto
 — boto3
 — botocore
- name: Install the “yum-utils” and “createrepo” packages
 yum:
 name:
 — yum-utils
 — createrepo
 state: latest
- name: synch RedhatInfraClientConfigServer RedHatEnterpriseLinuxServer RedHatEnterpriseServerCommon epel
 shell: | 
 reposync — gpgcheck -l — repoid=rhui-REGION-rhel-server-rh-common — download_path=/tmp
 reposync — gpgcheck -l — repoid=rhui-REGION-rhel-server-releases — download_path=/tmp
 reposync — gpgcheck -l — repoid=rhui-REGION-client-config-server-7 — download_path=/tmp
 reposync — gpgcheck -l — repoid=epel — download_path=/tmp
###################################### ALL yum repo ####################
- name: unset proxy
 shell: |
 unset http_proxy
 unset https_proxy
- name: deploy to rhel-server-rh-common
 shell: |
 for d in /tmp/rhui-REGION-rhel-server-rh-common/Packages/* ; do
 base1=`basename $d subset.fq`
 cd “$d” && FILES=”$d”/* 
 for f in $FILES
 do
 base=`basename $f subset.fq`
 curl — upload-file “${base}” -u admin:’{{nexus_admin_password}}’ \
 -v “http://localhost:8081/repository/rhel-server-rh-common/${base1}/${base}"
 done 
 done
- name: deploy to rhel-server-releases
 shell: |
 for d in /tmp/rhui-REGION-rhel-server-releases/Packages/* ; do
 base1=`basename $d subset.fq`
 cd “$d” && FILES=”$d”/* 
 for f in $FILES
 do
 base=`basename $f subset.fq`
 curl — upload-file “${base}” -u admin:’{{nexus_admin_password}}’ \
 -v “http://localhost:8081/repository/rhel-server-releases/${base1}/${base}"
 done 
 done
- name: deploy to epel
 shell: |
 for d in /tmp/epel/Packages/* ; do
 base1=`basename $d subset.fq`
 cd “$d” && FILES=”$d”/* 
 for f in $FILES
 do
 base=`basename $f subset.fq`
 curl — upload-file “${base}” -u admin:’{{nexus_admin_password}}’ \
 -v “http://localhost:8081/repository/epel/${base1}/${base}"
 done 
 done
- name: Populating {{ client_config_server_7_url }} repo with rpm 
 shell: |
 cd /tmp/rhui-REGION-client-config-server-7 
 FILES=/tmp/rhui-REGION-client-config-server-7/*
 for f in $FILES
 do
 base=`basename $f subset.fq`
 curl — upload-file “${base}” -u admin:’{{nexus_admin_password}}’ \
 -v “http://localhost:8081/repository/client-config-server-7/${base}"
 done
- name: Create nginx directory if it does not exist
 file:
 path: /tmp/nginx
 state: directory
 mode: ‘0755’
- name: download nginx packages
 shell: |
 cd /tmp/nginx
 wget {{ css_nginx_url }}/{{ item }}
 loop: “{{ css_nginx_url_item }}”
- name : Populating nexus nginx repo with CSS packages
 shell: | 
 cd /tmp/nginx
 curl -v -u admin:{{nexus_admin_password}} — upload-file {{ item }} {{ css_nexus_nginx_repo }}
 loop: “{{ css_nginx_url_item }}”
- name: Create elasticsearch directory if it does not exist
 file:
 path: /tmp/elasticsearch
 state: directory
 mode: ‘0755’
- name: download elasticsearch packages
 shell: |
 cd /tmp/elasticsearch
 wget {{ css_elasticsearch_url }}/{{ item }}
 loop: “{{ css_elasticsearch_url_item }}”
- name : Populating nexus elasticsearch repo with CSS packages
 shell: | 
 cd /tmp/elasticsearch/
 curl -v -u admin:{{nexus_admin_password}} — upload-file {{ item }} {{ css_nexus_elasticsearch_repo }}
 loop: “{{ css_elasticsearch_url_item }}”
- name: Removing content of /etc/yum.repo.d boostrap will install the config file
 file:
 path: /etc/yum.repos.d
 state: absent
- name: Recreate yum directory
 file:
 path: /etc/yum.repos.d
 state: directory
 mode: ‘0755’
- name: shell script to remove contents of tmp
 shell: rm -rf /tmp/*

Thanks to Fathi Mikadoor