How to deploy from Gitlab to Elastic Beanstalk (AWS)
Gitlab has been getting a lot of traction the last months even Github has changed in pricing model for private repositories. At the moment key player for Git repository hosting are Github and Bitbucket while Gitlab is popular since it is offering a self-hosted edition (CE) and free for public and unlimited private repositories on-premise as well as in their cloud option Gitlab.com.
Assuming you are hosting your application in AWS using Elastic Beanstalk and your Git repositories sit in Gitlab, so how do you setup continuous integration and deployment into AWS Elastic Beanstalk?
Quick steps (for experts):
- Create your repository in Gitlab
- Create Elastic Beanstalk app in AWS
- Create IAM user with custom policy and Elastic Beanstalk permissions
- Add variables to Gitlab with your IAM user credentials
- Opt for shared or your own Gitlab CI runner
- Add .gitlab-ci.yml to root of project
- Run eb init
- Create folder .ebextensions
- Define environments in Gitlab pipelines
- Push and let it fly
or take a look into my sample repositories:
Source:
https://gitlab.com/meixnertobias/funwitheb
Docker image for CI runner:
https://hub.docker.com/r/meixnertobias/dockerpipawscli/
1. Create your repository in Gitlab/Gitlab.com
Start your development on your app by creating a repository in Gitlab/Gitlab.com as public, internal or private repository as you like. Clone it to your machine and start building as you wish. Whatever you build should not relate to the CI/CD setup later, just make sure your app is supported by Elastic Beanstalk such as Docker, PHP, Go, NodeJS and so on.
2. Create Elastic Beanstalk app
As a next step you want to create a Elastic Beanstalk app in AWS Console.
For that select Elastic Beanstalk under the Compute services in AWS.
Select a name for your app, select web server environment, follow other steps in configuration on your application.
See a sample below for a simple PHP app on a t2.micro instance without load balancing/auto-scaling:
3. Create IAM user with full-access Elastic Beanstalk permission
Next you should create an IAM user under the IAM service in AWS.
Give it a meaningful name and follow the steps by downloading the credentials. (Keep them safe and don’t share them)
After the user is created — select the user and hit the tab “Permissions” and select “Attach Policy”
Search for “elastic” and select “AWSElasticBeanstalkFullAccess” (Use with caution if you use in your organization, this is a sample for private use-cases)
Note that if you have multiple apps in Elastic Beanstalk or different rights on deployment in your organization you should create separate users for each environment or application.
You may also restrict the permissions to certain services since AWSElasticBeanstalkFullAccess has full access to all services for provisioning.
In best case you may specify an the Elastic Beanstalk ID/ARN in the policy and restrict the user to deployment actions only but it depends on how much you will interact with your EB environment during Ci/CD.
At the end your user have your customer or the “AWSElasticBeanstalkFullAccess” attached.
This is required to have the user able later to deploy the application into AWS using the user credentials.
4. Add variables to Gitlab
For deployment into Elastic Beanstalk environment in AWS you can use the IAM user you have just created.
Since you should not store your IAM credentials in your source or repository you can use “Variables” provided by Gitlab which are available during build process for your runner.
As a next step you go to your project in Gitlab in menu on right select “Variables”
Create 2 new variables according to your IAM user credentials you created before in step 3.
You can name them however you like but you will use them later in your deployment configuration.
In my case I call them:
AWS_SECRET_ACCESS_KEY
AWS_ACCESS_KEY_ID
5. Opt for shared or your own runner
Continuous integration normally includes certain steps such as automated test, configuration, make/build of an artifact etc. — for that you require a runner that runs all of your integration steps.
Here it depends if you are using the Community Edition in your own environment or gitlab.com.
In the Community Edition you need to make sure you have Gitlab CI installed and a runner installed as well. More information here about Gitlab CI .
If you are using gitlab.com there is a shared runner by default which you can use. Note this runner is shared with other users that means it may mean you have to wait in a que until another user’s build has completed, cannot scale and also mean that you may not use the shared runner builds including variables which may be exposed in a shared runner.
So for gitlab.com you should be using your own runner either on your machine or your integration server/machine. How to setup your own runner you can follow this tutorial.
I will shortly explain how to setup a runner in AWS as a quick sample.
5a. Launch an EC2 instance in AWS (I use Amazon Linux AMI)
5b. SSH into your new CI server and install Gitlab multi-runner
5c. Run commands as per Gitlab tutorial in my case I will use docker runner
https://gist.github.com/MeixnerTobias/3d8bbc723454649399faf8fde10e424a
~/.gitlab-runner/config.toml
https://gist.github.com/MeixnerTobias/93025793f1a0ebde61fbc8569fa30a34
5d. Disable shared runner and check if your runner is activated for this project
6. Add .gitlab-ci.yml to root of project
Next we should configure continuous integration for our Gitlab repository. We enable CI by adding a .gitlab-ci.yml in our project root directory.
Inside the gitlab-ci.yml file we need to define what we would like our runner to do with our source on push or tag in git.
In our sample let’s assume we do not have any actual continuous integration but rather a deployment to Elastic Beanstalk on every push to master branch.
https://gist.github.com/MeixnerTobias/db5b477279f585b2b19e1b876633f9ae
In our file we define the AWS credentials in our docker runner based on the variables from Gitlab we set before. After we use AWS CLI command eb deploy to deploy our application as a new version into AWS.
The configuration file can be use for test, different test steps or define actions by branch etc. More information on configuration can be found here.
7. Run eb init
Open your command line or terminal and run
eb init
Running eb init will create a config.yml in the .elasticbeanstalk folder as below:
https://gist.github.com/MeixnerTobias/197ff7276f1fb4eac844b65d521df0c9
8. Create folder .ebextensions
In case you need specific configuration for provisioning of your Apache or PHP for example you can use bash, python or specific configuration files to modify the default configuration of Elastic Beanstalk.
For that create a folder .ebextensions.
For Linux and EC2 you can follow samples from here to define Linux environments.
We create a file apache.config in the .ebextensions folder as below:
https://gist.github.com/MeixnerTobias/9b308444882f0d7506533c8a1233325d
During deployment EB CLI will create a Apache configuration file that loads the rewrite module into Apache as a result.
So you can use any command you can think of for provisioning and all under version control in Git.
As a result your project folder may look like that:
9. Define environments in Gitlab pipelines
Now our project folder is all set and for better overview in the future in if would like to have different environment such as production, staging or development you can define different environments within Gitlab too.
10. Push and let it fly
Now on your next push into master to Gitlab it should trigger a build and following a deployment into AWS Elastic Beanstalk.
If all goes well under your Pipelines in Gitlab your build should take around 40 seconds looking with an output as below:
Congratulations you now have continuous integration and deployment setup with Gitlab and AWS Elastic Beanstalk.
Should you have any issues with the setup please let me know in the comments or contact me directly through the contact form.
Note this setup is great if you have a simple PHP, Go or NodeJS app. This can be perfectly scaled as well for different environment such as development, staging and production. For that you need to extend your .elasticbeanstalk/config.yml and link your branch with your Elastic Beanstalk environments. Additionally .gitlab-ci.yml to be extend to different environments or triggers on tags if you are using Gitflow workflow for example.
One final note — remember to use proper IAM user with custom policies and keep your credentials safe (do not share or store in your repositories).
Thank you.
