A guide to LocalSTack with docker-compose

LocalStack provides an excellent environment for developers to test and experiment with AWS services locally. In this step-by-step guide, I’ll walk you through the process of setting up LocalStack using Docker Compose, creating an S3 bucket, and uploading data. By following these simple steps, you can harness the power of AWS S3 without the need for actual cloud resources. refer here for a less technical article that dives into what localstack and s3 buckets are as well as a guide to troubleshooting and asking for help.

Step 1: Install Docker and Docker Compose

Before we begin, ensure you have Docker and Docker Compose installed on your system. If not, follow the official Docker documentation for installation instructions here

Step 2: Prepare Docker Compose File

Create a new directory for your LocalStack project and create a docker-compose.yml file. Below is an example docker-compose.yml file, I will take you through each of the lines and the necessary commands.

mkdir localstacktest

cd localstacktest

touch docker-compose.yml

version: ‘3
services:
   localstack:
       image: localstack/localstack:1.0.4
 ports:
      - "4567:4566"
    environment:
      - SERVICES=s3:4566
      - HOSTNAME=localstack
      - HOSTNAME_EXTERNAL=localstack
      - DEFAULT_REGION=us-east-2
  aws-cli:
    image: amazon/aws-cli
    depends_on:
      - localstack
    volumes:
      - ./testbucket:/aws
    environment:
      - AWS_ACCESS_KEY_ID=test
      - AWS_SECRET_ACCESS_KEY=test
      - AWS_DEFAULT_REGION=us-east-2
    entrypoint: /bin/sh -c
    command: >
      "
        aws --endpoint-url=http://localstack:4566 s3api create-bucket --bucket my-bucket --region us-east-1
        aws --endpoint-url=http://localstack:4566 s3 cp testData.csv s3://cbk-data-bucket//testData.csv
        aws --endpoint-url=http://localstack:4566 s3 cp testDataOne.csv s3://cbk-data-bucket//testDataTwo.csv
        aws --endpoint-url=http://localstack:4566 s3 cp testDataTwo.csv s3://cbk-data-bucket//testDataTwo.csv
        aws --endpoint-url=http://localstack:4566 s3 cp testDataThree.csv s3://cbk-data-bucket//testDataThree.csv
      "

let us go through the file step by step to get an in depth understanding of what is happenning.

version:'3' this specifies the version of the Docker Compose configuration file format, in this case we are using version 3.

services This defines the services that will be executed on running the docker-compose up command.

localstack this is the first defined service, it uses localstack docker image version 1.0.4 and defines port 4567 which is the port the localstack service will run on inside the container and map to port 4566 on the host machine. feel free to read up on docker, images and ports I will not delve deep as this is a local stack set up guide.

environment here, we set the environment variables for the localstack service. lets explore the env vars a bit further.

SERVICES=S3:4566 : This specifies the AWS service to start in local stack, which is s3 in our case and will be running on port 4566

HOSTNAME=localstack This sets the hostname inside the container to ‘localstack’

HOSTNAME_EXTERNAL=localstack sets the external hostname to ‘localstack’

DEFAULT_REGION=us-east-2 : This sets the default AWS regions in localstack to us-east-2

AWS CLI-service The second defined serive is named aws-cli which uses the official aws cli docker image. This sets up the aws cli service. the depends_on section ensures that the cli service starts after the localstack service is already up and running.

volumes section mounts the local ./testbucket containing our csv files, to /aws inside the aws-cli container. This allows data to be shared between the host machine and the container. The environment section, just as in locastack servive sets environment vars for the aws-cli service. note that you do not need real values you can go ahead and configure dummy env vars.

entrypoint the entrypoint and command sections specifies the commands to be run once the aws-cli service is up. the command is a multiline aws-cli commands, in our case the commands created a bucket named my-bucket and copies four CSV files (testData.csv , testDataOne.csv testDataTwo.csv, testDataThree.csv, and repayments.csv) from the testbucket to the bucket my-bucket

Step 3: Launch LocalStack

Open a terminal or command prompt, navigate to the directory containing your docker-compose.yml file, and execute the following command:

docker-compose up

Docker Compose will now download the LocalStack image and launch the services specified in the docker-compose.yml file. you can add commands like ls and pwd to the comand section to confirm that the files have been uploaded successfuly.

Conclusion

This docker-compose.yml file sets up LocalStack with S3 services on port 4566. It also starts an AWS CLI container that communicates with LocalStack to create an S3 bucket and upload CSV files to it. By running this Docker Compose configuration, you can easily simulate AWS S3 functionality locally for development and testing purposes.