Software Architecture and Deployment using Docker

Learning new things on DevOps

Source: Google Images

DevOps is a software development approach which involves continuous development, continuous testing, continuous integration, continuous deployment, and continuous monitoring of the software throughout its development lifecycle. This is the process adopted by all the top companies to develop high-quality software and shorter development lifecycles.

Software Architecture

Source: Google Images

Architecture is the foundation that holds everything together. Using the immense resources given to us by PPL and the product owner. We use Docker to set up our architecture.

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.

Container images become containers at runtime and in the case of Docker containers — images become containers when they run on Docker Engine. Available for both Linux and Windows-based applications, containerized software will always run the same, regardless of the infrastructure. Containers isolate software from its environment and ensure that it works uniformly despite differences for instance between development and staging.

Docker containers that run on Docker Engine:

• Standard: Docker created the industry standard for containers, so they could be portable anywhere
• Lightweight: Containers share the machine’s OS system kernel and therefore do not require an OS per application, driving higher server efficiencies and reducing server and licensing costs
• Secure: Applications are safer in containers and Docker provides the strongest default isolation capabilities in the industry

Here we use the link 152.118.2021.222 at port 24150 for development and 24100 for staging.

Api Root - Django REST framework

Dockerfile

# Dockerfile
# FROM directive instructing base image to build upon
FROM python:3-onbuild
WORKDIR /app
COPY . /app
EXPOSE 8000
CMD ["./start.sh"]

start.sh

#!/bin/bash
python manage.py makemigration
python manage.py migrate
python manage.py populatedb2
# Start Gunicornn  processes
echo Starting Gunicorn.
exec gunicorn api.wsgi:application \
    --bind 0.0.0.0:8000 \
    --workers 3

Deployment

Source: Google Images

CI stands for Continuous Integration and CD stands for Continuous Delivery/Continuous Deployment. You can think of it as a process similar to a software development lifecycle. In this project, we use the CI/CD for 2 main things. First is for the mobile app APK so that it can be built and deploy automatically for testing and using. Second is for the API we are using Django Rest Framework for working around the databases and models of the content.

We set up the deployment on the gitlab-ci.yml file for the CI/CD of our project, Scrum Booster.

stages:
#  - linter
  - test
  - deploy
  - show

variables:
  DOCKER_IMAGE_NAME: "registry.docker.ppl.cs.ui.ac.id/ppld1"
  APP_NAME: "scrum-booster"

#FlutterLint:
#  stage: linter
#  image: jro7/flutter_lcov
#  script:
#    - flutter analyze
#  except:
#    changes:
#    - scrum-booster-api/**/*

FlutterTest:
  stage: test
  image: jro7/flutter_lcov
  script:
  - echo Testing $APP_NAME
  - flutter doctor -v
  - flutter test --coverage
  - lcov --summary coverage/lcov.info
  - genhtml coverage/lcov.info --output=coverage
  coverage: '/lines......: \d+\.\d+\%/'
  artifacts:
    name: mobile-coverage
    paths:
    - $CI_PROJECT_DIR/coverage

BackendTest:
  stage: test
  image: python:3.6
  only:
    changes:
      - scrum-booster-api/**/*
  before_script:
    - cd scrum-booster-api
    - pip install -r requirements.txt
    - python manage.py makemigrations
    - python manage.py migrate
    - python manage.py collectstatic --no-input
    - python manage.py runserver 8000 &
  when: on_success
  script:
    - coverage run manage.py test
    - coverage report -m

BackendDeployToProduction:
  variables:
    ENV_NAME: "production"
  stage: deploy
  image: gitlab/dind:latest
  only:
    changes:
      - scrum-booster-api/**/*
    refs:
      - master
  before_script:
    - docker info
  script:
    - DOCKER_IMAGE_TAG=$DOCKER_IMAGE_NAME/scrumboosterapi/$ENV_NAME
    - docker build -t $DOCKER_IMAGE_TAG ./scrum-booster-api
    - docker push $DOCKER_IMAGE_TAG
  tags:
    - docker
    - build
  environment:
    name: production

BackendDeployToStaging:
  variables:
    ENV_NAME: "staging"
  stage: deploy
  image: gitlab/dind:latest
  only:
    changes:
      - scrum-booster-api/**/*
    refs:
      - staging
  before_script:
    - docker info
  script:
    - DOCKER_IMAGE_TAG=$DOCKER_IMAGE_NAME/scrumboosterapi/$ENV_NAME
    - docker build -t $DOCKER_IMAGE_TAG ./scrum-booster-api
    - docker push $DOCKER_IMAGE_TAG
  tags:
    - docker
    - build
  environment:
    name: staging

BackendDeployToDevelopment:
  variables:
    ENV_NAME: "development"
  stage: deploy
  image: gitlab/dind:latest
  only:
    changes:
      - scrum-booster-api/**/*
    refs:
      - /^US-.*$/
  before_script:
    - docker info
  script:
    - DOCKER_IMAGE_TAG=$DOCKER_IMAGE_NAME/scrumboosterapi/$ENV_NAME
    - docker build -t $DOCKER_IMAGE_TAG ./scrum-booster-api
    - docker push $DOCKER_IMAGE_TAG
  tags:
    - docker
    - build
  environment:
    name: development

FlutterDeployToProduction:
  variables:
    ENV_NAME: "production"
  image: runmymind/docker-android-sdk:latest
  stage: deploy
  tags:
    - docker
    - build
  only:
    refs:
      - master
  before_script:
    - wget --quiet --output-document=flutter.tar.xz https://storage.googleapis.com/flutter_infra/releases/stable/linux/flutter_linux_v1.0.0-stable.tar.xz && tar xf flutter.tar.xz -C /
    - export PATH=$PATH:/flutter/bin
    - apt-get update && apt-get install gnupg -y
    - curl -sL https://deb.nodesource.com/setup_11.x | bash - && apt-get install -y nodejs
    - flutter doctor -v
    - flutter packages get
    - npm install --global appcenter-cli
  script:
    - flutter build apk --release --build-name=$APP_NAME --build-number=1
    - appcenter
    - appcenter login --token $APP_CENTER_API_KEY
    - appcenter distribute release -f build/app/outputs/apk/release/app-release.apk --app $APP_CENTER_APP_NAME_PROD --group $APP_CENTER_GROUP_TARGET
  environment:
    name: production

FlutterDeployToStaging:
  variables:
    ENV_NAME: "staging"
  image: runmymind/docker-android-sdk:latest
  stage: deploy
  tags:
    - docker
    - build
  only:
    refs:
      - staging
  before_script:
    - wget --quiet --output-document=flutter.tar.xz https://storage.googleapis.com/flutter_infra/releases/stable/linux/flutter_linux_v1.0.0-stable.tar.xz && tar xf flutter.tar.xz -C /
    - export PATH=$PATH:/flutter/bin
    - apt-get update && apt-get install gnupg -y
    - curl -sL https://deb.nodesource.com/setup_11.x | bash - && apt-get install -y nodejs
    - flutter doctor -v
    - flutter packages get
    - npm install --global appcenter-cli
  script:
    - flutter build apk --release --build-name=$APP_NAME-$ENV_NAME --build-number=1
    - appcenter
    - appcenter login --token $APP_CENTER_API_KEY
    - appcenter distribute release -f build/app/outputs/apk/release/app-release.apk --app $APP_CENTER_APP_NAME_STAGING --group $APP_CENTER_GROUP_TARGET
  environment:
    name: staging

FlutterDeployToDevelopment:
  variables:
    ENV_NAME: "development"
  image: runmymind/docker-android-sdk:latest
  stage: deploy
  tags:
    - docker
    - build
  only:
    refs:
      - /^US-.*$/
  except:
    changes:
      - scrum-booster-api/**/*
  before_script:
    - wget --quiet --output-document=flutter.tar.xz https://storage.googleapis.com/flutter_infra/releases/stable/linux/flutter_linux_v1.0.0-stable.tar.xz && tar xf flutter.tar.xz -C /
    - export PATH=$PATH:/flutter/bin
    - apt-get update && apt-get install gnupg -y
    - curl -sL https://deb.nodesource.com/setup_11.x | bash - && apt-get install -y nodejs
    - flutter doctor -v
    - flutter packages get
    - npm install --global appcenter-cli
  script:
    - flutter build apk --release --build-name=$APP_NAME-$ENV_NAME --build-number=1
    - appcenter
    - appcenter login --token $APP_CENTER_API_KEY
    - appcenter distribute release -f build/app/outputs/apk/release/app-release.apk --app $APP_CENTER_APP_NAME_DEV --group $APP_CENTER_GROUP_TARGET
  environment:
    name: development

FlutterDeployToColdfix:
  variables:
    ENV_NAME: "coldfix"
  image: runmymind/docker-android-sdk:latest
  stage: deploy
  tags:
    - docker
    - build
  only:
    refs:
      - /^coldfix-.*$/
  except:
    changes:
      - scrum-booster-api/**/*
  before_script:
    - wget --quiet --output-document=flutter.tar.xz https://storage.googleapis.com/flutter_infra/releases/stable/linux/flutter_linux_v1.0.0-stable.tar.xz && tar xf flutter.tar.xz -C /
    - export PATH=$PATH:/flutter/bin
    - apt-get update && apt-get install gnupg -y
    - curl -sL https://deb.nodesource.com/setup_11.x | bash - && apt-get install -y nodejs
    - flutter doctor -v
    - flutter packages get
    - npm install --global appcenter-cli
  script:
    - flutter build apk --release --build-name=$APP_NAME-$ENV_NAME --build-number=1
    - appcenter
    - appcenter login --token $APP_CENTER_API_KEY
    - appcenter distribute release -f build/app/outputs/apk/release/app-release.apk --app $APP_CENTER_APP_NAME_COLDFIX --group $APP_CENTER_GROUP_TARGET
  environment:
    name: coldfix

FlutterShowCoverage:
  image: alpine
  stage: show
  dependencies:
    - FlutterTest
  script:
    - mkdir -p mobile-coverage/
    - mv coverage/* mobile-coverage/
  artifacts:
    paths:
    - $CI_PROJECT_DIR/mobile-coverage/
  except:
    changes:
      - scrum-booster-api/**/*