Structuring and Deploying Multi-Container Apps with Docker
In this article, we’ll be exploring the world of containerization and how it can be utilized to structure and deploy multiple applications effectively.
We’ll be referencing the open-source project, Celluloid, a collaborative video annotation platform, and its GitHub repository as a case study. By exploring the code, we’ll uncover the techniques and tools I used to manage and deploy their applications, providing valuable insights for fellow developers.
Table of Contents
- Introduction to Containerization and Docker
- Analyzing the Celluloid Repo
— Project Structure
— Database and Storage Configuration - Turbo, Caddy, and PM2: The Dynamic Trio
— Turbo Builds
— Caddy as a Reverse Proxy
— PM2 for Multiple Server Processes - End-to-End Testing with Playwright and GitHub Actions
- Conclusion and Key Takeaways
1. Introduction to Containerization and Docker
Containerization is a lightweight, portable, and scalable approach to application deployment. It involves packaging an application and its dependencies into isolated containers, ensuring consistent behavior across environments. Docker is the leading containerization platform, providing a convenient and efficient way to build, deploy, and manage containers. With Docker, developers can create reproducible builds, streamline deployments, and scale applications effortlessly.
2. Analyzing the Celluloid Repo
Celluloid is an educational video annotation platform with a multi-container architecture. Let’s explore the GitHub repository to understand how it’s structured and configured the applications for deployment.
Project Structure
The Celluloid repo is organized with a clear separation of concerns. It follows a monorepo structure, housing multiple applications and shared packages within distinct folders:
├── apps/
│ ├── frontend/
│ ├── backend/
│ └── admin/
├── packages/
│ ├── config/
│ ├── i18n/
│ ├── passport/
│ ├── prisma/
│ ├── trpc/
│ ├── types/
├── utils/
├── tests/
├── packages.json
└── envThis structure promotes code reusability and maintainability, allowing for independent development and deployment of each application while sharing common configurations and utilities.
Database and Storage Configuration
Celluloid rely on PostgreSQL for database needs, I provide a Docker Compose file, stack.yml, which sets up the entire stack, including the database. This simplifies the setup process for developers, ensuring a consistent database environment.
For storage, I opted for an S3-compatible service, specifically Minio. This provides a scalable and reliable object storage solution, essential for handling video assets and user data.
3. Turbo, Caddy, and PM2: The Dynamic Trio
Celluloid employs a combination of tools to build, serve, and manage their applications: Turbo, Caddy, and PM2.
Turbo Builds
Turbo is a build system that accelerates the development and deployment process. Celluloid uses Turbo to build the Docker images. Let’s take a look at Dockerfile.compact:
FROM node:20-alpine AS custom-node
RUN apk add -f --update --no-cache --virtual .gyp nano bash libc6-compat python3 make g++ \
&& yarn global add turbo \
&& apk del .gyp
FROM custom-node AS pruned
WORKDIR /app
COPY . .
RUN turbo prune --scope=admin --scope=frontend --scope=backend --docker
FROM custom-node AS installer
WORKDIR /app
COPY --from=pruned /app/out/json/ .
COPY --from=pruned /app/out/yarn.lock /app/yarn.lock
RUN \
--mount=type=cache,target=/usr/local/share/.cache/yarn/v6,sharing=private \
yarn
FROM custom-node as builder
WORKDIR /app
ARG API_URL
ARG COMMIT
ENV COMMIT=${COMMIT}
ENV API_URL=${API_URL}
COPY --from=installer --link /app .
COPY --from=pruned /app/out/full/ .
COPY turbo.json turbo.json
COPY tsconfig.json tsconfig.json
COPY ecosystem.config.js ecosystem.config.js
RUN turbo run build --no-cache
RUN \
--mount=type=cache,target=/usr/local/share/.cache/yarn/v6,sharing=private \
yarn --frozen-lockfile
#############################################
FROM node:20-alpine AS runner
WORKDIR /app
RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories \
&& apk add -f --update caddy
COPY --from=builder /app .
COPY Caddyfile /etc/caddy/Caddyfile
CMD ["sh", "-c", "caddy run --config /etc/caddy/Caddyfile & yarn start"]In this snippet, Turbo is used to create a multi-stage build. It installs dependencies, copies the source code, and executes the build process, optimizing the image for production.
Caddy as a Reverse Proxy
Caddy is a powerful web server and reverse proxy, providing secure and efficient request handling. Celluloid uses Caddy to proxy requests to the appropriate application containers. Here’s an example of the Caddyfile:
{
servers {
trusted_proxies static private_ranges
}
}
:80 {
@trpc path_regexp ^/trpc(/|$)
reverse_proxy @trpc localhost:2021
@admin path_regexp ^/admin(/|$)
reverse_proxy @admin localhost:4000
reverse_proxy localhost:3000
}In this configuration, Caddy redirects HTTP requests to HTTPS and proxies them to the `app` container listening on port 3000. It also enables gzip compression and TLS encryption for secure and efficient content delivery.
PM2 for Multiple Server Processes
PM2 is a process manager for Node.js applications, enabling the management and monitoring of multiple server processes. Celluloid uses PM2 to run their backend and admin applications concurrently. Here’s a code snippet from their ecosystem.config.js:
module.exports = {
apps: [
{
name: "PrismaMigrate",
script: "yarn",
args: "prisma migrate:deploy",
interpreter: "sh",
autorestart: false,
watch: false,
},
{
name: "frontend",
script: "yarn",
args: "frontend start",
interpreter: "sh",
watch: false,
},
{
name: "admin",
script: "yarn",
args: "admin start",
interpreter: "sh",
watch: false,
},
{
name: "backend",
script: "yarn",
args: "backend start",
interpreter: "sh",
watch: false,
},
],
};With PM2, We can ensure that frontend, api and admin servers are running smoothly, restarting automatically if any issues occur.
4. End-to-End Testing with Playwright and GitHub Actions
Celluloid uses Playwright, a powerful end-to-end testing framework, to ensure the quality and functionality of their applications. Playwright integrated with GitHub Actions for automated testing during their CI/CD pipeline. Here’s a glimpse of their GitHub Actions configuration:
name: E2E Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
jobs:
tests:
timeout-minutes: 60
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.39.0-jammy
services:
redis:
image: redis
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
postgres:
image: postgres:14
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: celluloid
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'yarn'
- run: yarn install --frozen-lockfile
- name: Build apps
run: |
cp .env.ci .env
yarn build --filter=frontend --filter=backend
- name: Run Playwright tests
# with workground https://github.com/microsoft/playwright/issues/6500
run: npx playwright test
env:
HOME: /root
- uses: actions/upload-artifact@v3
if: always()
with:
name: playwright-report
path: playwright-report/
retention-days: 30
- name: Publish HTML report to gh-pages
if: github.ref == 'refs/heads/main'
uses: peaceiris/actions-gh-pages@v3.9.3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: playwright-report/
# destination_dir: playwright-report/
keep_files: false
user_name: "github-actions[bot]"
user_email: "github-actions[bot]@users.noreply.github.com"
commit_message: ${{ github.event.head_commit.message }}In this setup, the action installs dependencies, sets up Chromium, and executes the end-to-end tests defined in the apps/frontend folder. This automated testing ensures that the frontend application functions as expected with each change.
Example of Testing results :
5. Key Takeaways
In this article, we explored the Celluloid project to gain valuable insights into structuring and deploying multi-container applications with Docker:
- Containerization with Docker provides a portable and scalable deployment solution.
- A well-structured monorepo promotes code reusability and maintainability.
- Turbo builds optimize the application for production, streamlining the deployment process.
- Caddy, as a reverse proxy, securely handles requests and enables TLS encryption.
- PM2 manages multiple server processes, ensuring smooth and continuous operation.
- Playwright, integrated with GitHub Actions, automates end-to-end testing during CI/CD.
By following these practices and this tools outlined, developers can efficiently structure and deploy their applications, benefiting from the power of containerization and the dynamic trio of Turbo, Caddy, and PM2.
Happy coding !
If you’re looking to start a new React Native / Expo project, check out my expo starter kit. It will help you bootstrap your project quickly and efficiently :
https://expostarter.com
