Migrate Kotlin backend APIs into Nx workspace

Published in

Code道

6 min readJun 11, 2021

Nx, a TypeScript based monorepo tool, is not only a great tool for Node.js based technology such as Angular, Ionic or NestJS but also works well with Kotlin based backend APIs with multiple modules.

Modular Monolithic Kotlin Backend APIs

At Lapis, we are busy migrating existing manyrepo projects into the Nx workspace, in order to simplify and streamline the development life cycle.

We have recently built and released AquaOnline, a Commercial Aquaculture Management System for NSW DPI. AquaOnline manages government administrative information for commercial aquaculture farms (land-based facilities) and leases (sea and rivers) with regulated permissions. The following diagram shows an initial domain model identified prior to the development.

We use a Modular Monolithic architecture approach, with a single SQLServer database and multiple schemas (each schema contains feature specific entities) and run on AWS Fargate, a managed Docker Container Service.

Manyrepos by feature modules

The following diagram shows the Kotlin-based Backend-For-Frontend APIs and feature modules, which use to have their own GIT repository and CI processes before migrating to the Nx workspace.

Apps and Libs

Nx workspace provides a standard folder structure to organise project artefacts. The apps folder contains frontend and backend applications, such as Angular or NestJS. The libs folder contains any reusable feature modules, which could be shared between the apps, both the frontend and backend. We can simply follow the same convention and place our Kotlin based modules in apps and libs folders.

https://nx.dev/latest/angular/getting-started/nx-setup

Generators to create apps and libs

Nx comes with a set of generators to scaffold an application or a library and configure it in nx.json, workspace.json, Typescript, ESLint and Jest configuration files. We could even create a custom generator plugin for Kotlin. (The Nx community plugins have a generator for SprintBoot).

Generate Kotlin based Feature Library

Since we’re migrating existing repositories, we use a Node generator to scaffold a library instead of building a new generator.

Naming Conventions

We decided to prefix Kotlin based modules with kotlin-, so we can differentiate them from other Typescript based modules. The following command will generate a new folder structure in the libs folder and configure it in the configuration files. We delete the generated files in libs/kotlin-lib-core and copy Kotlin resources from the existing repository.

nx generate @nrwl/node:library --name=kotlin-lib-core --no-interactive --dry-runCREATE libs/kotlin-lib-core/README.md
CREATE libs/kotlin-lib-core/.babelrc
CREATE libs/kotlin-lib-core/src/index.ts
CREATE libs/kotlin-lib-core/src/lib/kotlin-lib-core.spec.ts
CREATE libs/kotlin-lib-core/src/lib/kotlin-lib-core.ts
CREATE libs/kotlin-lib-core/tsconfig.json
CREATE libs/kotlin-lib-core/tsconfig.lib.json
UPDATE tsconfig.base.json
UPDATE workspace.json
UPDATE nx.json
CREATE libs/kotlin-lib-core/.eslintrc.json
CREATE libs/kotlin-lib-core/jest.config.js

nx.json

nx.json defines projects and the generator adds the new project in it. The project specifies 2 attributes, implicitDependencies and tags. implicitDependencies specifies a list of dependant modules. Nx affected command will build the module when the dependant module has been changed in the build pipeline. tags will limit the usage of the library for apps and libs having the same tag.

“kotlin-lib-core”: {
 “implicitDependencies”: [“kotlin-aquaonline-pom”],
 “tags”: [“scope:kotlin”]
},
"kotlin-lib-transaction": {
  "implicitDependencies": ["kotlin-lib-core"],
  "tags": ["scope:kotlin"]
},

nx dep-grapth uses implicitDependencies to visualise your dependency graph.

workspace.json

workspace.json defines the project configuration such as the root folder (libs/kotlin-lib-core) and the builders, a set of tasks executed via Nx. Nx provides a set of builders for known architecture (Angular or NestJS), including build, serve, lint and test. We’ll follow similar DSLs in order to build a CI pipeline seamlessly.

The above configuration shows 4 tasks, lint, build, test and deploy. These can be executed as follows:

nx lint kotlin-lib-core
nx build kotlin-lib-core
nx test kotlin-lib-core
nx deploy kotlin-lib-core

Run Commands Builder

We use run-commands builder, which allows you to run any CLI. The following shows the build task details. The builder simply executes the defined commands. install.sh runs maven install within a Docker container; change the owner of the target folder and copy the result into the dist folder. This is mimicking how Nx builds a typescript based project and the transpiled files are stored in the dist folder.

"build": {
          "builder": "@nrwl/workspace:run-commands",
          "options": {
            "commands": [
              "./install.sh",
              "./chown.sh",
              "mkdir -p ../../dist/kotlin-lib-core",
              "mv ./target ../../dist/kotlin-lib-core"
            ],
            "cwd": "libs/kotlin-lib-core",
            "parallel": false
          }
        }

You can define any tasks for a project or set of projects with the same architecture. For Kotlin APIs, we also want to build a Docker image and push it to a Container Registry.

The following shows slightly different tasks for a Kotlin API, including prepare-docker and build-push-docker tasks:

The project configuration for the Kotlin application

Force build order with strictlyOrderedTargets

When you define your custom tasks, it’s important to specify the build order to ensure they run sequentially, as Nx, by default, executes tasks in parallel to minimise the total execution time. You can specify it using strictlyOrderedTargets in nx.json.

"tasksRunnerOptions": {
    "default": {
      "runner": "@nrwl/nx-cloud",
      "options": {
        "cacheableOperations": ["build", "lint", "test", "e2e"],
        "accessToken": "...",
        "canTrackAnalytics": false,
        "showUsageWarnings": true,
        "strictlyOrderedTargets": [
          "build",
          "prepare-test",
          "test",
          "lint",
          "e2e",
          "prepare-docker",
          "build-push-docker",
          "deploy",
          "deploy-ec2"
        ]
      }
    }

CI/CD Pipeline

We now have a set of tasks that can be executed in the CI pipeline. We use the GitLab CI and execute them, by mapping these tasks with GitLab CI stages and jobs.

stages:
    - init
    - init-test-db
    - build
    - test
    - build-push-docker
    - deploy

You can simply execute nx affected command in each stage and jobs. Nx looks up any changes in the repository and only run tasks for projects which have changed or dependant on the changed projects.

nx affected:build --base=HEAD~1 --head=HEAD --with-deps
nx affected:test --base=HEAD~1 --head=HEAD --with-deps
nx affected:build-push-docker --base=HEAD~1 --head=HEAD --with-deps
nx affected:deploy --base=HEAD~1 --head=HEAD --with-deps

nx affected greatly simplifies the pipeline configuration for multi-module Kotlin-based APIs as you no longer need to worry about which repo should be pushed first or how to trigger another pipeline that is dependant on a particular feature module.

Summary

Nx workspace can manage apps and libs using any technology, including Kotlin-based APIs
Nx Generators help you to configure a new project, even you don’t need scaffolded resources
Having a good naming convention will help to organise related apps and libs
implicitDependenciesspecifies Kotlin module dependencies and let you leverage nx affected command
run-commands builder lets you run any CLI via Nx on any platform (Linux, Mac and Windows)
strictlyOrderedTargets specified how the task should be executed sequentially
nx affected simplifies multi-module project pipeline by automatically discover changes and only run the pipeline that is affected

Thanks for reading!