Configure renovate bot for npm at Homegate

Renovate bot is an automated dependency upgrade for npm packages. It supports many services such as gitlab & github.

Homegate repositories structure

At Homegate, we have over 200 repositories in micro-services architecture. Keeping services up to date is quite a big overhead. However, the benefits are obvious:

• Bugfixes
• New features
• Security issues / less vulnerabilities

Evaluation session result from our engineering team

The standard configuration

We try to make process as automated and standardised as possible. The configuration will automatically create MRs for minor and major upgrade in our gitlab repos and auto-merge them.

It is necessary to allow merge without approval in these repositories. Below you’ll see our standard configuration with the following features:

• Schedule MR creations every hour on weekdays
• Limits apply (concurrent MRs / Rate of creation)
• Smart bundling can be configured, such as having all linters together or all of our shared frontend library related MRs together
• Auto-merge for minor and dev-dependencies
• Separation of major upgrade (requiring review) vs. minor upgrade (auto-merged)
• Configuration for custom npm host
• Each project has renovate.json that extends the standard homegate.json with custom config if necessary

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "config:base",
    ":prHourlyLimitNone",
    ":automergeLinters",
    ":automergeTesters",
    ":automergeTypes",
    ":automergeMinor",
    "group:definitelyTyped",
    "group:postcss",
    "group:linters",
    ":rebaseStalePrs",
    ":maintainLockFilesMonthly"
  ],
  "prConcurrentLimit": 2,
  "schedule": [
    "after 6am and before 5pm every weekday"
  ],
  "ignoreDeps": ["node-sass",
    "stylelint", "stylelint-scss", "stylelint-config-standard", "stylelint-config-css-modules"],
  "packageRules": [
    {
      "matchPackagePatterns": ["vue", "vue-router", "vuex"],
      "matchUpdateTypes": ["major"],
      "enabled": false
    },
    {
      "matchPackagePatterns": ["^@aws-cdk", "aws-cdk"],
      "matchUpdateTypes": ["major"],
      "enabled": false
    },
    {
      "depTypeList": ["devDependencies"],
      "automerge": true
    },
    {
      "packageNames": ["aws-sdk"],
      "extends": ["schedule:weekly"]
    },
    {
      "sourceUrlPrefixes": ["https://github.com/getndazn/dazn-lambda-powertools"],
      "groupName": "Dazn lambda powertools packages"
    },
    {
      "sourceUrlPrefixes": ["https://github.com/middyjs/middy"],
      "groupName": "Middy middleware packages"
    },
    {
      "sourceUrlPrefixes": ["https://gitlab.com/homegate/projects/npm-packages/project-configs"],
      "groupName": "@homegate project configs packages"
    },
    {
      "sourceUrlPrefixes": ["https://github.com/Turfjs/turf"],
      "groupName": "TurfJs packages"
    }
  ],
  "addLabels": ["renovate-bot"],
  "hostRules": [
    {
      "hostType": "npm",
      "matchHost": "homegate.jfrog.io/homegate/api/npm/homegate-npm",
      "token": "{{ secrets.JFROG_TOKEN }}"
    }
  ],
  "npmrc": "@homegate:registry=https://homegate.jfrog.io/homegate/api/npm/homegate-npm/\n//homegate.jfrog.io/homegate/api/npm/homegate-npm/:_authToken={{ secrets.JFROG_TOKEN }}"
}

Team responsibility

It is important that each team allocate time to maintain and upgrade their libraries. We setup that renovate bot is mandatory in our services and even with the help with automation, team must take care of the status of what the bots are doing.

Most MRs have a team label added by the bot, this allows to query it. For example

https://gitlab.com/groups/homegate/projects/-/merge_requests?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=Rubber-Ducks

Sometimes, renovate MRs fail and need manual intervention. Here are some of the most usual issues and corresponding remediation:

• Flaky tests → retry job/pipeline
• Rebase has conflict → manual rebase
• Breaking change in MR that needs some update in codebase:
  - Small changes → just do it
  - Larger changes → create a ticket and close MR

Appendix: Custom configuration and process

Our team, Honey Badgers, in Homegate owns the insertion-funnel which we found the standard process didn’t work out for us due to two main reasons:

• Front-end input heavy service -> may bring breaking changes & UI bugs especially
• Many teams working on it in parallel -> when working on a MR, continuous rebasing is required and we are in a reactive mode on fixing a problematic auto-upgrade (e.g. cypress) that may block other pipelines, which leads to unplanned distraction in case of issues.

We started to employ an extended configuration and process from the standard one:

• 2 MRs on renovate (non-major & major) for each project
• Automatic generation of a ticket on JIRA board each week. The ticket is being picked up by the team and and MRs merged manually each week

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "gitlab>homegate/projects/hg-renovate-bot:homegate",
    "gitlab>homegate/projects/hg-renovate-bot:label_honey-badgers"
  ],
  "timezone": "Europe/Zurich",
  "schedule": ["before 8am on Monday"],
  "prConcurrentLimit": 10,
  "packageRules": [
    {
      "groupName": "bot: devDependencies (non-major)",
      "matchDepTypes": ["devDependencies"],
      "matchUpdateTypes": ["minor", "patch", "pin", "digest"],
      "automerge": false
    },
    {
      "groupName": "bot: dependencies (non-major)",
      "matchDepTypes": ["dependencies"],
      "matchUpdateTypes": ["minor", "patch", "pin", "digest"],
      "automerge": false
    },
    {
      "groupName": "bot: devDependencies",
      "matchDepTypes": ["devDependencies"],
      "matchUpdateTypes": ["major"],
      "automerge": false
    },
    {
      "groupName": "bot: dependencies",
      "matchDepTypes": ["dependencies"],
      "matchUpdateTypes": ["major"],
      "automerge": false
    },
    {
      "groupName": "bot: serverless eco-system",
      "matchPackageNames": [
        "serverless",
        "serverless-dotenv-plugin",
        "serverless-finch"
      ],
      "automerge": false
    },
    {
      "groupName": "bot: disabled dependencies",
      "matchPackageNames": [
        "@homegate/domus",
        "@types/googlemaps",
        "sass-loader",
        "mock-xmlhttprequest",
        "@vue/test-utils",
        "@homegate/hg-accounts-client",
        "vue-tel-input",
        "eslint",
        "file-type",
        "ts-jest",
        "stylelint-config-css-modules"
      ],
      "enabled": false
    }
  ]
}

Results: This provides control on upgrades while having benefits of a semi-automated process. We spend around 4 hours a month to keep up dependencies. Here are the main takeaways for us:

• Optimize time to work or react on upgrading libs issues
• Reduce number of MRs created
• Have a semi-automated process to keep libs upgraded
• Avoid risks of broken pipelines or unwanted feature changes by having a manual merge process

Credits

Homegate engineers.