Open in app

Sign in

Write

Sign in

Howto setup Gitlab CI/CD for developing and publishing a Flutter Plugin

markdark
12 min readJan 31, 2020

--

After some search on the internet for a Twilio Programmable Video plugin for Flutter we found out that it didn’t exist. So we decide to go ahead and create one. And since we are proud users of Gitlab we decided to host the source code on Gitlab and setup the CI/CD to check/test/publish our plugin. We would like to share our approach on this CI/CD setup.

I am assuming that you already have a certain knowledge about Gitlab CI/CD. If not check https://docs.gitlab.com/ee/ci/

You can find the complete CI/CD setup in our repository on Gitlab:

.gitlab-ci.yml · master · twilio-flutter / programmable-video

Flutter plugin for Twilio Programmable Video, which enables you to build real-time videocall applications (WebRTC)

gitlab.com

Prerequisites

This guide focuses on using the shared Gitlab CI/CD runners. So activate these runners in your Gitlab project or adjust the jobs to run on specific runners if you like.

We also chose to write native code in Kotlin (Android) and Swift (iOS) so these jobs could be different if you choose different native languages.

Stages

We will be setting up a few stages, being:

  • lint — Jobs for Dart Analysis, Kotlin linting and Swift linting
  • test — Jobs for doing unit testing, intergration testing and E2E testing
  • publish — Jobs for dry-run publishing on merges, tagging the repository and actually publishing to pub.dev

lint — Dart Analysis

If you go through the pub.dev help they recommend to use pedantic. So the first thing we need to do is add pedantic to our plugin pubspec.yaml and our plugin example pubspec.yaml.

# Contents of pubspec.yaml
name: your_plugin_name
description: your_description_of_the_plugin
version: 0.0.1
homepage: <YOUR_HOMEPAGE>
repository: <YOUR_REPOSITORY>
issue_tracker: <YOUR_ISSUE_TRACKER>

environment:
  sdk: ">=2.5.0 <3.0.0"

dependencies:
  flutter:
    sdk: flutter

dev_dependencies:
  flutter_test:
    sdk: flutter
  pedantic: ^1.9.0

After that we need to run flutter pub get and we will notice the following error:

/opt/flutter/bin/flutter --no-color packages get
Running "flutter pub get" in programmable-video...              
Because twilio_programmable_video depends on flutter_test any from sdk which depends on pedantic 1.8.0+1, pedantic 1.8.0+1 is required.So, because twilio_programmable_video depends on pedantic ^1.9.0, version solving failed.
pub get failed (1; So, because twilio_programmable_video depends on pedantic ^1.9.0, version solving failed.)
Process finished with exit code 1

So what is the problem here? Well like it says the flutter test sdk depends on an older version of pedanticand therefore it seems we can’t use the latest version. But pub.dev will calculate the health of our plugin based on the latest rules. Luckily for us there is a workaround for this. Make the next adjustments:

# Contents of pubspec.yaml
name: your_plugin_name
description: your_description_of_the_plugin
version: 0.0.1
homepage: <YOUR_HOMEPAGE>
repository: <YOUR_REPOSITORY>
issue_tracker: <YOUR_ISSUE_TRACKER>

environment:
  sdk: ">=2.5.0 <3.0.0"

dependencies:
  flutter:
    sdk: flutter# TODO: Remove the pendantic here and remove the override once the following issue is resolved:
# https://github.com/flutter/flutter/issues/48246
dev_dependencies:
  flutter_test:
    sdk: flutter
  pedantic: ^1.8.0+1dependency_overrides:
  pedantic: ^1.9.0

Now when we run flutter pub get again it will resolve the correct dependencies. Also make sure to adjust the example/pubspec.yaml on the same way as we did for the project.

After installing the correct dependencies we also need to use these rules from the pendantic package. We can do this by creating a file called analysis_options.yaml in the root of the plugin (same level as the pubspec.yaml). Remember to create this file in the example/ directory as well.

# Contents of analysis_options.yaml# Be aware the health of this package is based on these
# analyse options, we will get the most health points if
# we do not change anything, see:
# https://pub.dev/help#health

# Defines a default set of lint rules enforced for
# projects at Google. For details and rationale,
# see https://github.com/dart-lang/pedantic#enabled-lints.
include: package:pedantic/analysis_options.yaml

# For lint rules and documentation, see http://dart-lang.github.io/linter/lints.
# Uncomment to specify additional rules.
#linter:
#  rules:
#    - prefer_const_constructors

# analyzer:
#   exclude:
#     - path/to/excluded/files/**

Now we are done of configuring the correct Dart Analysis as pub.dev prefers. Let’s dive into the job we need for this to run on our feature branches in Gitlab. First of all create a file called .gitlab-ci.yml in the root of your plugin directory (same level as the pubspec.yaml).

# Contents of .gitlab-ci.ymlimage: cirrusci/flutter:stablestages:
  - lintflutter_analyze:
  stage: lint
  script:
    - cd example || exit 1
    - flutter pub get
    - cd .. || exit 1
    - flutter analyze --pub
    - flutter format -l 240 -n . --set-exit-if-changed
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*.dart
      - test/**/*.dart
      - example/lib/**/*.dart
      - example/test/**/*.dart

The above job will run on the image cirrusci/flutter:stable and will only run on merge_request if this merge_request has changes in the files that matter to analyze. Feel free to tweak and adjust this, but I guess if you follow the standards you do not need to change anything.

Well done, our first job has been configured. And when you push this configuration to Gitlab in a merge request with changes to one of these Dart files the pipeline should trigger.

lint — Kotlin

Setting up a job for linting Kotlin is much more easier. Linting can be done using ktlint and there is an excellent image for that too. We will be using kkopper/ktlint:0.36.0. Open up the .gitlab-ci.yml once again and add the following job:

# Contents of .gitlab-ci.ymlimage: cirrusci/flutter:stablestages:
  - lintflutter_analyze:
  stage: lint
  script:
    - cd example || exit 1
    - flutter pub get
    - cd .. || exit 1
    - flutter analyze --pub
    - flutter format -l 240 -n . --set-exit-if-changed
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*.dart
      - test/**/*.dart
      - example/lib/**/*.dart
      - example/test/**/*.dartkotlin_analyze:
  image: kkopper/ktlint:0.36.0
  stage: lint
  script:
    - cd android || exit 1
    - ktlint
  only:
    refs:
      - merge_requests
    changes:
      - android/**/*.kt

lint — Swift

We are working on the Swift code and linting at the moment. Once we added this I will update this article with the linting job.

test — unit_test

Now the linting is done and makes sure the code is properly formatted and has no analysis errors. We can also execute the unit test we have written in the plugin. Add the test stage and the following job to the .gitlab-ci.yaml:

# Contents of .gitlab-ci.ymlimage: cirrusci/flutter:stablestages:
  - lint
  - testflutter_analyze:
  stage: lint
  script:
    - cd example || exit 1
    - flutter pub get
    - cd .. || exit 1
    - flutter analyze --pub
    - flutter format -l 240 -n . --set-exit-if-changed
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*.dart
      - test/**/*.dart
      - example/lib/**/*.dart
      - example/test/**/*.dartkotlin_analyze:
  image: kkopper/ktlint:0.36.0
  stage: lint
  script:
    - cd android || exit 1
    - ktlint
  only:
    refs:
      - merge_requests
    changes:
      - android/**/*.ktunit_test:
  stage: test
  script:
    - flutter test --pub test/*
    - cd example || exit 1
    - flutter test --pub test/*
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*
      - example/lib/**/*
      - test/**/*
      - example/test/**/*

We have combined the plugin and example tests into 1 job. We did this because the flutter image is quite big and takes more time to download rather then running both tests in the same job. But feel free to split this into 2 jobs and only run them when needed.

publish

Before we dive into the publish stage. We would like to point out that to this point we only executed jobs on merge_requests and specific changes. So all the above jobs will run on feature branches created with a merge request. This is exactly what we want.

For the publish stage we also want to do some testing on merge_requests but once we reach master we would like to do jobs specific on master and even after tagging we would like to run some jobs. So in the next few jobs you will notice differences between them.

publish — dry-run

One job we would like to run on merge_requests also is the flutter pub publish --dry-run. This will test if the changes in the merge request aren’t throwing any errors when publishing to pub.dev.

Let us edit the .gitlab-ci.yml once again and add the new publish stage and also add the dry-run job:

# Contents of .gitlab-ci.ymlimage: cirrusci/flutter:stablestages:
  - lint
  - test
  - publishflutter_analyze:
  stage: lint
  script:
    - cd example || exit 1
    - flutter pub get
    - cd .. || exit 1
    - flutter analyze --pub
    - flutter format -l 240 -n . --set-exit-if-changed
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*.dart
      - test/**/*.dart
      - example/lib/**/*.dart
      - example/test/**/*.dartkotlin_analyze:
  image: kkopper/ktlint:0.36.0
  stage: lint
  script:
    - cd android || exit 1
    - ktlint
  only:
    refs:
      - merge_requests
    changes:
      - android/**/*.ktunit_test:
  stage: test
  script:
    - flutter test --pub test/*
    - cd example || exit 1
    - flutter test --pub test/*
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*
      - example/lib/**/*
      - test/**/*
      - example/test/**/*dry-run:
  stage: publish
  script:
    - flutter pub get
    - flutter pub publish --dry-run
  only:
    - merge_requests

publish — dartdoc

If someone changes the Dart API of the plugin. Documentation should also be done by using dartdoc. Within the Flutter documentation they also tell us how you can run the dartdoc locally and generate this documentation.

What they do not tell us is which commands parameters they use to generate these docs. And how they handle warnings and/or errors. But what we do know is that the documentation is necessary and we also know we want to get the best possible health score on pub.dev, so why don’t we configure dartdoc to generate the best possible docs?

So first let us create a file called dartdoc_options.yaml. This file can be placed in the root of our plugin on the same level as the pubspec.yaml. This file does not need to be created in the example/ dir as we did before.

# Contents of dartdoc_options.yamldartdoc:
  include: ['twilio_programmable_video']
  errors:
    - ambiguous-doc-reference
    - ambiguous-reexport
    - broken-link
    - category-order-gives-missing-package-name
    - deprecated
    - ignored-canonical-for
    - missing-from-search-index
    - no-canonical-found
    - no-library-level-docs
    - not-implemented
    - orphaned-file
    - reexported
    - private-api-across-packages
    - unknown-file
    - unknown-macro
    - unresolved-doc-reference

What we have done here is mapping all warnings to errors, since we want to have the best documentation ever. Feel free to adjust it to your wishes. Also the include should be adjusted with the name of your plugin from pubspec.yaml.

After creating this file we can add a job to the .gitlab-ci.yml file that will test the generation of documentation on merges with changes to Dart files. IF the job succeeds we know all is oke, otherwise it will fail!

# Contents of .gitlab-ci.ymlimage: cirrusci/flutter:stablestages:
  - lint
  - test
  - publishflutter_analyze:
  stage: lint
  script:
    - cd example || exit 1
    - flutter pub get
    - cd .. || exit 1
    - flutter analyze --pub
    - flutter format -l 240 -n . --set-exit-if-changed
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*.dart
      - test/**/*.dart
      - example/lib/**/*.dart
      - example/test/**/*.dartkotlin_analyze:
  image: kkopper/ktlint:0.36.0
  stage: lint
  script:
    - cd android || exit 1
    - ktlint
  only:
    refs:
      - merge_requests
    changes:
      - android/**/*.ktunit_test:
  stage: test
  script:
    - flutter test --pub test/*
    - cd example || exit 1
    - flutter test --pub test/*
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*
      - example/lib/**/*
      - test/**/*
      - example/test/**/*dry-run:
  stage: publish
  script:
    - flutter pub get
    - flutter pub publish --dry-run
  only:
    - merge_requestsdartdoc:
  stage: publish
  script:
    - flutter pub get
    - dartdoc --no-auto-include-dependencies --quiet
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*

We are no finished with all jobs that we want to execute on merges. We now know that the code is properly formatted with the help of these jobs. Now let us focus on the real publishing steps.

publish — tag

During the development of our Twilio Programmable Video Flutter plugin we agreed on the following flow:

  • Someone can contribute and fix bugs or features.
  • When they do, they need to bump the version manually in pubspec.yaml
  • Also they need to provide an entry in the CHANGELOG.md for this version
  • This flow is described and provided as a checklist in the default merge template. An example can be found here.

Tagging your repository with the same versions released on pub.dev and as mentioned in your CHANGELOG.md is recommended. Therefore we added the following job:

# Contents of .gitlab-ci.ymlimage: cirrusci/flutter:stablestages:
  - lint
  - test
  - publishflutter_analyze:
  stage: lint
  script:
    - cd example || exit 1
    - flutter pub get
    - cd .. || exit 1
    - flutter analyze --pub
    - flutter format -l 240 -n . --set-exit-if-changed
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*.dart
      - test/**/*.dart
      - example/lib/**/*.dart
      - example/test/**/*.dartkotlin_analyze:
  image: kkopper/ktlint:0.36.0
  stage: lint
  script:
    - cd android || exit 1
    - ktlint
  only:
    refs:
      - merge_requests
    changes:
      - android/**/*.ktunit_test:
  stage: test
  script:
    - flutter test --pub test/*
    - cd example || exit 1
    - flutter test --pub test/*
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*
      - example/lib/**/*
      - test/**/*
      - example/test/**/*dry-run:
  stage: publish
  script:
    - flutter pub get
    - flutter pub publish --dry-run
  only:
    - merge_requestsdartdoc:
  stage: publish
  script:
    - flutter pub get
    - dartdoc --no-auto-include-dependencies --quiet
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*tag:
  image: docker:stable
  services:
    - docker:stable-dind
  stage: publish
  script:
    - |
      if [ -z "${GITLAB_API_TOKEN}" ]; then
        echo "Missing GITLAB_API_TOKEN environment variable"
        exit 1
      fi

      export TAG_NAME="$(awk '/^version: /{print $NF}' pubspec.yaml)"
docker run --rm curlimages/curl --fail --request POST --header "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" \
  --data-urlencode "tag_name=v${TAG_NAME}" \
  --data-urlencode "ref=master" \
  --data-urlencode "release_description=Check the [CHANGELOG.md](/CHANGELOG.md)" \
  "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/repository/tags"
  environment:
    name: pub-dev
    url: https://pub.dev/packages/twilio_programmable_video
  when: manual
  only:
    - master

As you can see this job adds a tag to the repository by using the Gitlab Tag API specification. A few things about this job are worth mentioning:

GITLAB_API_TOKEN

This token can be obtained from a user that is authorized to tag the repository. It is called Personal Access Tokens. For example in our project only maintainers are allowed to add tags. And version tags are protected.

Once you obtained a token configure this token as a CI/CD environment variable. Make sure to enable protected and masked. So that this token is not exposed to anyone you don’t want to expose it to. So the configuration for this variable looks like this:

environment

We also configured an environment in this tag job. You should adjust the url accordingly to your plugin. Note that we also added the when: manual. This means the job should be triggered manually. Because there could also be merged only related to README.md or something else that does not require to have a version bump or release to pub.dev. Therefore we made this a manual job.

Also the environment pub-dev we mention in this job is a protected environment which only allows maintainers to deploy to. Therefore the job can only be triggered by maintainers of the repository.

publish — pub-dev

Last but not least we need to publish the plugin to pub.dev. Therefore we will add a job pub-dev to our .gitlab-ci.yaml:

# Contents of .gitlab-ci.ymlimage: cirrusci/flutter:stablestages:
  - lint
  - test
  - publishflutter_analyze:
  stage: lint
  script:
    - cd example || exit 1
    - flutter pub get
    - cd .. || exit 1
    - flutter analyze --pub
    - flutter format -l 240 -n . --set-exit-if-changed
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*.dart
      - test/**/*.dart
      - example/lib/**/*.dart
      - example/test/**/*.dartkotlin_analyze:
  image: kkopper/ktlint:0.36.0
  stage: lint
  script:
    - cd android || exit 1
    - ktlint
  only:
    refs:
      - merge_requests
    changes:
      - android/**/*.ktunit_test:
  stage: test
  script:
    - flutter test --pub test/*
    - cd example || exit 1
    - flutter test --pub test/*
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*
      - example/lib/**/*
      - test/**/*
      - example/test/**/*dry-run:
  stage: publish
  script:
    - flutter pub get
    - flutter pub publish --dry-run
  only:
    - merge_requestsdartdoc:
  stage: publish
  script:
    - flutter pub get
    - dartdoc --no-auto-include-dependencies --quiet
  only:
    refs:
      - merge_requests
    changes:
      - lib/**/*tag:
  image: docker:stable
  services:
    - docker:stable-dind
  stage: publish
  script:
    - |
      if [ -z "${GITLAB_API_TOKEN}" ]; then
        echo "Missing GITLAB_API_TOKEN environment variable"
        exit 1
      fi

      export TAG_NAME="$(awk '/^version: /{print $NF}' pubspec.yaml)"
      docker run --rm curlimages/curl --request POST --header "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" \
        "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/repository/tags?tag_name=v${TAG_NAME}&ref=master&release_description=Check%20the%20%5BCHANGELOG.md%5D(%2FCHANGELOG.md)"
  environment:
    name: pub-dev
    url: https://pub.dev/packages/twilio_programmable_video
  when: manual
  only:
    - masterpub-dev:
  stage: publish
  script:
    - |
      if [ -z "${PUB_DEV_PUBLISH_ACCESS_TOKEN}" ]; then
        echo "Missing PUB_DEV_PUBLISH_ACCESS_TOKEN environment variable"
        exit 1
      fi

      if [ -z "${PUB_DEV_PUBLISH_REFRESH_TOKEN}" ]; then
        echo "Missing PUB_DEV_PUBLISH_REFRESH_TOKEN environment variable"
        exit 1
      fi

      if [ -z "${PUB_DEV_PUBLISH_TOKEN_ENDPOINT}" ]; then
        echo "Missing PUB_DEV_PUBLISH_TOKEN_ENDPOINT environment variable"
        exit 1
      fi

      if [ -z "${PUB_DEV_PUBLISH_EXPIRATION}" ]; then
        echo "Missing PUB_DEV_PUBLISH_EXPIRATION environment variable"
        exit 1
      fi

      cat <<EOF > ~/.pub-cache/credentials.json
      {
        "accessToken":"$(echo "${PUB_DEV_PUBLISH_ACCESS_TOKEN}" | base64 -d)",
        "refreshToken":"$(echo "${PUB_DEV_PUBLISH_REFRESH_TOKEN}" | base64 -d)",
        "tokenEndpoint":"${PUB_DEV_PUBLISH_TOKEN_ENDPOINT}",
        "scopes":["https://www.googleapis.com/auth/userinfo.email","openid"],
        "expiration":${PUB_DEV_PUBLISH_EXPIRATION}
      }
      EOF
    - flutter pub get
    - flutter pub publish -f
  only:
    - tags

This jobs needs a few environment variables. To be precise we need:

  • PUB_DEV_PUBLISH_ACCESS_TOKEN
  • PUB_DEV_PUBLISH_REFRESH_TOKEN
  • PUB_DEV_PUBLISH_TOKEN_ENDPOINT
  • PUB_DEV_PUBLISH_EXPIRATION

But how do we get them? We found a way to do this without publishing the package first. Execute the following command from you not already published plugin:

flutter pub uploader add some@email.org

This commands ask you to login. Do it. But eventually it throws an error, because your plugin is not published yet. But now we can get the credentials we need for the CI/CD job.

The credential can be found in the file located <FLUTTER_INSTALL_DIR>/.pub-cache/credentials.json. Now that you found the credentials add the variables once again to the CI/CD variables through the Gitlab UI. But to secure the credentials the PUB_DEV_PUBLISH_ACCESS_TOKEN and PUB_DEV_PUBLISH_REFRESH_TOKEN should be masked. Masked variables are only allowed if the content is base64 compatible. Therefore we added them base64 encoded and within the job they are decoded. To get the value encoded we used the next command:

echo "VALUE_TO_ENCODE" | base64 -w 0

For reference here a complete screenshot of all environment variables:

Trying it out

Now you have completed all steps and once someone opens a merge request all checks will be done to get the most of the health of the repository.

Feel free to adjust jobs, make more jobs to your liking.

Enjoy!

Flutter
Gitlab
Gitlab Ci

--

--

markdark
markdark

Written by markdark

2 Followers
3 Following

Maintainer of https://gitlab.com/twilio-flutter/programmable-video

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams