Google Apps Script without pain OR success story of Atlassian Cloud for Gmail

10 min readNov 2, 2018

Atlassian is the company behind products such as Jira, Confluence, Bitbucket and many others. We released our Atlassian Gmail add-on on July 18 and you can check out the live demo from Google Next’18 talk by Tim Pettersen and Caroline Bartle. We’ve open sourced that project to the world so other add-ons developers can take a look at practices we discovered and applied during the development of Atlassian Cloud for Gmail.

TL;DR. Here is the source code on Bitbucket

Prerequisites

The platform Google provides for Gmail add-ons (and other Apps Script projects) resembles a serverless model. You provide the code, and Google runs it in their cloud on your behalf. This approach makes it simple to get started developing add-ons, and rules out an entire class of issues relating to hosting. However, it does come with some limitations that our team has had to work around to make add-on development feel like a modern software project. Some of the issues we’ll discuss in this post include:

How to use a modern version of JavaScript
How to structure your Gmail Add-on project
Automated testing
Continuous Integration and Deployment
Analytics

We hope the patterns discussed below help you with your own Google Apps Script projects!

CLASP — almost the solution

Google provides a web-based IDE to write, release, and deploy Apps Script projects. However, for any non-trivial add-on, you’re probably going to want to use your own IDE and development toolchain on your desktop. This is where clasp comes in. Command Line Apps Script Project (clasp) is a separate CLI tool developed by Google which is used to manage and deploy Apps Script projects from your terminal.

So in our team’s workflow, we check in all the source code to the version control system and every developer has a separate Apps Script development environment. This means we can each use our favorite code editor, develop, debug, and run tests locally as we would do with any modern software project.

One nice feature of clasp is that it supports files ignoring out of the box by defining .claspignore file. It’s similar to .gitignore, but instead prevents certain local files from syncing with your remote Apps Script project.

In our add-on, we ignore everything by default, but use a negating pattern (prefixed with a !) to export the contents of our build directory and our appsscript.json manifest:

**/**             # ignore everything
!build/**         # except the build folder
!appsscript.json  # and the project manifest

clasp makes Apps Script developer’s life a lot easier. However, it does have some limitations related to environment variables and service accounts support which complicate Continuous Integration and Deployment systems (more on this later).

Bundling and Transpiling

Apps Script is based on JavaScript 1.6, with a few features from 1.7 and 1.8. This makes it a far cry from the modern JavaScript syntax your team is likely working on. There is no support for ES2015+ features, no XMLHttpRequest or fetch, and no asynchronous code. Seriously, there are no promises, callbacks, timers, or timeouts of any description. Unfortunately there’s not much we can do about async support in Apps Script, but using Webpack and Babel we can at least support the latest ES language features, ES modules, and write the beautiful JavaScript we love.

Image result for ecmascript — Google Apps Script is still somewhere in 2005

Here’s the prototype of the Webpack configuration we use in our project:

It’s probably pretty similar to other applications you’ve seen which use Webpack to bundle their modules, except for two special parts.

Firstly, in Google Apps Script functions called by google.script.run or otherwise referenced in your Manifest should be declared at the top level. This means we have to expose them from our modules. We do this by setting output.libraryTarget: ‘this’, which exposes functions you’ve exported from Webpack’s entry point to the global this scope.

Secondly, you may have noticed that we copy an empty file (static/_.js) to our build directory. This solves a problem we encountered with the Apps Script online IDE. When opening your Apps Script project online, the IDE defaults to opening the first file in the project (sorted alphabetically). This is typically bundle.js, which — once you have a few dependencies bundled — is so huge that it hangs the Apps Script editor. Introducing an empty file named _.js (which comes alphabetically first) neatly sidesteps the problem, and allows you to open your project online without killing your browser.

I’d also like to mention the Babel configuration file we have for our add-on:

{
  "presets": ["@babel/env"],
  "plugins": [
    "@babel/transform-member-expression-literals",
    "@babel/transform-property-literals"
  ]
}

@babel/env compiles ES2015, ES2016, ES2017 syntax down to ES5, and along with two plugins that transform object literals, makes our modern JavaScript code compatible with Google Apps Script.

Finally, you should add a core-js import to the top of your entry point. This will bring all the standard library polyfills into your code which Apps Script doesn’t yet fully support.

Project structure

The way you structure your add-on project will have a big impact on your development experience during the life of the project. In Atlassian Cloud for Gmail, we use an MVC pattern adapted for Gmail’s structured view system, in which on every action we should return a new Card. We’ve split our application into three layers: controllers, views, and a layer which takes care of interaction with external APIs. Our project is organized into the following folder structure:

.
├── common/
│   ├── net/
│   │   ├── HttpClient.js
│   │   ├── Bitbucket.js
│   │   └── Jira.js
│   ├── AddonError.js
│   ├── analytics.js
│   └── utils.js
├── controllers/
│   ├── addon/
│   ├── bitbucket/
│   ├── jira/
│   ├── ActionController.js
│   ├── Controller.js
│   ├── SuggestionsController.js
│   ├── UniversalActionController.js
│   ├── UpdateDraftActionController.js
│   └── index.js
├── views/
│   ├── addon/
│   ├── bitbucket/
│   ├── jira/
│   ├── Colors.js
│   ├── Icons.js
│   ├── View.js
│   └── utils.js
└── index.js

Let’s briefly look at each layer.

Controllers

These are responsible for managing incoming events, validating form parameters, and returning Cards (or other data structures that Gmail expects) as a response to an action such as a button click or form submission.

For example, here is the controller responsible for creating Bitbucket pull request comments. It validates the content of the comment and throws a validation error for a blank string. If validation passes, it calls the Bitbucket API and returns a new Card to refresh the current pull request view.

You may notice other files with *Controller suffix in the project’s tree such as ActionController.js, SuggestionsController.js, etc. These are abstract classes which define common behaviours for how concrete controllers should handle a response. For example, controllers inherited from UniversalActionController are used to handle clicks on universal menu, and the SuggestionsController wraps an array of strings into a specific SuggestionsResponse object expected by the Gmail add-on API.

Views

View classes take raw JavaScript objects and render them as Gmail add-on Cards. They utilize properties — this.props, just like React, which are plain JavaScript objects. Each View inherits from the base View class and defines a few methods, as in the example below:

We’ve found defining header, sections and url as separate getters is cleaner and far easier to debug and test than direct usage of the CardService builders.

In the add-on, this card looks like this:

Link returned from url getter goes to contextual menu

Using separate View classes instead of returning Cards from some functions directly allows us to inherit one view from another, perform instanceof checks on Views, and improves our overall project structure. For example, we use instanceof here to handle alerts differently to regular card updates:

function getNavigation (view) {
  const nav = CardService.newNavigation()  const card = view.render()  if (view instanceof AlertView) {
    // alerts should be pushed to the navigation stack
    // so user could click `back` arrow and try again the action
    nav.pushCard(card)
  } else {
    nav.updateCard(card)
  }
  
  return CardService.newActionResponseBuilder()
    .setNavigation(nav)
    .build()
}

Requests to external APIs

All the code related to communication with external APIs lives in the common/net directory, with one class per API provider. This allows us to encapsulate error handling, authorization, and everything else related to a particular API in one place. Although we can’t make async requests, Apps Script provides a UrlFetchApp.fetchAll method, which allows us to send parallel HTTP requests (while maintaining our provider specific logic inside our net classes). Example:

Analytics

At the time of writing, there is no built-in way to collect analytics data in Google Apps Script. We decided to use Google Analytics’ REST API since request latency is pretty low from Google Cloud to GA (in our tests it took less than 100ms) and they have an endpoint to send multiple events in a batch request. Performance is super important in this case since we can’t simply fire an asynchronous request as we might in other programming languages, as everything is synchronous in Apps Script and is therefore blocking the user! We encapsulated this logic in a dedicated GoogleAnalytics class in our code.

Our analytics implementation uses a strong hash of the user’s email as a client id to avoid storing the user’s actual email. We also batch requests to GA once every 20 analytics events to minimize blocking execution of the main code. This resulted in an analytics API that is simple to use throughout the rest of the codebase:

Automated Testing

At Atlassian, we’re strong believers in automated testing, even in relatively small projects such as as Google add-ons. Our team loves Jest as it has everything we need out of the box: simple configuration, assertions, mocking, and code coverage. So obviously we wanted to use Jest for Gmail add-on development too.

Since Google Apps Script provides many built-in services for interacting with user data, Google APIs, and external systems, it’s difficult to use Jest (or any other JS testing framework) as Google’s Service classes will not be available when running offline. So we created a small library of mock Apps Script globals — gas-mock-globals — which can be used to inject Apps Script polyfills into tests to run them locally, on CI systems, or any other Node.js environment.

While gas-mock-globals doesn’t cover 100% of all the possible Apps Script globals, it does have most of the classes and enums you need for Gmail add-ons, and we’re expanding the list. As always, PRs are welcomed :)

Let’s look at a simple example.

Here we have an Alert class that later can be rendered into an instance of Card, which will be displayed as part of the add-on. Jest has a great feature which helps to make sure your UI does not change unexpectedly: snapshot testing. It’s commonly used in the React world, but nothing prevents us from using this technique for Gmail cards since each Card can be represented as JSON.

The Alert view is used to show errors or warnings in the add-on, such as validation or network errors.

And here is a snapshot test for Alert view:

Say we later update the Alert view to be a ‘warning’ by default rather than an ‘error’:

...
     const {
       title = 'Error',
-      type = 'error',
+      type = 'warning',
     } = this.props
...

Since we’ve just updated our card’s header to have a different icon, it’s reasonable to expect changes in the snapshot for this view. Our snapshot test case will fail because the updated snapshot no longer matches the original one for this test case:

As you can see, with just a few lines of code we’ve covered an important part of the application logic via snapshot testing, which will help us identify regressions in the future. That’s one simple use case of gas-mock-globals, you can see a few more examples in the library’s repo.

CI/CD

In this final section, I’d like to share how our team achieved Continuous Integration & Deployment for our Gmail add-on using clasp, the Google API Node.js Client, and Bitbucket Pipelines & Deployments.

We have two separate add-on environments: test and production. These correspond to two deployment steps in our Bitbucket Pipelines configuration file. For example, here we have “Deploy to Production” step:

- step:
    name: Deploy to Production
    deployment: production
    trigger: manual
    script:
      - npm install
      - export SCRIPT_ID=$PROD_SCRIPT_ID 
      - export DEPLOYMENT_ID=$PROD_DEPLOYMENT_ID
      - export NODE_ENV=production
      - npm run deploy

It simply installs our npm dependencies, defines the ID of our production Apps Script project as well as production deployment ID and runs deploy.js.

Let’s take a closer look at deploy.js:

You can check out the full version of deploy.js in the repo.

initClasp: we first create configuration files on our CI server (a Docker container), which are used by clasp to synchronize the built script files with the remote Apps Script project. Since service accounts are not supported in clasp, we provide our REFRESH_TOKEN obtained after authorization to clasp locally.
build: then we trigger the build command defined in our package.json, which runs Webpack to transpile and bundle our source code into a single file, and copies static data.
push: clasp then pushes our newly created build/ directory to our Apps Script project as the HEAD deployment.
deploy: we run clasp deploy command which creates a new version of the script and updates existed deployment linked with G-Suite Marketplace.

Our Bitbucket Pipelines configuration continuously deploys our add-on to the test environment automatically, whenever master is updated. This makes deployments dead simple, and means we always have the ability to easily test recent changes in a safe environment before deploying to production. It also allows anyone at Atlassian to easily “dogfood” a bleeding edge version of the add-on without asking us to spin up an environment for them. When we’re ready to update our Production add-on, we simply hit the Deploy button in the Pipelines UI: