Automating translation context screenshots with Storybook and Playwright

Photo by Alvaro Reyes on Unsplash

When you have a business that serves customers all around the world, one thing that’s really important to get right is translations. At Bounce (we’re hiring!), we currently support almost 20 languages across our different apps, and we often got feedback from our translators that they needed more context so that they could make sure the translations were good. The best way to do that would be to provide screenshots, but that traditionally involves a huge amount of manual work. It would mean we’d need to:

• Go through the app and try to trigger every possible scenario a user could end up in (e.g. a failed booking).
• Take screenshots of all those scenarios.
• Upload the screenshots to Crowdin, our translation management tool.
• Go through each screenshot and tag which translations keys are used in it.

It would take ages. The best people to do the job would be engineers because they’re the ones who create the keys and would therefore know which keys to tag in each screenshot. But no engineer wants to do that much manual work, so we worked on a way to automate the whole flow.

What we need to be able to automate adding translation context

1. Go through the app and take screenshots of everything

There are plenty of end-to-end testing tools let you open URLs and take screenshots. But that won’t be enough: apps are interactive, and we’ll only be able to get all the screenshots we need by running through a whole bunch of scenarios. Writing end-to-end tests that go through all of those scenarios would be a huge amount of work and get difficult to maintain.

Luckily, there’s another way to go about this: Storybook. We use Storybook heavily in our development process. It gives us a way to develop screens and components in isolation without needing to run the app. One thing that really comes in handy here is being able to create stories for any UI states we want. If you think of test coverage as being how much of your code is covered by tests, we have a pretty high UI coverage by having stories for lots of different scenarios that appear in our apps. It makes it the ideal tool to use here, because instead of writing end-to-end tests that go through all the steps that lead to a particular scenario, we can just open a story where everything is mocked and ready to go.

2. Figure out which translation keys are being used

The screenshots are only useful if we know exactly which keys are used in them. We use i18next to handle translations in our apps. Whenever we want to translate something, we call the t function with the translation key (e.g. t("screens.payment.addPaymentMethod", "Add payment method")). So we just need to find a way to track every key that gets passed to the t function in each story.

3. Upload the screenshots to Crowdin and tag the keys that are used in it

Once we’ve got our screenshots and we know which translation keys are being used in each one, we’ll need a script to upload all that info to Crowdin.

In summary, we need to:

• Use an end-to-end testing tool to take screenshots of things in our Storybook.
• Track which keys are used in each story by hooking into i18next.
• Write a script to upload that info to Crowdin.

Since the whole solution depends on being able to track the translation keys, let’s look at that first.

Tracking translation keys

To track which translation keys are called in a story, we made a custom i18next post-processor. It’s nothing too clever: whenever the t function is called, the processor checks which translation key was passed to it and saves it to an object.

// This is where we store the translations for each story, separated by translation file.
let storyTranslations: Record<string, string[] | undefined> = {}

/*
 * This is a custom post-processor. i18next will call this every time
 * we translate something.
 */
const extractUsedKeys = {
  type: "postProcessor",
  name: "extract-used-keys",
  process(
    value: string,
    translationKey: string,
    { ns: namespace }: { ns?: string },
  ) {
    if (!namespace) return
    // Add the translation keys separated by namespace/file name.
    const nsKeys = storyTranslations[namespace] || []
    nsKeys.push(translationKey)
    storyTranslations[namespace] = nsKeys
    return value
  },
}

We then added this to our custom i18next Storybook decorator so that it could be used in all of our stories.

import { Decorator } from "@storybook/react"
import { useEffect } from "react"

import { i18next, I18nextProvider } from "../src"

/* ... */

export const WithI18next: Decorator = (Story, context) => {
  /**
   * After rendering the story, wait for a little bit before setting the translations
   * on the window. We can increase the delay in each of the stories if we want
   * to wait longer, e.g. for animations to finish.
   */
  const delay = context.parameters.someCustomParams?.delay ?? 50
  
  useEffect(() => {
    const timeout = setTimeout(() => {
      Object.assign(window, { storyTranslations })
    }, delay)
    return () => {
      clearTimeout(timeout)
    }
  }, [delay])

  return (
    <I18nextProvider i18n={i18next}>
      <Story />
    </I18nextProvider>
  )
}

After our story renders, the translations that are used in it are set on the window, ready for the end-to-end tool to pick up. The result looks something like this:

{
  customer: [
    "cmp.durationField.label.checkIn",
    "cmp.durationField.label.checkOut",
    "cmp.durationField.open"
  ]
}

Programmatically visiting stories and taking screenshots

To be able to upload the screenshots of all of our stories, we first need to know how to find those stories. Luckily, whenever you create a Storybook build, it produces a stories.json file (index.json in Storybook v8), which essentially works like a sitemap. It gives us the ID of each story, which we then use to create the URL we need to visit that story (iframe.html?id=the-story-id).

Next, we need to be able to programmatically visit each of those URLs in the browser. Playwright is intended for writing end-to-end tests, but it works perfectly for this. We can instruct it to:

1. Visit a URL (in this case, a story).
2. Wait for the storyTranslations object to appear on the window.
3. Read that value, and save the translations in a JSON file.
4. Take a screenshot.

We can even tell it do it for different browser sizes, which is important because some translations only appear at particular screen sizes.

In the end, we end up with a folder structure like this:

.
└── translations-context/
    └── [story-id]/
        ├── desktop/
        │   ├── translations.json
        │   └── screenshot.png
        └── mobile/
            ├── translations.json
            └── screenshot.png

We now have everything we need to start uploading screenshots to Crowdin and tagging them with the correct translation keys!

Uploading the screenshots to Crowdin

In our case, we created a NodeJS script and use the Crowdin SDK for interacting with the API. The script goes through the folder structure outlined above and does the following for each story ID, screen size and translation file/namespace mentioned in the translations.json file:

1. Delete any existing screenshots stored for the current story ID, screen size and translation namespace.
2. Fetch information from Crowdin about the translation file and the strings in them.
3. Filter the strings information from Crowdin to include only the strings that appear in the screenshot.
4. Upload the screenshot using a fixed naming convention to make deleting/updating it later easier. In our case, auto__[story-id]__[screen-size]__[translation-namespace].png.
5. Tag the screenshot with the IDs of the strings that appear in it.

In our project on Crowdin, we can see all the screenshots we’ve uploaded under “Screenshots”:

The number next to the checkmark in the thumbnails shows how many keys were tagged in that screenshot.

And our translators can check the screenshots in the translation editor by clicking on the thumbnail under the source text:

How it fits into our workflow

Whenever we merge a pull request, we have a CI pipeline that checks which parts of the UI might have been affected by the files that were changed. It then re-takes/re-uploads screenshots for those parts to make sure that translators always see the latest version of the UI where the text will appear.

Closing thoughts

The result is pretty cool. Something that would have been a really laborious process is now taken care of without us even thinking about it, and our translators have a ton of context to refer to while working on translations.

The solution isn’t perfect though. Not every translation key will get a screenshot because we don’t have stories for every situation, e.g. error messages that only get shown after taking an action, such as logging in with an incorrect verification code. We might also delete and re-upload screenshots unnecessarily. But it’s a good first step towards having a fully automated way of providing context.

We’re hiring! 💙 Check out our careers page for all open roles.