How We Automated Cross-Organization Tasks at Outbrain
“[Action Required] upgrade log4j to log4j2 — Please upgrade all your repositories and make sure your services are functioning properly after the upgrade’’.
This is one example of a cross-organization task we had a few years ago that was managed via email and sync meetings.
To tell the truth, managing these cross-org tasks among 30 teams and 200 engineers was inefficient, difficult, and frustrating for many engineers due to the tight deadlines and the lack of a familiar standard approach.
For instance, there was no standardized process for generating Jira tickets, so some teams created Jira tickets and progress dashboards by themselves. This process was time-consuming and resource-intensive, and often prone to errors, as tracking the team’s progress was almost impossible.
It was a real Wild West party. :(
The Need for Task Management Automation
Manual task management is a common problem for many organizations, and Outbrain was no exception. Not only did it require a lot of time and effort to identify the relevant components and people, but the process was also riddled with errors or oversights, causing delays.
Additionally, manual task management was a burden on our infrastructure team, as we had to spend valuable hours identifying teams and owners, creating Jira tickets, and tracking progress. As our company grew and the number of cross-organizational tasks increased, we realized that we needed a more efficient and accurate approach.
So, we decided to develop a new system called Just do it, which automates the task management process and provides real-time progress tracking.
Let’s explore how Just do it enables this transformation.
The Initial Steps for the Task Owner
In general — Collect the relevant data to a specific data source (Elastic in our case) and Just do it will use this data to create or update the corresponding Jira tickets.
To get a clear sense of how to leverage Just do it, let’s look at a recent case. At Outbrain, we have recently started one of the largest and most complex tasks we’ve ever had to do– moving all our services, which are based on an in-house framework (Ob1k — The RPC Framework), to the Spring-boot framework. To make the Just do it solution work, the task owner needs to follow only a few manual steps.
Step 1 — Configure and send the data. There are many types of tasks: per Service, Deployment, Machine, Specific file, etc… Each task requires its own configuration, and what Just do it provides is an infrastructure for configuring and sending data to Elastic. In our specific case we only needed to feed a prometheus metric specific to the task being configured. The data was then sent to Elastic, where each event task was assigned an event ID. The aim of this process was to identify the relevant teams and owners that rely on the configuration and data provided.
Step 2 — Create a Kibana visualization to see how much work each owner/team has and track progress. To track progress, you (the task owner) must create mandatory filters. For instance, in the image below, the “HAS_WORK/NO_WORK” filter indicates whether to work on a task or whether the task has already been completed. Here for instance we can see the Kibana visualization for the SpringBoot migration.
For example:
The Kibana visualization for the SpringBoot migration task.
Step 3 — Create a Jira task in the JDI project for the cross-organization task (once per task), and fill in the URL for the Kibana visualization created in Step 2.
For example:
We can see the Jira ticket for SpringBoot migration
The Just do it Automation
Run – The offline job runs every hour and creates all Jira sub-tasks based on the Kibana visualization added in step 2, which also tracks the team progress and closes tickets as the sub-tasks are completed.
Benefits of Task Management Automation
Since implementing Just do it, we’ve seen a range of improvements to cross-organization task management:
- Increased efficiency: The time and effort required for manual task management has been significantly reduced, allowing our infrastructure team to focus on higher-level tasks.
- More accurate tracking: The Kibana dashboard provides real-time updates on task progress and completion rates, ensuring that everyone involved in a task is aware of its status.
- Improved communication and collaboration: By automatically assigning tasks based on guild membership and providing progress updates, Just do it improves communication and collaboration between teams.
- Better task prioritization: With a clearer view of task progress and completion rates, we’re able to prioritize tasks more effectively and address any bottlenecks more quickly.
For example:
This is a visualization of an organizational task to migrate our services to SpringBoot
Overall, the Just do it system has been a game-changer for Outbrain’s task management process. With the Just do it system in place, we’ve eliminated the headache of manual task management and empowered our backend guild to focus on what they do best — building innovative solutions. By embracing automation and streamlining our workflows, we’ve set a new standard for efficiency and collaboration at Outbrain. If you’re struggling with manual task management, it’s time to take the leap and explore the world of automated solutions.
For the sake of your team and your company’s bottom line — just do it.
