Set: 01Path: Technologies: Henosis: AutomataFriday, 15 September 2017

Automata

Invisible at its core is both a service and a technology company. We take technology very seriously without fetishizing it. Technology is just a tool, one of many we have, to further the goals of our business. It just so happens that it is one of the best tools for what we’re trying to accomplish.

Invisible has been around for nearly two years now, starting humbly with just one product: a consumer app for saving and organizing snippets of text through email. We’ve since shifted most of our tech to be internal-facing. Our software makes our team members more efficient, so that they can better serve our clients. This shift has empowered us to be nimble and iterate rapidly in response to feedback, since our users are (mostly) our team members.

We’ve built, and let die, a lot of tech. The pieces that stuck around are the ones that were battle-tested and actually provided utility that justified their upkeep. The most ambitious piece of tech we’ve built is called Automata.

Why

At Invisible, we do your work for you, so you can do your real work.

Imagine you are hiring for your company and have to schedule a day of onsite interviews for a candidate. The process has the following constraints:

• The candidate must speak with the CEO and CTO of the company for one hour.
• The candidate must meet with three other people relevant to the position for one hour each.
• The candidate will have lunch with the team for 1.5 hours.
• All meetings must have a conference room reserved.
• Interview times can override recurring meetings like Standups, but these should be minimized.

Once the schedule has been assembled that meets those constraints:

• Create the relevant calendar invites and send them out.
• Write an email and send it to the Candidate, giving them an overview of their schedule.

This seems like a fairly straightforward, while detailed, process, and a skilled executive assistant could probably manage it in a couple hours. Now, imagine scheduling tens of such interviews, having many instances of this task open at the same time. It’s likely that even the most diligent person will eventually make a mistake, skip a step, or drop the ball in some way.

In our business and personal lives, we do a lot of things every day that could be abstracted out into repeatable processes.

If you were the executive assistant executing this task, your first stab at abstracting it out might be to just make a detailed checklist, and then go through it step by step. This is already a huge improvement over trying to keep the entire process and its constraints in your head, but we can do better.

Computers have some major advantages over humans in that:

• They never forget.
• They never get bored.
• They won’t skip steps.
• They have no trouble keeping many things open and separate.
• And with AI and Machine Learning and a large enough data set, they can even do classification and prediction tasks.

Computers have some major disadvantages over humans in that:

• They are poor at making judgment calls that don’t fit within defined parameters.
• They are not (yet) good at creative tasks, or ones that require human empathy.
• They don’t always have all the information, and it’s not always possible nor practical to parameterize everything.

If we were to create a perfect system for executing detailed, repeatable tasks, using technology that is available today or in the near future, what would it look like? We would want to maximize the benefits of computers and minimize the costs, while doing the same for human actors that are part of this system.

Hence, we created Automata, a system that abstracts out repeatable processes and executes them, allowing the computers to automate away the steps that can be automated, and coordinate the human beings at key times when their input is needed.

Example

A computer with access to the entire company’s calendar could figure out a potential schedule for a candidate much faster than a human being. But, how do you tell the computer which people at the company are relevant to the candidate? How do you tell the computer which meetings are OK to override? How would a computer incorporate information that is not necessarily stored in the calendar, like that a particular person is (unscheduled) out sick for the day?

You could theoretically try to automate this process 100% by building a more robust calendar system that mapped to an organizational chart and contained every piece of metadata needed to build this schedule. You could create an elaborate decision tree that accounted for every edge-case scenario you could think of. But, this would still require fallible human beings to keep their calendars and org chart 100% up to date, and it would also require us to invest heavily in creating these systems around the calendar software already in use. It would be impractical, as we would have shifted the burden of doing this task to the rest of the company to manually update their data, and it would still invite mistakes.

A better approach might be, let the computer do what it’s good at, and let humans check its work. Empower the computer to suggest potential schedules, reducing the possibility space, and let humans look it over and make changes before sending the final email on behalf of your company.

This would greatly reduce the amount of active time a human has to spend on this process, allowing them to increase their throughput and/or spend their time on other more valuable tasks, while simultaneously reducing the error rate and amount of dropped balls.

How

That sounds nice and all, but how do we do it, really?

Playbooks

Our first challenge was, we needed a way to represent these detailed, repeatable processes in a way that a computer would understand. Even more importantly, we wanted this representation to be accessible to our employees who are not engineers.

We did a quick survey of existing tech in this space, but none fit our needs. So we created our own domain specific language, which we simply call Playbooks.

Playbooks, first implemented just as Google Sheets that follow a specific format, can be as simple as just a list of steps, but can also:

• Ask the Client for input at key points and validate this input
• Ask the Agent for input
• Store and retrieve data in variables
• Generate template text with these variables
• Make decisions and execute different branches of the process based on this input or other parameters
• Execute simple loops
• Call other Playbooks (what we like to call Handoffs)

For any programmer, this seems straightforward enough, as you would expect any programming language to be able to do these things. Our key insight was that we had to make it simple enough that non-programmers (the people close to the actual execution of a process) could easily read, write and edit Playbooks.

It doesn’t take long to train someone to write these, and once the initial investment is made, the Playbook can be executed over and over.

Automata

Playbooks themselves are useful in that they force us to think about things step by step. But you still need the computer to actually parse and understand the Playbook, so that it can execute it. Because we do most of our coordination and communication over Slack, we decided that the natural fit for this would be a Slack bot that ties into our backend. We call this bot Automata.

Automata reads, parses, and validates Playbooks, letting the Playbook writer know of any errors or potential infinite loops. It also saves every version of a Playbook, so we can see how it has evolved over time, or rollback to a previous version if needed.

Through a series of commands, Humans tell Automata to start running a given Playbook for a client. Automata will execute each step, asking for input when necessary, or just displaying info back to the Client or the Agent. Automata is smart enough to skip steps that are not needed (it won’t ask questions it has the answer to).

In the future, Automata will get more integrations with existing technology, enabling it to fully automate some steps by just making API calls, making the execution of these processes even more efficient. For example, instead of just telling the Agent to send a calendar invite with the given parameters, it just makes the calendar invite itself and asks the Agent for confirmation.

Automata is also fairly sophisticated in that it can manage many open Instances of a Process at once, seamlessly switching among them when asking the client for input.

Human Agents

Computers can’t do it all. As mentioned, it’s often impossible or impractical to give the computer all of the data it needs to fully execute a task, and things often come up that simply cannot be programmed for ahead of time. Thus, we allow our Agents to pause a process at any time, directly set variables, communicate with the Client if necessary, or escalate the problem to another Agent with more access or context.

This is a key part of the system and I consider the Policies and training of agents an important piece of this technology.

Conclusion

Automata is the most direct expression of our thesis. We put our money where our mouth is and invested a lot of engineering and design time into it. Our next challenge is: how do we efficiently turn processes into Playbooks? More on that later.

Next