Amazon Connect: Scaling Development and Testing — Part 1
By Cody Jenkins
If you work in the call centre space, you’ve probably heard of Amazon’s cloud contact centre offering Amazon Connect. And if you’re based in Australia you might also have heard that Amazon made Connect available here in early 2017. A diverse range of companies including NAB, Square and Subway (to name a few) have already started migrating parts of their call centre operations to Amazon Connect.
If you’re like us, you’ve already had a play with Connect yourself, and you might have even set up your own basic “Contact Flow”. But how can we take it to the next level and get some real value out of this service?
The “Hello, world!” of Amazon Connect
A basic Connect set-up might have a simple conversation flow like this:
- An existing customer, Jen, dials the support number and is connected to the main Contact Flow.
- The Contact Flow plays a short welcome message and prompts Jen to state the reason for her call in a few short words.
- The Contact Flow delegates to Lex to determine what Jen is asking for.
- Once Lex matches an intent, the Flow routes Jen to the queue to speak with an agent.
This is a bit of an upgrade from the usual dial-pad entry entry menu. But there is a lot more that Connect has to offer.
How Amazon Connect can save agent time
There likely will be situations in which Jen needs access to some sensitive information. In cases like that, the agent will need to perform an identity check with Jen before continuing. An identity check might require Jen to provide her customer number, date of birth, and postcode for verification. Once the agent collects the information and verifies it, they may continue the call. While this identity verification is a critical step in protecting Jen and other customer’s private information, it is a time consuming one.
This is a perfect opportunity for Amazon Connect to shine.
Connect provides us with a large set of tools to automatically collect information from the customer, validate it and package it up to hand off to an agent. In the case of identity verification, we may design a Contact Flow in the Connect GUI which authenticates Jen before she speaks with an agent.
The benefit here is obvious: we can save the time agents spend authenticating users over and over and redirect it to more important conversations that are best suited for people to handle.
Easy win, right?
How do we scale development?
When we start thinking about productionising this solution, a natural question arises: how does Connect fit into a continuous deployment strategy? How can we run automated tests to detect regressions in logic? How can we promote changes through dev, test and prod and roll them back in case of error?
Answering these questions will allow us to build a strategy that empowers teams to develop Connect solutions at scale.
What if I don’t want to use Lex for intent matching?
Just because you’re using Amazon Connect doesn’t necessarily mean you’re happy to shelve your existing AI assistant (eg., Rasa) and rebuild everything in Lex. Is it possible to integrate Amazon Connect with a different Natural Language Understanding (NLU) solution?
What we set out to answer
The value proposition of Amazon Connect is clear. But before we scale up development on our solution, there are a few questions we need answers to.
- Can we build a test automation framework for validating the Contact Flow logic?
- Can we use the AWS API to automate creation and configuration of Connect and the Contact Flows?
- Is there some way to “pass through” the conversation from Connect to another NLU solution like Rasa?
We’ve spent the last two weeks trying to answer these questions. Here’s what we found.
Investigating automated testing
We spent some time looking to see what other groups had done to test their Contact Flow logic. The most useful resource we found was from the Amazon Contact Centre blog. While this post covered some good techniques for creating test cases, the steps to run them were almost all manual. And while manual testing is better than no testing, we had our hearts set on automating as much of the process as possible.
One decision that we settled on after some early failures was to test the Contact Flow logic via the chat interface rather than the voice interface. In the beginning we experimented with some pretty hilarious flows in which one Contact Flow would dial out to another Contact Flow and run a dialog back and forth. It felt a bit like that scene in Taxi Driver where Robert De Niro talks to himself in the mirror. But we realised that we were spending unnecessary effort testing the voice-to-text and text-to-voice aspects of Connect when we could make do with the chat interface.
Thankfully, the Connect API in this regard allows us to run a Chat session in its entirety. We are able to initiate a chat session with the same Contact Flow that powers the voice interaction and exercise the logic through that interface.
We encapsulated the chat protocol logic into a simple Python module which we could import into a test in pytest or another Python testing framework to validate the Contact Flows. To validate the logic, we wrote tests to have multiple back-and-forth interactions with the Contact Flow to test various call scenarios.
As we experimented testing different flows, we found that some of the Contact Flow blocks error out for chat inputs and are not testable without voice inputs. One import block that always errors out is “Store Customer Input”. This block always follows the “Error” path in the Contact Flow. Another block type, “Get Customer Input”, fails when it gets customer input using “DTMF” (from the dial pad). This setup also goes down the “Error” path.
With these limitations in mind, we moved the failing Contact Flow blocks out of our “Contact Flow under test” and focused on testing just the conversation logic. This worked quite well and made it possible to fully automate testing the more complicated flows! Hooray!
Investigating automated deployments
With most AWS services, AWS provides APIs which allow us to fully automate the creation and management of instances and configurations. While Amazon Connect does expose some functionality through the API, as of June 2020 we are not yet able to create or manage Connect instances or Contact Flows. This prevents us from performing fully automated deployments with Connect, and that makes it impossible to make deployments repeatable. If you get something working in your pre-prod environment, you won’t be able to promote it automatically to your production environment.
If you look closely at the Contact Flow GUI, you’ll see that there is one option which saves us from re-creating everything manually. In the top-right corner of the interface and to the right of the “Save” button, there is a chevron you can click to expand a context menu. In this menu there are options “Export flow (beta)” and “Import flow (beta)”. You may export the flow to JSON and then re-import it in another Contact Flow. Using this tool, you can manually export the configuration from your pre-prod environment and then manually import it to your production environment. However, if you use different Lex bot and Lambda instances in your pre-prod and production environments, you’ll have to manually re-configure the contact flow to point at the correct Lex bot and Lambda instances. And that’s a bit of a pain.
It’s a bit of a disappointment, but as of today there is not an out-of-the-box solution to do continuous deployments of Contact Flows.
To be continued…
We managed to get quite a few successes and failures out of the first part of our investigation. As we complete the journey in Part 2, you’ll see there was one unexpected twist in store for us! But for now, here’s where we got to:
Can we build a test automation framework for validating the Contact Flow logic?
Yes, we can use the Chat API to exercise the logic. But some Contact Flow blocks are problematic. By moving those blocks into separate Contact Flows, we may achieve some level of automated testing on the remaining flows, albeit with a long feedback loop. It is worth mentioning, though: you still have to “Publish” a flow before you may test the logic.
Can we use the AWS API to automate creation and configuration of Connect and the Contact Flows?
No, but there might be another way. Given the limited Connect API today, it’s not possible to automate Connect or Contact Flow deployments. But as you’ll see in part 2, we discovered an interesting “loophole” that made us rethink the question altogether!
