Creating a theory test that is fit for the future
Enabling people to take the test from any place, any device at any time whilst ensuring the authenticity and integrity of the test.
To become a qualified driver in the UK, the first step is to take the driving theory test. As the government agency responsible for carrying out theory and driving tests, DVSA has been running the theory test since 1996. The assessment is taken by over a million people every year — making it the world’s largest computerised test.
With over 70% of applicants under 25, the theory test is often young people’s first engagement with government. As such, it’s hugely important that DVSA gets it right.
At the moment, this test must be taken in person at a physical test centre. However, that’s no longer suitable for today’s digitally-native customers. Taking the test in this way is also costly for both trainee drivers, and the DVSA as an organisation.
With DVSA’s vision of creating a digitally-enabled organisation, the team wanted to consider other, more innovative methods of running the test.
We conducted a collaborative, five-week proof of value design sprint with DVSA to explore how the test could be administered from tester’s home while ensuring that the test integrity and authenticity were maintained. Prevention of fraud during the driving theory test is crucial due to the high stakes nature of the test. However, it is equally important to provide a truly brilliant customer experience.
By approaching the challenge through a design sprint, we provided a framework for teams of any size to solve and test design problems in a matter of weeks.
Working at speed and scale meant that we needed to define some core design principles, all driven by the DVSA digital, data, and technology (DD&T) Strategy. Those principles were:
• Mobile first mindset
• Efficient and use sustainable technology
• A way of working that naturally promotes collaboration
• Accessible to majority of the general public
• A service iterated through customer feedback
• Simpler, faster, and seamless digitally enabled service
Design sprint steps we followed:
· Understand the current state: To better understand the end to end journey of the driving theory test, we needed to place ourselves in the shoes of the customer. We carried out an as-is analysis of the current journey test takers experience and explored social media, research reports, and other data.
Personas: We created 3 key personas to enable us to understand the users’ key needs, desires, and frustrations. These personas include:
c. Edge cases
· Start-up exploration: To bring in an outside point of view, we explored the start-up ecosystem. Specifically, we examined start-ups focused on authenticating candidates and detecting fraud. Through this process, we came across a state of the art start-up working within the keystroke recognition space that enabled us to authenticate users simply based on the way they type.
· Ideation: To open up the problem space, we conducted a number of ideation sessions exploring how we might solve the problem. This led to the development of multiple hypotheses, including everything from delivering the driving theory test through Virtual Reality to having the test conducted from your car’s computer system.
While it was important to diverge, we needed to come up with an agreed upon hypothesis to define the scope of our proof of value. Two key hypotheses emerged:
· Authenticity: “We believe that the theory test can be taken using emerging technology to ensure the person undertaking the test is the correct person throughout the duration of the test”
· Integrity: “We believe that the theory test can be taken and invigilated remotely using emerging technologies preventing/detecting suspicious and fraudulent behaviours”
When working through the customer journey, we explored what technologies could be used to ensure both authenticity and integrity throughout the test. Working with start-ups and partners like Amazon, Google and Early Metrics, we built a platform that integrated the best breed of technologies, including facial recognition, keystroke analysis, object recognition, background sound analytics, voice authentication, and machine learning for intelligent invigilation.
We also wanted to ensure we included a modulated approach that enabled services to be swapped in or out as part of DVSA’s vision of a future-proof technology platform.
To deliver this, we instilled the following platform principles:
Here’s a list of technologies we recommended to solve potential issues:
· Facial recognition to ensure the person taking the test is the correct person throughout the test. Solution chosen: Amazon Rekognition uses deep neural network models for facial recognition based off the test taker’s DVLA driving licence photo
· Object recognition to identify if the candidates are using forbidden items throughout the exam to aid them in any way. Solution chosen: Amazon Rekognition uses image analysis for object recognition
· Noise detection to identify if the test taker is getting help from anyone in the room or through a device. Solution chosen: a solution developed by our in-house team uses automated flagging of potential collusion through background noise detection
· Keystroke analysis as an alternative way of authenticating users. Solution chosen: Typing DNA uses AI-based behavioural biometrics to identify a candidate by their unique typing pattern
· Voice verification to prevent impersonation during the theory test. Solution chosen: Microsoft Azure offers AI-based voice verification
· Restricted aids monitoring to check if the user was accessing restricted aids (e.g. answer documents, web content). Solution chosen: a solution developed by our in-house team offers user browser monitoring using AI
“Capgemini and the AIE have helped us plot a roadmap to a digital version of the theory test that is both realistic and achievable. The real advantage of the approach that we took was that was presented to us was real feedback, real personas of real people. That prototype has given us the confidence that where we’re aiming is exactly the right place to go,” commented Alex Fiddes, Programme Delivery Executive at DVSA.
DVSA has an exciting vision for the future and is leading the way to create a digitally-enabled government.