ASOS Tech GenAI Hack: Revolutionising requirements with GenAI
In the world of software development within ASOS Tech, the quest for streamlining and efficiency whilst not sacrificing on quality is a constant journey. We recently held a GenAI Hack Day within ASOS, where we asked for ideas to business challenges that GenAI could help solve.
We had 5 different teams (made up of different disciplines from across ASOS Tech) who were given different challenges and would come together on the day to tackle the problem and (hopefully) build out solutions for them. We had challenges that were themed around Product Attribution, an ASOS Tech Chatbot and more.
The idea I had was around the challenge of crafting good testable requirements. I recently attended a talk, and 16.9% of defects arise from “bad” requirements. I felt that if we could use GenAI to create requirements for us, which teams can then refine and understand, it will save time in creating requirements from scratch. You can’t edit a blank page right?
The Team
We had a team of 4, and armed with our various skill-sets — from App Development, Platform Engineering, Software Engineering and Test Engineering, we were committed to work together to solve the challenge presented to us.
We had a good idea of what we wanted to do, we knew we could utilise Service Hooks within Azure DevOps which could send the User Story to an Azure Function. The Azure Function could then make a request to GenAI and ask for requirements based on the User Story, and finally the Azure Function could then update the User Story in Azure DevOps with the new description and requirements. Sounds simple, right?
The Hack Day itself
With one of us in the office, and the other 3 remote, we decided to run a mob programming session for the whole day. This we felt would enable us to utilise everyone’s strengths, and made sure that everyone felt a part of the journey and that they were contributing.
We stayed on a Teams call all day, only breaking for lunch (🍕 Pizza was of course provided in the office), and together we built out the ADO Service Hook, the Azure Function, and refining of the prompt. The heart of the solution was the Azure Function, this tied everything together. We initially thought about using a Logic App, which whilst would have achieved the same goal, we felt that with an Azure Function, we could utilise source control and it was a better developer experience where we could actually write code.
The Example
By the end of the day we had created something that met the initial brief and more.
- A User creates a User Story in Azure Dev Ops. For the sake of this example, it’s relatively generic, and involves migrating Cypress tests to Playwright.
2. User comments on the story “/create-requirements”
3. User Story is updated with a new description and requirements. User Story is also updated with a tag of “Updated By GenAI” so you can see it’s been updated accordingly.
In this example it takes out the requirements from the description and does a good start for the team to then expand further as and when required.
The Challenges
No hack day is complete without a unique set of challenges. Whilst we initially thought that my team all being remote would have been difficult, it actually ended up working out well, staying on a Teams call all day and having the support of 3 other Engineers at any point in time is definitely a good thing.
The first challenge was in mapping the original ADO request, we had trouble mapping the parameters appropriately, and the fact the ADO description field had HTML in it as well. We decided to leave that in for the GenAI prompt, but as a future improvement, we could strip out the HTML potentially.
The second challenge was around refining the prompt to get exactly what we wanted out of it. This proved challenging because to start with we were making a change, redeploying the app and then running it end to end. We worked around this in the end through utilising the chat in the UI and then refining it there. Whilst we didn’t get the whole flow of updating the ADO User Story we could see what the response was going to be like to give us a good idea. Whilst we played around with a lot of prompts, the prompt we ended up with (and this could yet still be refined) was:
Given the following title and description for a user story.
Generate a description (if empty),or update it to make it clearer
of the ticket (to be defined as ##description)and a set of
acceptance criteria (to be defined as ##requirements).
If it makes sense to,also add in some Given When Then statements that
support the requirements (to be defined as ##gwt).
For each individual requirement, wrap it <div> tags.
For each gwt requirement, wrap it <div> tags.
Do not put your response inside a code fence/block.
title: {originalTitle},
description: {originalDescription}
Return the ##description and ##requirements in the response.
To be formatted as follows:
<div><b>Description:</b> ##description <br /></div>
<div><b>GWT:</b> ##gwt <br /></div>
<div><b>Requirements: </b> ##requirements <br /></div>The above prompt incorporated Given When Thens into the response, this would only be activated if the User Story had a tag of “GWTRequired”, as we appreciated that not every story would want GWT’s. If a story didn’t have that tag, then the prompt was the same, minus the GWT statements.
The final challenge was around the temperature of the model, with the initial, default temperature sometimes responding with code blocks around the response, meaning that it would appear in ADO as a code block. This wasn’t ideal. To fix this we changed the temperature to be closer to 0, which meant the response from GenAI would be closer to what the actual ask of it was.
Next Steps
As we wrapped up our Gen AI Hack Day, we reflected on what we had achieved. Beyond the outcome itself, our journey was one of innovation and collaboration that drives progress in the software development landscape. By using GenAI and Azure Functions, we had taken a massive step towards redefining how requirements are generated. Helping teams become more efficient, without compromising quality.
There are still steps that we need to undertake, in terms of productionising what we have created. Including, creating a pipeline for it, a managed identity and some error-handling before rolling it out further across ASOS Tech. There’s also the potential of making the model more context aware and to take into account other quality attributes (such as Performance, Security, Accessibility etc) in order to create more detailed requirements.
Gareth is a Principal TestEngineer, helping to make ASOS Tech and ASOS QA rock 🎸🤘. He’s also a Sunderland AFC season ticket holder, a football coach and massive sports fan.
