AI for Tech Support: The Way to Cut Down on Routine and Boost Your Team’s Productivity

Technical support plays a crucial role in product development by handling critical tasks such as resolving issues, cataloging feature requests, and prioritizing user demands. This support is vital for ensuring a product’s success and responsiveness to external feedback. GitHub is cited as a significant platform for collaboration between development and technical support teams, with many industry leaders like Microsoft, Amazon, and Google using it. The 15–10–5 rule in Scrum methodology allocates percentages for technical debt, bug fixes, and research, emphasizing the importance of efficient technical support management.

The article suggests that AI-powered tech support tools might offer a solution to reduce time and expenses in technical support while improving customer satisfaction.

The Challenges Faced by Technical Support Teams

Besides resolving issues and fixing bugs, teams often handle routine tasks that AI for tech support could potentially automate.

1. Sorting and managing similar and related issues is a common challenge for tech support teams, as they often encounter issues already addressed by the development team, and finding previous solutions in databases takes significant time.
2. Issue classification, including determining urgency and differentiating genuine issues from feature requests, can be time-consuming for tech support teams.
3. Identifying syntax errors in user-provided code is a significant challenge for support specialists, and it can result in wasted time when trying to replicate issues that stem from syntax or versioning errors.

We utilized OpenAI’s GPT-3.5 for data preparation and generating predictions in our LLM backend. This was complemented by traditional machine learning and statistical methods for analysis.

Duplicate search

We’ve developed a duplicate search method using LLM encoders to transform parsed issue data into machine-readable embeddings. This method then merges vector similarity search with TF-IDF encodings to derive a unified analysis metric.

What is TF-IDF?

TF-IDF stands for Term Frequency-Inverse Document Frequency. It is a numerical statistic used in information retrieval and text mining to measure the importance of a word in a document relative to a collection of documents (corpus).

Term Frequency (TF) measures how frequently a term (word) appears in a document. The intuition behind TF is that the more often a word appears in a document, the more important it is to that document.

Inverse Document Frequency (IDF) measures how important a term is across a collection of documents. Terms that occur rarely but in many documents contribute more to the overall score.

TF-IDF Score for a term in a document is obtained by multiplying its TF and IDF scores. The resulting TF-IDF score reflects how important a word is to a document in a collection or corpus. High TF-IDF scores are attained by terms that appear often in a particular document but rarely in the entire corpus, indicating that those terms are unique to that document and are likely to be meaningful and relevant in the context of that document.

After identifying potential duplicate issues, we employ GPT to confirm the similarity scores fall within our designated parameters and determine that the tickets address the same issue or request.

Duplicate search step-by-step process

Now let’s see an AI-powered support tool in action. As the support team receives a new issue (in our case, it’s issue #108 in the “setup-python” repository), instead of looking for a similar solution manually, the tech specialist can specify it and use the tool to look for related or similar issues.

Based on the similarity score, the tech support specialist can identify the issues that match the set criteria. In our case, the AI-powered tech support tool has highlighted issue #678 as the one with the highest similarity score.

Priority estimate and code snippet analysis

Our AI-driven tech support tool excels at determining issue priority and analyzing code snippets for potential syntax errors. We’ve developed a system that extracts code snippets from GitHub issues, primarily focusing on syntax and import errors. Typically, this code includes build logs, runner-image configuration in YAML, or node/python code. Since users occasionally submit configurations with errors, we deploy LLM to pinpoint and correct these inaccuracies.

The model readily detects blatant syntax errors, though there’s a limited number of such errors in the target repositories.

It’s also important to note that intricate errors, dependent on information not directly available in the issue text, are harder to identify. This encompasses details from user repositories using GitHub actions, local deployments, code installations on user systems, and other data outside the ticket’s content.

Code snippet analysis example

AI-powered tech support assistant is capable of detecting issues in the code snippets provided within the issues, saving time that would otherwise be spent on troubleshooting an error.

First-line answer generation

One of the standout features of our AI-powered tech support assistant is its ability to respond to common GitHub issues. We’ve grounded our method on training GPT-3.5 using comments from resolved tickets.

Initially, we simply parsed the first response from closed tickets, thinking these answers would bolster the model’s capability to forecast clear-cut issue responses. However, this approach didn’t meet our accuracy standards, leading us to adopt manual answer labeling.

First-line answer generation example

After the support specialist defines the issue, the LLM model generates an answer prediction.

AI-Powered Tech Support Assistant Implementation Results

Our team implemented AI for tech support internally to assess its effectiveness. Based on our findings, this tool conserves around 20 hours monthly for a team of 5–6 members, solely through its Duplicate Search feature. In general, using AI tool for technical support can save your team up to 40 hours monthly.

Such time savings not only boost team efficiency but also offer potential cost reductions in product support. Notably, the tool upholds quality while simultaneously enhancing user experience. This innovation holds promise for numerous businesses that use GitHub repositories and are committed to their support.

Addressing Common Concerns AI-Powered Support Usage Might Arise

As often happens with emerging technology implementation, certain concerns might prevent teams from adopting it — especially when considering the automation of user interactions.

Ease of use

Introducing new technology always comes with concerns about steep learning curves or potential ineffectiveness. Fortunately, with the AI-powered tech support assistant, these worries are alleviated. The tool integrates smoothly with Slack, ensuring that tech support specialists work within a familiar interface, making adoption straightforward and efficient.