Meet Marvin: Trendyol’s Helpful AI Assistant
Leveraging LLMs to automate our processes
At Trendyol, we operate as one team. This is only possible through open and direct communication in a hybrid working setup. So, Slack is our indispensable tool for day-to-day work. We receive and resolve technical issues, clarify HR-related questions, and onboard new joiners through Slack.
We also strive to be better every day. Not to brag but we achieve better constantly with our Trendyol Ways of Working (WoW) 😎
Adhering to certain standards in our communication and daily tools improves our efficiency. Moreover, it helps us to reduce the cognitive load and context switching during the day so that everyone can focus on the important — creating the business impact and shipping the features.
Bringing those together at the rise of the Large Language Model (LLM) era, Marvin was born:
Our AI Assistant to automate internal know-how search and processes
Let’s dive deep into Marvin’s capabilities and underlying architecture. I promise to keep it not too technical 🧡
- Capabilities
- Architecture and LLM models
Capabilities
Marvin is an LLM-based tool, aiming to automate Trendyol teams’ daily lives. The core capabilities involve replying to technical and HR-related questions.
We have an internal technology portal called Pandora. We use Pandora to accumulate technical know-how, measure developer productivity, and provide visibility of developer teams’ work.
We brought Pandora's know-how one step closer to the developers with Marvin. Now, they can find answers to their technical questions and resolve bugs with Marvin on Slack channels. We used Mixtral 8x7b model and Pandora documents as a source here.
Moreover, if the developer is unsatisfied with the answer, Marvin supports the user with links, allowing developers to search further in Pandora. All in all, our developer teams can find solutions to bugs and technical answers with less effort, with Marvin.
After observing the positive feedback from developers, Marvin started to serve HR teams.
We are growing fast, and welcoming new team members every month. As expected, they are warming up with the team and processes by asking questions during onboarding. Our HR teams are supporting them so that they adapt Trendyol WoW, culture, and internal processes faster. On average, HR teams spend around 35% of their time each month supporting such questions.
We automated this process with Marvin where anyone can find answers to their HR-related questions through a Slack bot, faster. We used our internal HR documents as a source here.
Now, let’s look into the inner workings of Marvin 🧐
Architecture and LLM models
Retrieval Augmented Generation (RAG), Prompt Engineering, and LLMs are at the core of Marvin. We evaluate and store our RAG metrics at Langfuse.
These components enabled us to use a pre-trained LLM with our internal documents. With prompts, we enabled Marvin to know the context it is in. A detailed architecture with the complete list of components is here:
We used Chroma DB as our vector DB. Every night, we ingest documents and store them as embeddings. You can think of embedding as a mathematical representation of the words as a vector and a Vector DB solution designed to efficiently store embeddings.
When LLM receives a question, it provides the answer by finding and retrieving the most similar embedding from Vector DB.
We selected the open-source Mixtral-8x7B model because it outperformed other LLMs in the global benchmarks. This is not set in stone though, LLM space is advancing wildly, and we are following up with the recent models and advancements.
Also, we track cost-related and conversational metrics. We carefully monitor answer relevancy, context precision, context relevancy, and faithfulness at Langfuse.
As the last component, we used Slack adapters for the human-Marvin interaction. This helped us to onboard the team easily to Marvin because Slack is our best friend at Trendyol 👩🏻🤝👨🏼
Marvin is already 10 months old, and 33 teams are using it. We saved 57 hours of our HR teams per month and are receiving positive feedback from our users.
Our user base is growing every day! So we are currently adding more features. Marvin is improving to support more teams at Trendyol, at scale.
Stay tuned to learn more about Marvin, and how we automated the technical assignment review within the recruitment process.
Join Us
Do you want to be a part of our growing company? We’re hiring! Check out our open positions from the links below.
