Taste the World: How Our New Machine Translation Feature Transforms Your Ordering Experience
Authors:
Ahmad Hamouda, Software Engineer IV at Glovo
Stefania Russo, Head of UX Content at Glovo
Introduction
Glovo’s mission is to “give everyone easy access to anything in their city”. Being live in 24 markets, we aim to provide a seamless experience without language barriers for all our customers.
While the interface of our apps and platforms are already localized in the languages of every country we operate in, there was still one more language barrier to overcome: our restaurant menus and store product lists, which are monolingual, were not translated into the user preferred language.
Imagine: you’re a native English speaker who lives in Barcelona and doesn’t speak Spanish. Your phone is in English and so is the Glovo app.
You are hungry on a Friday night and while browsing Glovo find all restaurant menus are either in Spanish or Catalan. There is no simple way in the app to translate the menu and having to copy/paste into Google Translate is frustrating. You’ll likely end up ordering something that you’re not sure of or close the app and go pick up the food yourself.
By giving our users the possibility to see a translated version of our menus in the language of their mobile devices, we can increase our reach and penetration in every market as well as improve the overall user experience.
In this article, we will give an overview of the solution we built, a deep dive into the localization challenges, and how we are measuring success.
Customer pain point
Glovo operates in countries with a high percentage of immigrants, so a monolingual catalog automatically prevents a significant number of users from placing an order due to language inaccessibility.
The lack of a translation solution for our restaurant menus and store product lists was already a known major pain point in many of Glovo’s countries.
On top of this, whenever a user had their phone in a language that was different from the country’s main language, they were exposed to an unexpected multilingual experience they did not choose.
Users prefer content in their language
In 2020, the content and language services firm CSA Research published ”Can‘t Read, Won‘t Buy” summarizing people‘s attitudes towards using products in their language versus other languages. The results were eye-opening:
- 65% prefer content in their language, even if it‘s poor quality
- 67% tolerate mixed languages on a website
- 73% prefer products with information in their own language
- 66% use online machine translation
- 40% will not buy from websites in other languages.
Our Product catalog challenges
Catalog ownership
Menu and store catalogs are owned by the partners and even though clear global guidelines are provided, partners choose the language and format of their catalogs.
Language mix and language detection
While most countries have catalogs in the same local language, some partners in multilingual countries choose to duplicate their menus on Glovo to offer a multilingual experience to their customers (ie. in Georgia many restaurants have their menus in both Georgian and Russian).
This quick solution on the partners’ side, to compensate for the lack of a translation feature for menus, becomes a challenge when trying to establish a scalable translation feature for menus. Most menu items consist of one or two words and language detection in such small units of text is a big technical challenge.
Catalog dimensions and updates
Our restaurant and store catalog contains millions of products and they undergo frequent updates.
Menus and product catalogs are structured into names (ie. Pizza Margherita) and descriptions (ie. ingredients like tomato, mozzarella, or basil). Partners can also organize content in sections, collections, and super-collections (ie. Top sales, Combos, Starters, Salads, Classic pizzas, etc.).
Catalogs are updated frequently depending on the type of business: seasonal menus, special offers, changes in product offerings, etc.
These updates occur across all markets, for all partners, every single day. Millions of menu and catalog entries are updated daily.
The Solution
It was clear that what we needed was a real-time Machine Translation (MT) solution, integrated into our systems. We aimed to translate our menus and catalogs into the user’s device language via an API; without requiring any human intervention.
As the result of an effective and rewarding cross-team collaboration between Localization, Engineering, and Product we closed the gap between the localized interface and the product catalog language.
Getting Started
We collected all the necessary requirements to select a third-party Machine Translation provider that could fit our needs.
Main considerations:
- Machine Learning Model customization: while the Machine Translation had to happen real-time, we needed a system to customize the machine learning process based on Glovo-specific content requirements and existing content
- Machine translation coverage for not-so-common language combinations like Armenian and Georgian into English and Russian
- Adaptive technology that could quickly learn from user feedback
- Low latency and high availability: maintaining low latency for personalized customer experience and stringent SLAs to ensure service reliability to avoid customer experience degradation
- Quality monitoring
- Scalability
- Cost-effective solution
- Robust data processing capabilities.
Our Machine Translation Partner
ModernMT is the provider who won the selection process, as it met our requirements in terms of tech solution, quality customization needs, and cost effectiveness.
ModernMT is an adaptive neural machine translation system and one of the top-rated in the market, developed by Translated.
ModernMT was recently recognized as a leader in the IDC MarketSpace for machine translation software, ahead of the likes of Google, Amazon, and Microsoft. Currently, ModernMT supports 200 languages, reaching over 6.5 billion native speakers worldwide.
The MVP
Enabling machine translation of the product catalog touches many phases of the customer journey from the moment a customer starts their search for a product until they pay for their order at checkout.
We adopted a lean approach to get started, so we prioritized enabling the machine translation feature on the store screen [Figure 1] to make sure our customers could get the most out of it.
We adopted the same lean approach when deciding where to first roll out the feature. For the aforementioned reasons, we began with two countries most in need for a machine translation solution: Georgia and Armenia.
After the first roll-out and applying a number of learnings from the initial trial, the feature was scaled to the store pages/screens for the remaining countries.
We determined the language combinations for enabling the machine translation feature based on the most used device languages in each country.
Overview of the feature
Figure 1
Localization deep-dive: the Machine Translation Customization effort
Customizing a machine translation engine means providing it with relevant material so it learns from it and becomes better over time.
In our case, it involved injecting samples of correct product translations and language glossaries into the engine so that it learns from them and applies the learnings across the whole system.
This practice is extremely important when the content to machine-translate consists of short units of text, such as menus and product lists, because the machine doesn’t have a lot of contextual text to support its choices. Long sentences or paragraphs provide the machine with better context and therefore imply less training and a faster learning process.
The customization work has been divided into the following phases:
Phase 1: Machine Translation Engine Pre-training
MT engines are fed with the following datasets for each language combination:
- Existing translation memory databases
- Exports of top-selling products for each language and market
- Do-Not-Translate glossaries (a list of terms which we never want to translate)
Phase 2: Sample Human Reviews
ModernMT learns dynamically and continuously. So, beyond the pre-training step above, we extracted samples of our product catalogs for each market, processed them through the engine, and had human translators perform a linguistic review.
This step allowed us to directly feed corrections and feedback directly into the ModernMT engine.
Phase 3: Machine Translation Glossaries
A glossary, in the context of Machine Translation, is a tool that facilitates the consistent translation of customer-specific terminologies, giving advanced control over the terms used.
ModernMT has an MT Glossary feature integrated into their API. This allows us to create Glovo-specific terminology lists per country that can help us boost the quality and nuances of the machine translation engine output.
Our MT glossaries include:
- Universal food terminology (ingredients, dishes, generic products, etc.)
- Local ingredients
- Local dishes
- Big Food chains
- Q-commerce products
The MT glossaries are a live asset, which will be updated regularly to make sure we keep up with the product catalog updates.
Phase 4: Continuous Feedback Loop
Being able to quickly implement any feedback is crucial for us and for our end users.
For this reason, we implemented an in-app feedback solution thanks to which Glovo employees using our beta app are able to easily report any wrong machine translation by clicking on a button.
The clicking of the button sends synchronized API requests along with necessary metadata to the TranslationOS platform, which triggers a human review by a professional translator.
Once the review is completed we receive the corrected text back via API, which is then integrated internally into our ModernMT model.
This feedback loop combined with the adaptive MT model aims at a continuous improvement of our solution.
The Engineering deep-dive: building a dedicated microservice
At first glance, connecting third-party services directly with ours may appear straightforward. However, our stringent requirements related to security, data control, quality assurance, and cost reduction make this integration more complex than it seems.
Our primary goal is to maintain exceptionally low storage latency while minimizing it as much as possible, all while retaining control over our Service Level Agreements (SLAs).
To navigate these challenges, we’ve implemented an asynchronous approach to manage client requests.
When a client queries us, we promptly provide the data if it’s available. If not, we notify the client of data unavailability while presenting the original text.
Simultaneously, we initiate an asynchronous request to ModernMT for the translation, storing it in our database. As a result, subsequent requests for the same word and language combination are instantly served from our storage. Although this method incurs a failure for the initial translation request, it significantly reduces costs by approximately 95%.
This cost reduction is attributed to two key factors:
- We translate only a minimal part of our entire catalog (items visible to users).
- We reuse the same translation without incurring additional translation costs for every subsequent request.
We use a scoring system to decide when to display translated content. If 85% of our items are translated, we show the translations. This helps us ensure that our pages stay relevant, and we can adjust the threshold based on our evolving business needs.
To make things easier for our users, we’ve introduced an auto-translation feature. When a user’s device language matches one of the supported languages configured for their current country (ie. your device language is English and you’re in Spain where English is a supported target language), the translation happens automatically without any input from the user.
We’ve designed this to accommodate different language preferences, providing an inclusive experience for all users.
Tech Mid-Level Solution: Making Informed Technology Choices
As we planned our service, we started by figuring out how much storage we needed, predicting the traffic mainly from the store screen, and setting our SLA goals.
In this process, we looked at three options, each with its own pros and cons.
Database:
Redis:
Redis emerged as an initial contender due to its cost-effectiveness compared to DynamoDB and its superior speed when compared to a standalone MySql setup. However, its challenge lies in data persistence. While there are more advanced options available, such as Elasticache with persistence, they come with increased costs.
DynamoDB:
Although DynamoDB offered speed, ensuring stable access patterns and understanding Read Capacity Units (RCU) and Write Capacity Units (WCU) were critical requirements.
SQL:
We considered SQL solutions, which seemed cost-effective, but using them might require adding Redis for extra features. After careful thought, we decided to start the service with Redis. This lets us gather data on reads and writes, validate new features, and plan for the future based on data. Our iterative approach allows us to continuously improve the project.
Even though we designed models for DynamoDB and SQL, we structured our data model so that switching from Redis to either DynamoDB or SQL in the future is still possible.
This decision gives us the flexibility to adapt based on metrics and user feedback, ensuring the service remains reliable and efficient.
Communications: Streamlined Machine Translation Services
To improve our machine translation services, we divided our system into two separate modules. Each module focuses on key aspects, including deployment, source code, and infrastructure:
Event-Based Module
This module functions on an event-driven architecture, allowing asynchronous communication and enabling the retrieval of translations from our service providers. This setup ensures effective handling of various translation requests by separating functionalities, promoting scalable, and independently configurable operations.
API Module
On the flip side, our API module handles user traffic and oversees fallback from TranslationOS, guaranteeing a responsive interface for users. This split allows for customized scaling and the use of distinct configuration libraries. This modular approach results in cleaner code, a secure platform, and improved organization. It also lays a solid foundation for adapting to future changes and scaling requirements.
Our reliance on Kafka, an asynchronous technology within Glovo, ensures the smooth operation of our event-based module. Additionally, we’ve incorporated robust features like retries, limiters, and circuit breakers in our communication protocols with ModernMT and TranslationOS. These measures are crucial for adhering to their limitations, respecting service capacities, and managing fallback scenarios effectively.
This detailed approach not only ensures optimized communication channels but also strengthens our system’s resilience, guarding against potential service disruptions and enhancing overall reliability.
Clean Up and Refresh Data
We manage our data in Redis with specific Time-to-Live (TTL) settings, which we refresh based on usage patterns. This means that frequently accessed items have their TTLs extended. However, if we update it with every access, we will have too many unneeded updates on our storage. For this reason, we decided to use a statistical model to extend the TTL only [ex: randomly once every 10 accesses]. This helps reduce write operations and overall costs by keeping them available for a longer time.
Moreover, our feedback loop lets us update and enhance specific items with poor translation quality as explained earlier.
As our machine learning model progresses, there comes a time when data cleanup becomes necessary. To address this, we’ve created an internal API for data cleansing based on language pairs or countries. This API becomes active when we reach a specific threshold or introduce a significant amount of new data to our glossary, ensuring efficient maintenance management.
Security
Ensuring security is a top priority for us, especially when dealing with third parties and managing incoming data that directly affects our company’s reputation and cost center. To minimize risks related to these interactions, we perform thorough risk analyses and make informed choices regarding password rotation, sharing practices, and identity validation.
Through close collaboration with ModernMT, we’re implementing crucial changes related to passwords and tokens. Their support and flexible approach have proven invaluable in strengthening our security measures to protect our systems and data.
Measuring Success: Impact in Numbers
Keeping track of the metrics below is pivotal in measuring the impact of our machine translation solution. This will help us refine strategies, confirm impact, and continuously improve the service based on user behavior and feedback.
Conversion Rate (CVR)
CVR is a key measure to evaluate how well our translation services work. By comparing the conversion rate before and after implementing translations, we can see how it affects user engagement and actions like purchases or interactions on the platform.
New Customer Acquisition
Measuring new customer acquisition serves as a lever in attracting foreign customers and expats to our platform. Tracking the influx of new users post-translation implementation helps quantify the service’s effectiveness in broadening our user base. It provides concrete data on how well our translation solutions resonate with a diverse audience, reflecting our ability to attract and retain foreign users, expatriates, and newcomers to the platform.
Served Orders with Machine Translation
Counting the orders handled through machine translation gives a clear sign of how widely the service is used and its practicality. Keeping an eye on this metric helps us understand the extent of user interactions made possible by translated content, highlighting its role in making transactions smooth.
Increased Orders
The noticeable increase in order placements directly linked to the introduction of machine translation indicates the impact of the service on user behavior.
This metric clearly shows how translated content positively affects user engagement, leading to a boost in platform activity.
Customer Satisfaction
Collecting and analyzing customer feedback on the machine translation feature provides qualitative insights that helps us continue improving the experience of our users.
Customer Retention
Assessing changes in customer retention rates after implementation allows us to gauge the impact of the service on user loyalty.
Future Developments
Our goal is to expand the machine translation feature to additional stages of the customer journey, ensuring a smooth user experience across the entire platform.
