How I earned $1M with no business experience, and spent it to create a brand-new translator
How it all started
10 years ago I worked as a software engineer and later quit the job to start my own projects. To save some extra money, I went to a small hometown, where I used to work on a website for students, accounting software and mobile games at the same time. Having no business experience caused some issues with generating income, so all projects had to be closed. I came back to the capital to get a job, again. The story repeated itself a few times.
When I was broke again then faced a total economical crisis. I couldn’t find a job and it felt awful. It was a good reason to see the world through sober eyes. I had to honestly admit that I didn’t know what niche to choose for my business. Doing projects you like seemed like a way to nowhere.
The only thing I was capable of is creating mobile applications. Several years of work in tech companies allowed me to gain some useful experience, so I decided to make fundamentally different apps (games, music, art, health, lifestyle, languages) and test the market needs. Prepared sets of assets and code libraries made it possible to simply create applications on various topics: 2d games, GPS trackers, simple utilities, etc. Most of them had several pictures, 2 buttons, and only one function. But it was enough to test the idea and monetization model. For example, a running app tracked the person’s speed, distance and burned calories. Nothing more. Purchasing graphics on stocks and source code re-usage helped me to create hundreds of simple applications over 2 years.
At first, the applications were free. Then I added ads and in-app purchases, picked up keywords and bright icons. Users started to download my apps.
When the income reached $30K per month, I offered a friend from a large tech company to work together since I was able to earn money on test applications. He said that they have only one application — a game with a monthly income of $60K and 25K users vs $30K revenue and $200K users I had. This completely changed my vision. Turned out that it was better to create one high-quality application than one hundred poor.
10X income from high-quality apps is kinda obvious, but I was alone in a small city with no experience and a team of designers and marketers. I had to pay rent and try to make a living. My applications were designed to test market niches and advertising strategies to learn what exactly I should create. It just happened that some of my apps began to generate revenue. Now you can’t make money with such apps.
Some applications differed in profit: translators, navigation for trucks, music simulators (piano, drums, guitar chords, players), as well as simple casual games.
Then I noticed that in just a month, translators had been downloaded more than 1M times taking 100th position in the ranking category. There are hundreds of languages in the world, and people enter a query for every language. The niche turned out to be promising.
About 40 simple translators were later created using Google API. It cost me $20 per 1M characters. Then appeared improved versions of apps, where I included ads, in-app purchases, and voice translation.
I earned enough money to move to big city and buy a house. By that time, I had 50–70 translation applications and 5M downloads in total. User growth increased the cost of the paid Google Translate API. So business profitability has seriously decreased. Paying users translated blocks of 1K characters at a time, which forced us to limit their requests. When they faced this translation limit, they left bad reviews and got their refunds. 70% of the income covered our expenses. With large translation volumes, this business wasn’t promising. To recoup expenses, it was necessary to add advertising to applications which always scares users. That’s why we needed our API for translation.
I tried to ask startup community for advice and investments but got no support in response. Most people didn’t understand why we work in a market where there is already a giant leader — Google Translator.
Besides Google, several companies provided an API for translation. I was ready to pay $30K for their technology licenses in 40 languages. This would allow me to translate an unlimited number of times for a fixed price and serve any number of users on my servers. But in response, I got the amount several times higher than expected. It was too expensive. I decided to recreate their technology for translation. I invited some friends to help me with development, but then most of them had families, small children, and loans. Everyone wanted stability and a good salary instead of joining a startup. I had no experience in public speaking, charisma, or a cool prototype application to get people interested. Earning $300K on test translation applications didn’t surprise anyone.
I turned to a friend who owns an outsourcing company. At the end of 2016, he allocated a team for me. I expected to solve the problem in six months on an outsourcing basis, not to depend on Google’s API.
On the way to my translator
The work has begun. In 2016, we found several opensource projects — Apertium, Joshua and Moses. It was a statistical machine translation suitable for simple texts. From 3 to 40 people supported these projects. Later it became clear that we need powerful servers and high-quality datasets which are expensive. Even after we spent money on hardware and a quality dataset for one of the translation pairs, the quality left much to be desired.
Technically, it didn’t boil down to the “download dataset and train” scheme. It turned out that there are a million nuances that we were not even aware of. We tried a few more resources but didn’t achieve good results. Google and Microsoft don’t disclose their achievements. Nevertheless, the work continued, freelancers joined the company.
In March 2017, we found a project called Open NMT — a joint development of Systran, one of the leaders in the machine translation market, and Harvard University. The project was just launched and offered translation based on new technology — neural networks.
Modern machine translation technologies belong to large companies and they are closed. Small players, realizing how difficult it is to integrate, don’t make such attempts. This inhibits market development. The quality of the translation among the leaders differed not so much from each other. Obviously, large companies also faced a shortage of enthusiasts, research projects, startups, and open source projects in order to take new ideas, hire people, cooperate.
Therefore, Systran made a bold move: they shared their developments in open source so that enthusiasts like me could get involved in this work. They created a forum where their experts began to help newcomers for free. And it brought a good return: startups, scientific works on translation began to appear since everyone could take the basis and conduct their experiments on the basis of it. Systran led this community. Then other companies joined.
At that time there was still no ubiquitous neural translation, and Open NMT offered their developments in this area, outperforming statistical machine translation in quality. I and other guys around the world could take these technologies and ask experts for advice. They willingly shared their experiences, and this allowed me to understand in which direction to move.
We took OpenNMT as a basis. This happened in early 2017 when it wasn’t perfect and contained only basic functions, purely for academic research. A lot of bugs were found, everything worked slowly, unstably and crashed under light loads. In our group chat, we all tested together, tracked errors, shared ideas, gradually increasing stability (then there were about 100 of us). At first, I wondered: how is it so, why does Systran grow its competitors? But over time, I understood the rules of the game, when more and more companies began to publish their developments for processing natural language in open source.
Even if everyone has the computing power to handle large datasets, the question of finding specialists in NLP (Natural Language Processing) is acute on the market. In 2017, this topic was much less developed than image and video processing. Fewer datasets, scientific papers, specialists, frameworks and more. There are even fewer people who are able to build a business and close any of their local niches from NLP research papers. Both top-tier companies like Google and smaller Systran players need to gain a competitive edge over players in their category.
How do they solve it?
It may seem strange, but in order to compete, they decide to add new players to the market. For them to appear there — you need to make the market attractive. The entry threshold is still high, and the demand for language processing technologies is growing fast (voice assistants, chatbots, translations, speech recognition, and analysis, etc.)
Nowadays there are public papers published by Google, Facebook, Alibaba. Their frameworks and datasets are open sources. Forums are created with Q&A.
Large companies are interested in such startups as ours developing, capturing new niches and showing maximum growth. They are happy to buy NLP startups to strengthen their positions.
After all, even if you have all the datasets and algorithms in your hands — this doesn’t mean you will make a high-quality translator or another startup in the NLP vector. And even if you do, then it’s far from the fact that you get a large piece of the market pie. Therefore, you need help, and if someone succeeds, buy or merge.
In March 2018, Systran invited the entire community to Paris to exchange experiences, and also hosted a free master class on the main problems that startups in translation usually face. Everyone was interested to see each other in real life.
Everyone had various projects. Someone created a bot-learning English, which talks to you like a person. Others used Open NMT to summarize the text. A significant part of startups represented plugins for SDL Trados Studio, tailored to a specific topic (medicine, construction, metallurgy, etc.) or language to help translators save time on editing the translated text.
In addition to enthusiasts, the guys from eBay and Booking arrived in Paris, who created a translator on the same platform as us, but optimized it for auction and hotel descriptions.
Also in May 2017, Facebook posted its solution for machine translation of Fairseq into open source along with trained models for tests. But we decided to stay on OpenNMT, watching how the community grows.
DeepL history
In September 2017, while analyzing competitors, I learned a lot about DeepL. They just started and already provided translation in 7 languages. DeepL was positioned as a tool for professional translators, helping to spend less time on proofreading after machine translation. Even a small quality change saves a lot of money for translation companies. They constantly monitor the API for machine translation from different suppliers, since the quality in many language pairs is different for everyone and there is no single leader. Although by the number of languages, Google has the most.
To demonstrate the translation quality, DeepL decided to run tests in some languages and published this article
Quality assessment was carried out by blind testing, when professional translators choose the best translation from Google, Microsoft, DeepL, Facebook. According to the results, DeepL won, the jury rated its translation as the most “literary”.
How did it happen?
Founders of DeepL own Linguee startup — the largest database of links to translated texts. Most likely, they have a huge number of datasets gained by parsers, and to train them, you need more computing power.
In 2017, they published an article on how they had launched a 5-petaflops supercomputer in Iceland being 23rd in terms of performance in the world by that time. To train a large-high-quality model was only a matter of time. At that moment, it seemed that even if we buy high-quality datasets, we will still never be able to compete with them without such a super-computer.
But everything changed in March 2018. Nvidia launched a DGX-2 computer sized nightstand and 2 petFlops (FP16) performance, which can now be leased starting at $5K per month.
Having such a computer, you can train your models with giant datasets quickly, and also keep a large load on the API. This dramatically changes the power balance of the entire machine learning startup market and allows small companies to compete with giants in the big data field. It was the best offer on the market in a price-performance ratio.
I started looking for information on DeepL statistics. For 2018, Google had 500M MAU (monthly active users). DeepL has 50M.
At the end of 2018, 10% of the monthly Google audience used DeepL, and they were not particularly advertised anywhere. In just over a year, they captured 10% of the market using word of mouth.
I thought about it. If the DeepL team defeated Google with a team of 20 people, having a supercomputer of 5 petaFlops in 2017, and now you can cheaply rent a supercomputer in 2 petaFlops and buy high-quality datasets, how difficult will it be to achieve Google quality?
Lingvanex Control Panel
(Here are some technical details of how our translator is made. If you are non-technical person , please scroll down to read the rest of the story )
To quickly deal with translation and stop running tests from the console, a Dashboard was created that allowed us to do all tasks, from preparing and filtering data to deploying translation tests. In the picture below: on the right is a list of tasks and GPU servers on which models are being trained. In the center are the parameters of the neural network, and below are the datasets that will be used for training.
Working on a new language began with datasets preparation. We took them from open sources such as Wikipedia, European Parliament, Paracrawl, Tatoeba and others. To reach an average translation quality, 5M translated lines are enough.
Datasets are lines of text translated from one language to another. Then the tokenizer splits the text into tokens and creates dictionaries from them, sorted by the frequency of meeting the token. The token can be either single characters, syllables, or whole words.
After the datasets were uploaded to the database, it turned out they have a lot of words with errors and poor translation. To achieve good quality, they must be strongly filtered. You can also buy already high-quality filtered datasets.
When the language is deployed to the API, you need to set a list of available functions for it (voice recognition, speech synthesis, image recognition, parser of a file, site, etc.)
Continue the work
In 2018 I spent my time on solving the problem of high-quality translation in the main European languages. I thought I needed another six months for everything to work out. I was limited in resources, only 2 people were involved in data science tasks. It was necessary to move fast. It seemed that the solution to the problem was something simple. I wasn’t satisfied with the translation quality. I had already spent about $450K earned on old translators, and it was necessary to decide on what to do next. Running this project alone and without investment, I realized how many management mistakes I made. But if a decision was made — it’s not the best time to give up!
I noticed that our community started talking about a new architecture for neural networks — Transformer. Everyone rushed to train neural networks based on this Transformer model and began to switch to Python (Tensorflow) instead of the old Lua (Torch). I decided to try it too.
We also took a new tokenizer, pre-processed the text, started filtering and marking up the data in a different way, otherwise processing the text after translation to correct errors. The rule of 10K hours worked: there were many steps to the goal, and at some point I realized that the translation quality was already enough to use it in the API for my applications. Each change added 2–4% of quality, which wasn’t enough for the critical mass where people continue to use the product instead of using competitors’ solutions.
Then we started connecting various tools that allowed us to further improve the quality of translation: named entity recognition, transliteration, specific dictionaries, a system for correcting errors in words. After 5 months of hard work, the quality in some languages became much better and people began to complain less. It was a turning point. You can already sell the software, and since you have your API for translation, you can greatly reduce costs. You can increase sales or the number of users, because your only expense is computing power.
To train a neural network I needed a good computer. But we saved money. We rented 20 regular computers ( each equipped with a GTX1080 video card) and simultaneously launched 20 simple tests on them through the Lingvanex Control Panel. It took a week for each test, it was a long time. To achieve better quality, you had to run with other parameters that required more resources. We needed a cloud computing and more video cards on one machine. We decided to rent a cloud service Amazon 8 GPU V100 x 4. It was fast, but very expensive. We started the test at night, and in the morning got a bill for $1200. At that time, there were very few rental options for powerful GPU servers besides it. I had to abandon this idea and look for cheaper options. Maybe try to create my own?
We began to consult with the team and decided that it is possible to make a computer using several powerful GPUs and at a price of up to 10K dollars, which will solve our problems and pay off in a month. Two weeks later, everything was ready.
At the beginning of 2019, I finally assembled this computer at home and began to conduct many tests, without worrying about what I need to pay for cloud services. I began to notice that the English-Spanish translation is close to the Google translation according to the BLEU metric. The computer buzzed all night, it was impossible to sleep. It was necessary to ensure there were no errors in the console. In the morning, I ran a test to translate 100 sentences with lengths from 1 to 100 words and saw that it turned out to be a good translation, including long sentences. This night has changed everything. I saw the light at the end of the tunnel and realized I can still someday achieve a good translation quality.
Here you can find our Translator Demo:
Improving application quality
Earning money from mobile translator apps with one button and one function, I decided to improve their quality, as well as make a version for Android, Mac OS, Windows Desktop. I was hoping that when I have my translation API, I will finish the apps development to enter other markets. But competitors went much further. Some core functions and features were needed.
The first thing I decided to do was offline voice translation for mobile applications without Internet access. This was a personal issue. For example, you go to Germany, download only the German package to your phone (400 mb) and get a translation from English into German and vice versa. Internet access abroad could be an issue. Wifi is often not available, slow or otherwise not useable. At the time, in 2017, there were thousands of high-quality translation apps that required an internet connection to use Google API.
Some technical notes
Since many had problems with the Lua (Torch) version of OpenNMT due to not popular programming language, the founders transferred the logic of the translate.lua script to the C++ version (CTranslate), which was used for more convenient translation experiments. On the Lua version, it was possible to train models, on the C version, to run. By May 2017, it was already possible to somehow use it as the basis of production for applications.
We ported CTranslate to work for applications and put it all in open source.
Here is the link to this thread:
https://github.com/hunter-packages/onmt
Porting CTranslate to different platforms is only the first step. It was necessary to understand how to make offline models of small size and normal quality to work on phones and computers. The first versions of the translation models occupied 2GB in the phone’s RAM, which obviously didn’t work.
I found guys in Spain with good experience in machine translation projects. For about 3 months, we jointly conducted research in the field of reducing the size of the neural network model for translation, to achieve 150 mb per language pair and then run on mobile phones.
The size had to be reduced so that in a certain size of the dictionary (for example, 30 thousand words) to embed as many options as possible for translating words of different lengths and topics.
Later, the result of our research was made publicly available and presented at the European Machine Translation Association in Alicante (Spain) in May 2018, and one of the team members got PhD on it.
At the conference, many people wanted to buy a product, but only one language pair was ready (English — Spanish). Offline translation on neurons for phones was ready in March 2018, and it was possible to do it in all other languages until the summer. But I hadn’t enough time and money. I had to pause this feature. A year later, I returned to it and completed it.
In addition to translating text, voice and pictures, we decided to add phone calls translation with transcripts, which competitors didn’t have. We knew that people in different countries often use mobile or landline phones to call support. And for call addresses there was no need to install the app. This function required a lot of time and expenses, so we put it in a separate application. This is how we launched the Phone Call Translator. Here you can check this app.
Phone Call Translator for Android
Translation apps had one problem — they are not used every day. There are not many situations in life when you need to translate daily. But if you study the language, the use of a translator becomes frequent. To learn languages, we created the cards, when words are added to bookmarks on the site through the browser extension or movie subtitles, and then knowledge is consolidated using the mobile chat bot application or the smart speaker application that will check the selected words later. All Lingvanex applications are connected via single account, so you can start translating on a mobile application and continue on your computer.
Also added voice chats with translation. This will be useful for tourist groups, when the guide can speak their language, and each of the visitors will listen in translation. And finally — the translation of large files on the phone or computer.
Business Strategy
To create an API for your applications and invest a lot of money, you need to understand the volume and prospects of the machine translation market. In 2017, there was a forecast that the market will become $1.5B by 2023, although the market volume for all translations will be $70B (by 2023).
Why such a difference — about x50?
Let’s say the best machine translation now translates 80% of the text well. The remaining 20% must be edited by a person. The biggest expenses in translation are proofreading, that is, people’s salaries.
Increasing the translation quality even by 1% (up to 81% in our example) can figuratively reduce the cost of proofreading text by 1%. 1% of the difference between the market of all translation minus the machine will be (70–1.5 = $68.5B) or $685M already. The numbers and calculation above are given approximately to convey the essence.
That is, even a 1% improvement in quality can significantly save large companies on translation services. As the quality of machine translation develops, more and more of it will replace the manual translation market and save on salary costs. It’s not necessary to try to cover all languages, you can choose a popular pair (English-Spanish) and one of the areas (medicine, metallurgy, petrochemicals, etc.). 100% quality — the perfect machine translation on all subjects —is impossible to attain in the short term. And each subsequent percentage of quality improvement will be more difficult.
However, this doesn’t prevent the machine translation market from occupying a significant part of the total market by 2023 (by analogy with DeepL, it imperceptibly grabbed 10% of the Google market), as large companies test various translator APIs every day. And improving the quality of one of them by a percentage point(for any language) will allow them to save money.
The large companies’ strategy of open-sourcing their products began to bear fruit. There are more startups, public papers, and people in the industry now, which allowed us to rock the market and achieve better translation quality, increasing our forecast for the machine translation market.
Every year, competitions are held on NLP tasks, where corporations, startups and universities compete who will have better translation in certain language pairs. By analyzing the list of winners, there is confidence that small resources can achieve excellent results.
Company incorporation
The project has grown. Applications have appeared not only for mobile platforms but also for computers, wearable devices, instant messengers, browsers, voice assistants. In addition to translating text, a translation of voice, pictures, files, websites and phone calls was created. Initially, I planned to make my translation API to use only for my applications. But then I decided to offer it to everyone. Competitors went ahead and had to keep up.
Until that time, I managed everything alone as an individual, hiring people to outsource. But the complexity of the product and the number of tasks began to grow rapidly, and it became obvious that you need to delegate functions and quickly hire people to your own team in your office. I called a friend, he quit his job and decided to open the Lingvanex LLC office in March 2019.
Until this very moment I never promoted the project I created. When I decided to assemble my team, I faced a hiring problem. Nobody believed that this could be done at all. I had to interview many people by talking for three hours about thousands of unobvious details. When the first article about the project came out, it became easier. I was always asked one question: “Why are you better than Google?”
At the moment, our goal is to achieve the quality of Google translation with the main European and Asian languages and after that to provide solutions for:
1) Translation of text and sites through our API x3 times cheaper than competitors, providing excellent support service and easy integration. For example, the cost of Google translation is $20 per a million characters, which is very expensive at volume.
2) High-quality thematic translation of documents on certain topics (medicine, metallurgy, law, etc.) by API, including with integration into tools for professional translators (such as SDL Trados)
3) Integration into the business processes of enterprises to run translation models on their servers under our license. This allows you to preserve data confidentiality, not depending on the volume of the translated text and optimize the translation for the specifics of a particular company.
You can make translation quality better than competitors for specific language pairs or topics. You can do everything. This is a matter of company resources. With sufficient investment, there is no problem. What and how to do — it’s known that you just need hands and money.
In fact, the NLP market is growing very rapidly in recognition, speech analysis, machine translation improvement, and can bring good profit for a small team. All hype here will begin in 2–3years when today’s market promotion by large companies will bear fruit. A series of mergers/acquisitions will begin. The main thing at this moment is to have a good product with an audience that you can sell.
Summary
Over the years I earned about $1M in revenue from simple mobile apps, and spent most of profit to create my own translator. In hindsight it’s now obvious that everything could’ve been done. Many managerial mistakes were made, but this is experience, and then there was no one to consult with. The article describes a very small part of this story, and sometimes it may not be clear why certain decisions were made. Ask questions in the comments below.
At the moment, we haven’t achieved the quality of Google translation, but I don’t see any problems doing this if the team has at least a few specialists in Natural Language Processing.
Here is the list of new translation apps that have been developed over 3 years and in which money was invested.
Translator Bots for Messengers
If you have any questions, ideas for joint partnerships, investments and offers — add me on Facebook or LinkedIn.