Improving the Precision of Location Search: The Secret Lies in Monitoring
When the industry experiences growth, the landscape of requirements undergoes a transformation and we have embarked on a remarkable journey. Through this journey, one invaluable lesson has become abundantly clear: the risks of failure and suboptimal performance multiply exponentially.
However, what often eludes our awareness are the insidious developments that grow gradually over time. These subtle shifts may evade our daily monitoring, but when analyzed over the course of a month or a year, their presence becomes unmistakable.
To address these long-term, incremental changes, we have implemented proactive monitoring measures.
Thanks to our monitoring tools, we were able to identify a significant uptick in Google Maps API requests.
In our specific context, the rate of API request growth should ideally correlate directly with the expanding user base. However, as we delved deeper into monitoring each individual request, a subtle anomaly emerged that had previously escaped our notice.
Upon meticulous analysis of the data, we discerned a discerning classification of all the requests into four distinct categories:
- Spelling Mistakes: Constituting a substantial majority at 76.8%, these requests were fraught with linguistic errors. For example, Mumbia, Mummbai were the most common mistakes for the location Mumbai.
- Numerical Dominance: Accounting for 5.14% of the total, this category encompassed requests primarily composed of numbers or containing a higher proportion of digits than alphabetic characters. For example Hno 122
- Academic Identifiers: A niche category, comprising a mere 0.46% of the total, consisting of requests related to specific college names. For example, users were submitting college names as their current city.
- Static Terminology: Representing 17.6% of the dataset, this category encapsulated requests featuring static, recurring terms such as “online” or “work from home,” demonstrating a distinctive pattern.
With this insightful data and a clear understanding of our challenges, we set our priorities straight. Given the relatively small volume of requests falling into the ‘Academic Identifiers’ category, we opted to allocate our resources elsewhere.
With a focused effort, we swiftly devised and implemented code fixes for cases 2 and 4, a relatively straightforward task. However, the real challenge lay in addressing the pervasive issue of spelling mistakes in case 1, which demanded a more comprehensive and intricate approach.
In our quest to tackle the issue of spelling mistakes, we recognized the need for a robust and flexible solution, and Elasticsearch emerged as the ideal choice. With Elasticsearch as our tool of choice, we embarked on a journey of implementation, data set construction and rigorous testing.
Our initial development into Elasticsearch yielded promising insights but fell short of our desired level of success. We refined our data set, making crucial adjustments. This strategic refinement resulted in a noticeable improvement in accuracy, though it still didn’t quite meet our expectations. We proceeded with the third iteration and finally began to achieve the precise and desired results we had been aiming for all along.
We modified our flow to incorporate elasticsearch and made sure that the data in elasticsearch also kept on growing.
As a result, we saw an 80% drop in Google API requests
In conclusion, our journey to address and rectify the issue of spelling mistakes within our data set has been a testament to our commitment to excellence and cost saving. With Elasticsearch as our trusted ally, we navigated through multiple iterations, each marked by learnings and adjustments. Through persistence and adaptation, we not only overcame the initial hurdles but also elevated our solution to a level of accuracy that aligns with our high standards.
