A Frontend Engineer in a Backend World
As a software engineer involved primarily in the frontend space, backend ecosystems have often felt mystical, though I’ve always had a soft spot for MongoDB. In part because their documentation is excellent, and the developer experience that the team at Mongo pride themselves on is evidently at the forefront of many design choices. when MongoDB is a NoSQL, document-oriented database solution that was released in February of 2009 and was my first introduction into the world of backend.
So with the year winding to a close, I took a day to attend a MongoDB.local event, which showcased their cloud-based fully managed platform Mongo Atlas, and to learn more about what their cloud services had to offer.
Located at the impressive Tobacco Docks venue in East London, there were multiple tracks, each named after a landmark area in the capital. A short walk from the Oxford Circus space to the Richmond Park room allowed me to engage in talks such as using AI to enrich retail experiences, Database Schema best practices, End to End Design and more.
The conference drew in hundreds of attendees to the point I was able to network on the way to the venue with Paul Done, a leading Solutions Architect at Mongo, who was signing copies of his new book Practical MongoDB Aggregations.
Morning Sessions
The day started at 8am with a breakfast that I skipped because #hotelLife, and the mornings sessions were bookended by a keynote from Mongo CPO Sahir Azam. The speech set the tone for a day full of detail and insight, and was punctuated with a tribute to historic figures in software engineering, and a call to ‘Love Your Devs’. Nice.
The discussion around AI capabilities as expected was deeply explored with plenty of use cases and examples. During the keynote Sahir spoke of an automotive company developing a platform that when coupled with Mongo’s Vector Search enabled mechanics to run engine diagnostics based on a sound bank of engine audio recordings similar to Shazam for car repair; resulting in vehicles being repaired more efficiently based on the identified engine sounds.
Mongo unveiled their AI Innovators Program, a platform for both encouraging and spotlighting companies doing exciting work in the AI field. To further support the developments in AI, MongoDB also announced the harnessing generative AI and natural language to query collections within a database. Goodbye MongoDB Query Language (Mongo’s alternative to SQL queries) and hello ‘Give me average revenue by industry’.
As part of the Mongo 7.0 release, Sahir also announced an industry first in Queryable Encryption, which allows users to run expressive queries on encrypted data. The result is data that’s encrypted throughout its lifecycle and only ever decrypted on the client-side, as such your data has reduced exposure to bad actors.
The sessions were plentiful (at over 60) and with all talks being recorded, missing out wasn’t a concern. One notable instance of tech being a hindrance however was that talk timetables had been replaced by scannable QR codes, which got a little tedious, though the Wi-Fi was top tier.
A major advantage of having such a large event in a concentrated space allowed networking encounters to flourish. I had interesting conversations with meetup organisers, Fintech managers, Venture Capitalists in the Medical space and passionate ambassadors of software services, frameworks and other devs.
Lunch Break
The merch was extremely sparse, yet the food was very good and the talks were even better. At lunch, the bangers and mash with grilled courgette filled stomachs and Asya Kamsky’s talk on Schema Design for Great Application Performance was an afternoon highlight. Using car and traffic analogies to discuss schema design and database operations she allowed me to consider my approach for several side projects that use Mongo’s cloud storage within Atlas.
Main takeaways:
- Schema, Indexes and Queries are the key areas of focus when striving for a performant DB
- Data that is used together should be stored together, though avoid deeply nested and unbounded arrays. Having records from 20 years ago on a product is not wise.
- Avoid polymorphic fields i.e. lots of different data types in one field
- There will always be a tradeoff between reads and writes, so base your decision on your company service level agreement or use case. Is reading the data quickly more or or less important than writing the data quickly?
- Having no index and also lots of indexes on a Mongo database are both red flags.
- A collection without an index is like having a bus route that goes to every house rather than going to a bus stop. The bus stop acts as a common marker so that the bus (or collection) doesn’t have to drive to (or scan) every house (document).
Afternoon Sessions
In amongst the live streaming sessions, vendor booths, talks and networking, the opportunities were an amouche bouche of content. I gained some insights on leveraging MongoDB and AWS in Igor Alekseev’s talk Building Resilient Data Pipelines with MongoDB, Confluent, and AWS.
The audience were encouraged to experiment and build with AWS Step Functions (similar to Azure Logic Apps) that have a plug and play simplicity allowing users to connect APIs and services with minimal code. AI tools were discussed such as Amazon Sagemaker and Amazon Bedrock that allow users to create machine learning models, build and scale generative AI apps. There was also a walk through of Amazon Titan, a service that converts natural language text, including single words and full documents, into numerical representations that can be used to power use cases such as search based on semantic similarity. Alongside these tools I also learned about the concept of AI hallucinations; which occur when a AI training model has no information a query it’s been given, and so the model comes to its own conclusion and generates inaccurate information which is presented as fact.
Rounding off the day was another standout talk by Genevieve Broadhead, Mongo’s Principal for Retail EMEA, on Building Modern Retail Architectures to Blend the Physical and Digital World. My reinterpretation of her talk is this:
One afternoon at work Micah gets a t-shirt delivery at his shop. He sells a t-shirt to Chantelle who used a click and collect service to reserve a tee. The sale is logged in the store-based client app, and then the data is pushed to the cloud updating the national warehouse stock levels. However during the unloading of the fresh new tees, the whole store network fails. Does the location of the t-shirt digitally align with where it is physically? Mongo’s Edge Server helps solve this problem by connecting siloed systems that share the same data.
The idea is that Micah’s Point of Sale (PoS) system would push data to Mongo’s on premise Edge Server, which would then talk to the main Atlas cloud database. If the store’s connection dropped, in-store devices such as PoS systems would continue to send data to the Edge Server. The Edge Server would function as the source of truth until the connection to Mongo Atlas was restored, at which point a bi-directional sync would occur as well as automatic conflict resolution between misaligned data. This system ensures one consistent data platform, and that if a connection is lost, sales can still be made, and stock will be updated when the connection returns.
With the day winding to a close I took a little time to do some people-watching in the warm weather before I made my way back to the underground, my belly and brain both stuffed.
