Tech Stack Roundup #2: March 2017

It’s that time of the month again! On the first Thursday of every month, we gather the top tech stack related blog posts from the past month written by some of the best companies in the world (sign up for weekly updates here).

For this issue, we’ve gathered posts from Uber, Spotify, Stripe, Discord, Segment, WePay, Sentry, Dropbox, and more.

The dominant theme for March: AWS. More and more companies are writing about their experience on AWS- a clear sign that its still the go-to infrastructure platform, and its not showing any signs of slowing down.

Top 10 Tech Stack Posts:

The million dollar engineering problem (Segment)

https://segment.com/blog/the-million-dollar-eng-problem/

Segment had a big problem. Like many companies, they use AWS for the majority of their infrastructure needs. After defaulting to adding more services and more resources whenever they needed to move faster, Segment knew they needed to make a change when their AWS bill was 2/3 the cost of their cost of goods sold 💸. After some serious receipt examination, they were able to shave off a cool $1M from their annual AWS bill!

Authors: Achille Roussel and Rick Branson

Tech Stack: Amazon DynamoDB, Amazon EC2, Amazon ECS, AWS ELB, Terraform

How Discord Indexes Billions of Messages (Discord)

https://blog.discordapp.com/how-discord-indexes-billions-of-messages-e3d5e9be866f

Last month, Discord made the February Roundup for their post on how they store billions of messages. In this follow up post, Discord outlined how they index those messages, using none other than Elasticsearch.

Authors: Jake Heinz, Lead Software Engineer at Discord

Tech Stack: Elasticsearch, Solr, etcd, Lucene

How New Relic Architects the Frontend for Faster Deploys and Better Workflows (New Relic)

Having a huge monolithic Rails app is so 2016. Nowadays, that type of architecture will slow you down big time- having to deploy your backend whenever you change your UI sucks if you’ve got a large codebase. New Relic finally had enough and decoupled their front end from their backend. This is how they did it (cool graphics included).

Authors: Edwin Knuth, Senior Software Engineer at New Relic

Tech stack: Rails, JavaScript, Amazon S3, GitHub, Jenkins, Fastly, Webpack, Babel

Powering UberEATS with React Native & Rewriting Uber Engineering’s Android Rider App with Deep Scope Hierarchies (Uber)

https://eng.uber.com/ubereats-react-native/

If you’re still on the fence about React Native, check this out:

We were able to get UberEATS up and running quickly by leveraging much of Uber’s existing technology stack…For React Native to be a viable option for us, it was also important that it work within our existing mobile infrastructure and support the kinds of features that had originally prompted our move towards a native application. In order to do this, we built a ‘demo’ application tailored towards verifying critical features. This included our ability to pull in native dependencies from other teams at Uber to test functionalities, including crash reporting, user authentication, and analytics…Overall, the demo was able to deliver our desired outcome.

So yeah, UberEATS is React Native.

Uber also decided to re-write their Android Rider app so they could take advantage of “deep scope hierarchies.” If you don’t know what that means, you’re not alone. tl;dr:

Deep scope hierarchies enable applications like our rider app, with its feature-dense screens and shared objects between subscreens, to increase separation of concerns, reduce possibilities for stale data, and increase developer velocity.

Authors: Chris Lewis, Software Engineer at Uber, and Brian Attwell, Senior Software Engineer at Uber

Tech stack: React Native, Redux, Android SDK

Get this post along with new dev tools delivered to your inbox every week

Scaling your API with rate limiters (Stripe)

https://stripe.com/blog/rate-limiters

Few companies have built an API as reliable and performant as Stripe. One secret to their success is a set of carefully implemented rate limiting strategies. They take the time to outline these techniques in this recent post, and even include a gist to get you started.

Authors: Paul Tarjan, Software Engineer at Stripe

Tech stack: Ruby

Streaming databases in realtime with MySQL, Debezium, and Kafka (WePay)

https://wecode.wepay.com/posts/streaming-databases-in-realtime-with-mysql-debezium-kafka

WePay wanted to integrate their microservices and downstream data stores with each other so every system could get access to the data it needed. After some research, and thanks to some help from Yelp, they were able to leverage Debezium and Kafka connect to stream their MySQL databases into Kafka.

Authors: Chris Riccomini, Principal Software Engineer at WePay

Tech stack: Kafka, Debezium, MySQL, BigQuery

Spotify’s Love/Hate Relationship with DNS (Spotify)

https://labs.spotify.com/2017/03/31/spotifys-lovehate-relationship-with-dns/

Spotify built their own DNS servers and infrastructure 😱. Learn all about how they did it, how they monitor their DNS, and their plans to move away from their homegrown solution.

Authors: Lynn Root, Site Reliability Engineer at Spotify

Tech stack: Java, Cassandra, Google Compute Engine, Git, Pingdom, Docker, Helios

Dodging S3 Downtime With Nginx and HAProxy (Sentry)

https://blog.sentry.io/2017/03/01/dodging-s3-downtime-with-nginx-and-haproxy.html

Remember that big S3 outage back in February? Seems like years ago in Internet time. Shortly after the outage, the good folks over at Sentry decided to document how they’re making sure they don’t have to rely on S3 so heavily moving forward. If you use S3, you don’t want to miss this.

Authors: Matt Robenolt, Director of Basketball Operations at Sentry

Tech stack: HAProxy, nginx, Amazon S3

How to Scale PostgreSQL on AWS–Learnings from Citus Cloud (Citus Data)

https://www.citusdata.com/blog/2017/03/10/how-to-scale-postgresql-on-aws/
Citus is a distributed database that extends (not forks) PostgreSQL for large workloads

When the postgres scaling experts tell you how to scale pg on the dominant cloud platform, you listen. In this post, Citus Data details how they launched their cloud offering on AWS.

Authors: Ozgun Erdogan, Co-founder at Citus Data

Tech stack: PostgreSQL, Amazon EC2, Amazon ELB, Amazon EBS

A Simple Pattern for Jobs and Crons on AWS (Skyliner)

https://blog.skyliner.io/a-simple-pattern-for-jobs-and-crons-on-aws-2f965e43932f

If you want to process jobs in the background without too much complexity with the tools AWS already gives you, listen up. Skyliner gives you the step by step playbook to make it happen (code samples and screenshots included).

Authors: Dan McKinley, Cofounder at Skyliner

Tech stack: Amazon SQS, Amazon CloudWatch

And now..for the Hidden Gems.

Get this post plus the latest updates on tools you love by subscribing to StackShare Weekly, featuring the best hand-curated new dev tools and infrastructure services.

Leave any questions you may have for the OPs above in the comments and we’ll make sure they see it :)