Randomly infrequent tales of interest from the serverless cloud mines.

Image for post
Image for post
Photo by Ian Battaglia on Unsplash

A deep analysis of AWS Lambda Cold Start latency for 2020.

Cold Start is much much better, but is still an overhead that needs to be factored in.

I learned that “Cool” Starts are a thing and that different can have dramatically different performance profiles.

A discussion on AWS pricing. Once your start scaling, you have to be pushing some quite serious dollars through AWS to justify spending any real time on reducing costs. Learning to manage costs early is a massive benefit as you scale.

Corey Quinn is brutally insightful (as always) on the lies we tell ourselves (and others). …


I predict that we’re in a post “flexibility” world.

In practice, flexibility boils down to “come in a bit late”, “leave a bit early”, “work from home if the repair person is coming”.

Flexibility is necessary, but it is not sufficient.

Image for post
Image for post
Work

The pandemic has shown that nearly everyone can work from anywhere at least some of the time.

Our practices probably need to change. “Zoom fatigue” is a symptom of lifting and shifting existing inefficient meeting-driven work practices to video conferencing. We’re not having more meetings, we’re just suddenly aware of how many meetings we have.

Regardless of our physical distance, we have an incredible moment to change our work practices for the better. We can encourage ways of working that are reflective, considered and asynchronous. Collaboration around shared artifacts that enhance shared understanding. Communication that embraces the asynchronous — written and recorded explanations and summaries that have a lifetime outside of the moment. …


Image for post
Image for post
Software Development
  • Ship small increments to production all the time
    Branches (if you even have them) should be small and short-lived
  • Aggressively remove obstacles that slow down releasing to production
  • Have tests and run them continuously, including in production (you can call this monitoring if it makes you feel better)
  • Lint and automate code-level checks
    Agree or (agree to disagree) on a code style
  • Have someone else review the code
  • Make experimentation easy and run lots of experiments
  • Hire people who can communicate as well as code
  • Programming languages don’t matter as much as you think
    Settle on a language for each of your domains, stop arguing and ship some code
  • If you can’t write a well-architected monolith, you also can’t write a well-architected micro-service application
  • Don’t do big re-writes

Adopted from this post on Hacker News https://news.ycombinator.com/item?id=15777442


Randomly infrequent tales of interest from the cloud mines.

Design Patterns for Microservices

A collection of common patterns for microservice architectures.

Predictions for the direction of serverless platforms

Some people are apparently going to run their own serverless systems and Open Whisk on Kubernetes is the main player. I think this is a ridiculous approach, but it’s an interesting counter point to my biases.

Also, Open Source really is the only play against AWS.

The Great Serverless Showdown

A great overview of the major players and the capabilities of the various platoforms in the FaaS space.

Run headless chrome on Lambda

Useful for screenshots or replacing your CI infrastructure with a series of Lambda functions.

AWS X-Ray and Lambda : the good, the bad and the ugly

Detailed analysis of using X-Ray with Lambda.

The future of querying JSON in postgresql

I ❤ you Postgres.

Syncing Redshift & PostgreSQL in real-time with Kafka Connect

An example of using Kafka Connect to consume writes to Postgres and automatically send them to Redshift.

Introducing Dexter, the Automatic Indexer for Postgres

Automatically examine slow queries and build indexes.
Also, you may be amazed to read about Hypothetical Indexes for Postgres that make all of this possible.


Tales of interest from the cloud mines.

Feature Branching is Evil

I like feature branches, but this deck is spot on. Long-lived branches are going to hurt you.

  • Branches should be short-lived
  • Feature toggle to always be rolling forward
  • Deploy off main/trunk/master

Disruptive Machine Learning

The machines are coming.

Enough with the microservices

Because of course:

If you can’t build a well-structured monolith, what makes you think microservices is the answer?

The biggest change I see in tech is that the barrier for entry to microservices has been lowered immensely:

1. Rapid provisioning
2. Basic monitoring
3. Rapid deployment

Any one the FaaS platforms will provide all of the above (with the caveat that you might find the basic monitoring pretty basic indeed). …


Periodic tales of interest from the bit mines.

Don’t Read Your Logs

The next time you start to write a log line, ask yourself whether another observability tool would be a better fit.

Not everything should be a log, performance and error information can be pushed to specific tools.

Also, debugging Lambda via console.log is totally a thing, and it’s hard.

Scaling the serverless summit requires environmental sympathy with dev & ops

Speaking of developing and operating Lambda apps, Matt Weagle, author of the most excellent http://gosparta.io has a wonderful analysis of some of the hard parts of serverless architectures.

The only way to gain confidence that a feature branch will work in the cloud is to run it in the cloud. …


In no particular order.

This was a tweet thread that became unwieldy.

I could be doing things very wrong, so feedback to improve is very welcome.

Debugging is hard.

I write a lot of tests. My advice for everyone working with serverless platforms is to try and side-step the need to debug in “the cloud” as much as possible. Small functions with concise, testable interfaces. Nothing to see here, it’s all programming.

Even with good tests I still find that I often end up resorting to console.log to understand the flow of events through the system.

Add console.log statements, push, trigger an event, dig around in CloudWatch to find the log. …


Tales of interest from deep in the Bit Mines.

This week I have been mostly playing with Lex in my spare time because I plan to win the AWS Chatbot challenge.

Five Factor Testing

Testing, itself, is complicated, because tests are techno-social constructs that support both the code and the team that works on it.

Why do we even write tests?

  1. Verify the code is working correctly
  2. Prevent future regressions
  3. Document the code’s behavior
  4. Provide design guidance
  5. Support refactoring

AWS Security Primer

A high level overview of the AWS Security surface.

Image for post
Image for post
AWS Security Surface

From my experience, customers spent most of their time securing their network because they know how to do this. I recommend that you shift your resources a little bit to make sure you get the AWS API Security right.


Tales of Interest of the Computing and Clouds

FaaSLang

Functions as a Service as a Language:

FaaSlang is a simple open specification intended to define semantics and implementation details around FaaS (“serverless”) functions, gateways and client interfaces (requests from any language / SDK).

Interesting idea, especially as more open source projects jump into the FAAS space. However, vendor lock-in remains a fairly moot issue because a whole lot of serverless as a model is tied in with platform-level events. Hard to abstract elements like kinesis.

Serverless + Modern Agile (pdf)

How will you transform your organization from a feature factory to a product laboratory?

An analysis of how serverless tech can actually make your process more good. …


Being an account of some things that captured my interest over a recent period of time.

We switched to ECS and you won’t believe what happened next

TL;DR They save a bunch of money through:

  • Better resource utilization
  • Use of spot instances
  • Centralized EC2 cost management

AWS Lambda — 3 pro tips for working with Kinesis streams

Working with Kinesis and Lambda

Yubl’s road to Serverless architecture

Overview of converting a monolith into a AWS Lambda and friends based serverless application.

Lessons learned in lambda

Biggest take-way ‘Optimize for Concurrency’. Lots of small, short-running functions.

Databases have failed the web

Databases haven’t really changed since the 80s

Why don’t modern databases simply provide these features? I don’t know. I can guess, but I wont be charitable. Because its hard. Because modern web apps are too new, and server rendered apps look too much like the old desktop apps to give anyone pause. Because database developers don’t write frontend code. …

About

Toby Hede

technology leader & serverless architect

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store