For Practitioners, by Practitioners: #QConNYC 2017 Recap

Question: What do Netflix, Google, Spotify, Hillary for America, Coinbase, and CrowdStrike all have in common?

Answer: They all presented keynotes or sessions at QConNYC 2017.

What separates QCon from most software engineering conferences is that it’s:

For practitioners, by practitioners

The conference delivers practical knowledge and not theoretical evangelism based on these three principles:

  • Engineers over Evangelists
  • Practitioners over Trainers/Coaches
  • Team Leads over Consultants

Centered around session topics and presenters who are thoroughly screened

InfoQ does not have a formal submission process, they usually directly invite their speakers, screen their sessions, and publish interviews with them prior to the conference.

Focused on the left side of the diffusion of innovations curve

For more information on this curve, please see our previous post:

Software is eating those who are not continuously learning

To maintain relevancy in software delivery, you must employ a continuous learning mindset. Trapped are those who don’t keep pace and end up running in place. The speed at which technology evolves requires software professionals to become canaries in a virtual coal mine.

Canary in a coal mine


Sentinel species serve as an early warning system for humans. Prior to the advent of carbon monoxide detectors, coal miners would employ canaries as a sentinel species. The canaries, who have an increased sensitivity to toxic gases, would enter the coal mine and immediately become sick when exposed to carbon monoxide. This action would alert the coal miners to the dangerous conditions and provide them enough time to equip a protective respirator or escape the mine. The role of innovators and early adopters is no different.

Like the canary, software leaders must act as an early detection system for the benefit of their projects. By attending QCon, I was exposed to practices being applied by the world’s most innovative companies; was I subjected to hazardous materials or was I able to return with valuable insights for my clients?


After digesting the QCon New York experience, I distilled the takeaways into three themes — ephemeral, chaotic, and secure.

The rise of serverless architectures, microservices, containers, and cloud computing, have ushered in an expectation that:

  • The lifespan of servers will be short-lived
  • The number of connected devices/servers/services will continue to grow
  • Securing our infrastructure and applications remains as important as ever

This evolution was at the heart of the following sessions:

  • NETFLIX, known for their Chaos Monkey tool, provided insight into Chaos Engineering. As defined by principles of chaos, “Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.” As our architectures embrace microservices, testing by means of unit, regression, and integration, will need to be augmented by chaos engineering. The main takeaway from chaos engineering is that the strategy doesn’t cause problems, it reveals them. Netflix also presented an excellent session on refactoring organizations. In this talk, Conway’s Law was defined as a description for dysfunction and a call for architecture before organizations was effectively made.
  • Dropbox also embraces chaos engineering as a means to remove single points of failure. To add credence, they ran through their approach which is currently being applied across 3 data centers and involves 10k machines.
  • Coinbase has a 30 day plan which stipulates that no server should live longer than 30 days. Their median fleet age is ~9 days. This means that each and every AWS VM is rebuilt within a month’s time. By rebuilding their entire fleet of servers so often, they are ensuring that their servers are patched frequently to mitigate exposure to published vulnerabilities. Having servers updated and reconfigured via rebuilds is all part of an even larger plan to codify and automate everything. Over the past 12 months, they have launched nearly 150k servers. Security played a major role in the 30 day plan. For Coinbase to remain the most popular way to buy Bitcoin, Ethereum, and Litecoin, they must prioritize security. Coinbase also discussed how they use machine learning for payment fraud detection. The call for an end to SMS-based two-factor authentication was also made. For more details on why you should move away from SMS-based 2FA, check out their medium post.
  • provided benchmark statistics on cold starts and performance as they relate to AWS Lambda (the poster child of serverless compute) and the JVM. As depicted below, increasing the amount of memory allocated to your Lambda function does not guarantee cold-start performance improvements.
AWS Lambda benchmark stats (start time vs memory allocation) from
  • Spotify discussed how ephemeral service design avoids hard coded host names, but requires centralized logging. Their “Ops in Squads” approach has improved the lives of over 900 developers by allowing them to provision their own machines. They have provisioned over 20k servers.
  • Hillary for America went into the chaotic scenario caused by what they named, the Frankenbump. When Senator Al Franken unexpectedly flooded the HFA site with traffic (148k requests per minute), their architecture remained resilient. The campaign’s website consisted of 150 serverless front-ends and 100 immutable backends. HFA also discussed some security tactics employed against script kiddies and DDoS attempts.
  • CrowdStrike focused on the role of bad actors in its keynote. There are numerous bad actors throughout the world and CrowdStrike identified the most prominent. They also provided examples of recent incidents. From the stealing of RSA token seeds to the theft of cloud backups (see image below), the threat from bad actors is escalating with the number of connected devices.
Cloud Provider Compromise example from CrowdStrike

So far we’ve explored ephemerality, chaos, and security as they relate to machines, but the most memorable talk was their relation to humans.

Google’s Matt Sakaguchi kicked off the conference with the most powerful keynote I’ve ever attended in over 15 years. The main messages were simple, yet powerful:

Make differences in people’s lives and live life to the fullest.

From his inspirational and chaotic back story to the data backed studies from Google, this talk went above and beyond software engineering.

While the presentation made it clear that life is more important than work, it also posited that you can make an impact at work by improving the effectiveness of teams. Backed by empirical data, Google identified that the foundation of team effectiveness is psychological safety (see image below).

Google’s take on Maslow’s Hierarchy of Needs for Teams

This wasn’t the only reference to safety and security.

Prior to working at Google, Sakaguchi served as a SWAT team member before severe back issues forced his career as a police officer to be short-lived.

This wasn’t the only reference to ephemerality.

Sakaguchi also revealed that he will not have a full life span as he’s currently battling Stage 4 cancer.

Thus, his message bears repeating, make differences in people’s lives and live life to the fullest.