AWS re:Invent 2017 Recap

From Las Vegas, the annual Amazon conference highlighted security, media services, letting serverless take over and developing for the future with machine learning.

What I like about re:Invent is it has a little something for everyone. Whether you’re an infrastructure engineer, data scientist, application developer, security, networking, testing, DBA, the list goes on… you’d be hard pressed to not find something significant to take away. And with so many disciplines it was interesting navigating through the sessions and keynotes and notice two over arching and connected themes — serverless compute and machine learning.

Andy Jassy and Dr. Werner Vogels keynotes reflected this (replays below). Per usual, Andy left me excited about the sheer array of new services and capability and Werner left me feeling like I just completed a lecture from my favorite college professor and I’m going home compelled to jump back into the lab. A full recap of all the services announced can be found here, but here are the ones that stuck out to me.

Lock it up

It should come as no surprise data security was a hot topic this week. If you’ve been paying attention to the news lately, the #1 priority of technology companies should be securing their networks, protecting the sensitive data we steward for our customers and staying out of the news. At least for that reason.

“There is no excuse not to use encryption any more” — Werner Vogels

Werner spent nearly 10 minutes outlining security best practices and that it must be everyone’s responsibility. Encrypting your data should be table stakes for application and data teams at this point and we should be leveraging automation and DevOps practices (DevSecOps?). Things like monitoring CloudWatch metrics, CloudTrail auditing and intelligently managing access to our resources.

The big dog is watching the yard. 
Make use of advisory services such as Trusted Advisor and the new service Amazon GuardDuty, which continuously monitors your AWS accounts, credentials, resources, operating systems, and applications for anomalies and indicators that something is amiss.

Lights, camera, Elemental

If I didn’t know any better, AWS wants me to create my own video production studio. This year saw the release of many media oriented services — AWS Elemental — ranging from how content is ingested and processed to how it’s targeted and delivered. Couple that with the machine learning services that Translates and Transcribes video and you have a means to produce content and distribute it world wide with really low overhead.

Speaking of Machine Learning…

Here’s what I think the play of the week was: Machine Learning. It seemed like over half of the services discussed ultimately ended at machine learning or enabling machine learning. Calling back to video services for a moment you had Kinesis Video Streams that allows you to securely stream time-encoded data directly from cameras and IoT/Connected devices and deliver it to ML and analytics services.

You also had new physical devices like DeepLens that combine an HD Wireless camera with on-board machine learning models, Lambda and other AI services. Pair that with Amazon Comprehend and Rekognition Video and there’s a lot of power on a single device.

The last piece of the machine learning puzzle comes in the form of what I think is machine learning DevOps, and that’s Amazon SageMaker. Via three modules, data scientists no longer have to worry about standing up large complex environments and now can build, train, and deploy their models all from a fully managed environment.

IoT on the Edge

Andy spent a fair clip on the internet of things covering everything from Device Management to Analytics and security with Device Defender. What was particularly interesting was the introduction of AWS Greengrass ML Inference which allows you to deploy and run machine learning inference locally on the IoT device. This allows for inference to be performed right on the device and only the relevant data is sent back to the cloud for further processing.

We also saw the birth of a new operating system — Amazon FreeRTOS. This open source OS is specifically designed for microcontrollers on low-powered edge devices. It comes with a collection of libraries that will make programming, deploying and collecting data easier.

Some big data news

Starting with Aurora, Amazon’s personal take at relational databases, we saw some pretty significant announcements starting with the feature — Multi-Master. This is the first relational database that provides multiple write nodes that scales both reads and writes across multiple availability zones and soon multiple regions.

Next, Aurora saw it’s Serverless implementation. Giving all the capability of the MySQL-compatible edition of Aurora, with the benefits of a fully managed environment. The database will automatically scale and adjust based on application demand.

If you’re into graph databases, Amazon Neptune is yet another fully managed offering. This allows easier management of applications with highly connected datasets and storing relationships on the order of billions all while querying the graph with extremely low latency.

Finally, to round up data and storage was the announcement that objects in S3 and Glacier are now query-able via two new services S3 Select and Glacier Select. Both services allow you to filter and access data within their respective storage types and only pull back what is needed. Ultimately lowering transmission costs and improving application performance.

Application Development for the Win

Now I might be a little biased here, but I’m most excited for the vast array of application development tools and services rolled out this year. Starting out with the DevOps world, you have a new service AWS Fargate that gives you the convenience and power of deploying with containers, without the headache of managing and scaling clusters of VMs.

However, if you’re still intent on managing your clusters and using Kubernetes, you’re covered here as well. Elastic Container Services (ECS) received a nice upgrade with ECS for Kubernetes or (EKS)

If you’re a fan of Lambda functions you also saw significant improvements with deployment options allowing you to assign weights between versions and gradually roll out updates. This update also saw the use of AWS CodeDeploy to manage rollout of Lambda functions, increased maximum memory thresholds, concurrency and throttling options and an updated Lambda console.

Something else I thought was pretty cool comes way of an application repository. Not so much a source code repo, but an AWS Serverless Application Repository. Here, producers can submit their serverless application components to the community for others to use and deploy. This, I hope, is going to open the door for sharing, collaboration and innovation much the way open sourcing code did.

Not your father’s IDE
I’m probably more excited than I should be for this one but the final service I’d like to point out is actually a brand new IDE. AWS Cloud9 is a cloud-based development environment that let’s you write and debug your code all from just your browser.

In a live demo, Werner and his colleagues were able to chat, collaborate, and even share their development environment to make real time updates to their pizza and beer ordering application.

Werner on the left — Dev Colleague on the right

Of course it all integrates seamlessly with the CodeStar tools for a full featured CI/CD pipeline

AWS re:Invent 2017 Takeaways

So after a great week in Vegas, what are the lasting impressions?

  • Data consumption and analytics is still on top of the pile and it’s fueling the massive growth of machine learning.
  • Serverless still is and will continue to be the way of the future.

Let me stick to that last point. I used the terms serverless and fully managed environment about 10 times in this article. A lot of the innovation in cloud computing isn’t in hardware and maintenance problems, which has already been solved over and over by the way. It’s in the business applications we’re trying to solve, the patterns hidden in the data we’re pulling from devices, and customer usage.

I’ll grant you, we’ll most likely lose those who know how to configure and tweak a system juuuuuust right to squeeze out every byte of potential. But I would argue — do we need that anymore? Heck, we’re heading towards to not even installing and managing IDEs anymore. As I’m thinking about it, for most every case of systems configuration wizardry couldn’t we find a elegant cost effective solution using serverless or darn near serverless implementations?

I don’t think we’re quite there yet, but with each passing year and re:Invent I’m starting to think it’s plausible. Soon we won’t have to worry about all that excess and system management when thinking about our applications, but focus on and reach the bold parting statement from Werner — “All the code you ever write is business logic”

re:Invent Keynotes
re:Invent 2017 — Andy Jassy Keynote
re:Invent 2017 — Werner Vogels Keynote