AWS re:Invent 2018 — Day 3— I think I got it :-)

Day 3 was a blast

• Made new friends and re-connected with old friends
• I managed to squeeze in two sessions that are very relevant to Solace’s mission “Choosing the Right Messaging Service for Your Distributed App” and “Building Massively Parallel Event-Driven Architectures”
• Talking to so many booth visitors about what they’re building, pain points and opportunities in the Cloud, I can now see how Solace’s Event Mesh vision will help them focus on their core business
• Customer references make “HUGE” difference, very effective to have the customer logo to point at
• Mentioning Hardware Appliances will make the conversation interesting and is an awesome icebreaker
• Oh yeah, I also learned that I have awesome and cool colleagues

API305: Choosing the Right Messaging Service for Your Distributed App” (slideshare)

“In the cloud, modern apps are decoupled into independent building blocks, called microservices, which are easier to develop, deploy, and maintain. Messaging is a central tool used to connect and coordinate these microservices. AWS offers multiple messaging services, which address a variety of use cases. In this session, learn how to choose the service that’s best for your use case as we present the key technical features of each. We pay special attention to integrating messaging services with serverless technology. We cover Amazon Kinesis, Amazon SQS, and Amazon SNS in detail with discussion of other services as appropriate.”

• Overall, the Software Dev Engineer gave a clear, crisp, to the point description of what SNS, SQS and Kinesis can do and cannot do, I enjoyed that part. Especially that he led with this slide:

• However the Solution Architect (SA) was not as effective and at some moments it sounded like he did not believe what he was saying, especially when showing the use case architecture (real customer solutions). Everybody has to start somewhere and I can understand he was nervous.
• They did not cover Amazon MQ (or ActiveMQ), and only introduced it as a transition, a solution for enterprise applications using JMS and enterprise messaging exchange patterns, moving to the cloud. So a temporary solution, maybe ? Not sure
• In all of the use cases, not surprising, the message flow was in one direction, no feedback, only upstream to downstream.
• Did not cover IOT messaging requirements or microservice consideration
• Oh yeah, heavy use of lambdas in the solution and that’s a good segue for the next session

SRV373: Building Massively Parallel Event-Driven Architectures (slideshae not yet available)

“Data and events are the lifeblood of any modern application. By using stateless, loosely coupled microservices communicating through events, developers can build massively scalable systems that can process trillions of requests in seconds. In this talk, we cover design patterns for using Amazon SQS, Amazon SNS, AWS Step Functions, AWS Lambda, and Amazon S3 to build data processing and real-time notification systems with unbounded scale and serverless cost characteristics. We also explore how these approaches apply to practical use cases, such as training machine learning models, media processing, and data cleansing.”

• It was a great introduction to AWS Lambda by Amit Kulkarni. He broke it down to its building blocks, covering the functional and non-functional requirements for using it.
• I liked how he simplified the core concepts (event generation, event routing and event processing)
• And took his time explaining the key considerations at every stage, for example in event processing you have to understand and fine tune your concurrency, velocity, duration, error-handling and transport vs transform (transform = memory and cpu-bound, transport = network bound)
• I was not surprised when he said, a lot of developers get it wrong or did not know about IO bound functions, by using Lambda for transport-bound operations (basically most of the time, the function is waiting on IO)
• You win some, you loose some. Things can get really wrong if you don’t fine-tune your lambdas and adjust the limits, and sweat the concurrency model you desire
• The customer use case presented in the session was real, but the presenter lost me half way and started asking for feature in AWS, right in the middle of the session, basically he said, if CloudWatch can format the logs a certain way, or give the customer a way to do so, I don’t need this elaborate lambda architecture :-)
• If I can take one thing from the customer use case is: developing lambdas is very easy, running lambda at scale is harder than you think, and you need to re-iterate and do performance analysis on your functions regularly.

That’s all for now, see you on Day 4