AWS Security Hero ~ How and Why
After being an AWS hero since 2016 and speaking exclusively about security, AWS has created a new hero category and I’m pretty excited about that! :)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
⚙️ Check out my series on Automating Cybersecurity Metrics | Code.
🔒 Related Stories: AWS Security | Application Security | Secure Code
💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As you might know (or might not) if you followed me for a while, I’m an AWS Hero. Most people that work at AWS don’t even seem to know what that is so if you’ve never heard of it, I understand. But people who have used AWS for a long time and go to AWS re:Invent have probably seen or heard of AWS heroes. AWS recognizes people who write or speak about AWS, contribute code and tools that others can use, and run AWS meetups to help people learn about AWS. I’ve been fortunate to meet some amazing AWS Heroes from all over the world and now call friends.
Quick recap on my initial AWS Hero title
If you’ve been following along, I’ve written about this before so I’ll keep it short. But you can skip to the next section if you already know this.
Technically I was an “AWS Community Hero” since 2016 but that title seems so long so I generally just said I was an AWS Hero. I ran a meetup for years in Seattle called the Seattle AWS Architects and Engineers and that group grew to over 3,000. I’ve written about that before in prior posts. I’ve since changed that meetup to an AWS Security Meetup and not specific to Seattle, but having a hard time finding the time to run it. It is primarily virtual when we do host an event now.
I also blogged a lot about AWS since 2016 because I was interested and using it. I wrote a couple of white papers and gave presentations for SANS Institute and AWS on cloud security and presented at SANS Networking when pretty much no one was talking about cloud security. SANS later gave me recognition as an innovator in security via their cybersecurity Difference Makers award in 2017.
At some point, along came some emails telling me I was selected as an AWS Hero which I thought was just spam or phishing at first. After all, I didn’t know what an AWS Hero was and I was really busy on some grad school papers or something. I don’t remember.
Then Jeff Bar, Chief Evangelist for the Amazon Web Services, reached out on Twitter and said AWS Heroes is trying to reach you and I was like, “Wait what?”
I’ve told this store before but the point of all that is that some people go out vying for recognition but I prefer to get recognized for just doing what I do. I never lobbied to be an AWS Hero. When we had a heroes event at re:Invent one year and people could come meet us, the main question people asked me was, “How can I be an AWS Hero?” Well, I’d rather help you with cybersecurity, but the bottom line is — I don’t know. I wrote what I’ve done and what I know in this post and I have one other new tip below that I recently learned.
AWS Security Hero — Woot!
Well, now — AWS made me an AWS Security Hero with a few others in this first ever new category of AWS Heroes. Pretty jazzed about that! I’ve been speaking about AWS security since my first presentation at my own meetup, at an AWS Community Day event, at re:Invent, re:Inforce, SANS Networking, Serverless Days, BSides, RSA, and other conferences around the world. (Full disclosure I’ve also spoken about Azure and GCP security, but not as much.) Security is my thing. My passion, even!
I can’t wait to get together with the other AWS Security Heroes next week in Seattle if they are there at the AWS Heroes event I’m headed out to or at AWS re:Invent if I can make it this year. I’ve also connected with them on social media and already knew Chris Farris (an excellent choice). It will be great to add some great new content to my social media feed if I wasn’t following them already.
How not to become an AWS Hero
One thing I did learn in this most recent round of AWS Heroes announcements, is that your relationship with product teams matters. It’s one thing to recommend improvements and changes. It’s another thing to be too snarky and perhaps mean or judgmental. Everyone at AWS is trying to do their job and you might not understand just how hard that job is. Some things that may seem simple on the outside are monumentally challenging on the inside where the complex machinery that makes AWS work spins away every day.
Werner Vogels, CTO of Amazon, recently wrote an excellent post on the complexity of S3. It was not only about the technical aspects of the cloud but also the people and processes required to keep all the services involved doing what they are supposed to do. I’ve been following Werner Vogels on twitter (@Werner) for years due to excellent technical posts like this. I also personally like his keynote the best at AWS re:Invent and never miss it.
Recommendation: Keep the snark to a minimum and try to be part of the solution rather than just point out problems.
No rubber stamps, but instead of snark — a wish list!
One thing I have not done is basically rubber-stamped everything AWS does. I definitely call out problems when I see them, but (most of the time) in hopefully a kind way with recommendations to solve the problem.
I will admit that sometimes when I have spent way too long trying to do something that should be simple, I get a bit frustrated. Don’t we all. But in the end, I’ve had product managers reach out to me. AWS listens and they want to solve the problems.
Sometimes I have to explain that I’m so overloaded with work or trying to earn a living that I don’t have time to have meetings and discussions when I have to get paid work done — but I do try to respond and I write blog posts to explain the problem and my take on how to fix it. Amazon may decide it’s a problem or not — that’s up to them. But at least I am trying to solve the problem not just criticizing everything.
There are things I’ve been requesting for years — like an AWS Bug Bounty. It took many requests for AWS to give up pre-approval for penetration testing (I wrote about that in the above post). I’m still hopeful the AWS bug bounty is on the way. I don’t know any more than anyone else outside of AWS about such things, but I keep hoping. :-)
You can add your own requests to the AWS wish list — and this is one thing I absolutely love about AWS. You put your requests on the wishlist and see if your wish is granted!
Why I like AWS — because I’m a hero? Nope. Other way around.
I’ve worked with all three clouds — AWS, GCP, and Azure. I still like AWS the best. No, it’s not perfect. Yes there are things I like from other cloud providers. There is always room for improvement.
But here’s why I like AWS:
- I like CloudFormation. I’ve been showing you how to use it in this series and I also wrote about things I would like to see fixed or changed with CloudFormation.
- I like AWS Networking. There’s no way to do ARP cache poisoning.
- It’s the only cloud with a stateless firewall option (AWS NACLS). The networking is closer to actual networking than some other clouds where networking appears to be handled at the application layer. I explain why that matters in my security classes. You can apply separate networking rules to subnets and security groups with different constructs, unlike cloud where the rules are always applied at the VM interface regardless of where you apply them. I feel like GCP is all at the application layer but I haven’t dug into that enough to say for sure. GPC networking is just different.
- There are no magic outbound rules on AWS private VMs that are hard and/or cost money to dismantle (like Azure).
- I don’t see any documentation saying you may or may not get the network traffic in your VPC flow logs, something I have seen in other cloud provider documentation (GCP, unless it changed since). I haven’t actually tested AWS to see if every packet gets tracked but I’ve never had a case where something I was looking at wasn’t logged (yet).
- I don’t like how many firewall rules you have to create to use some cloud services — like Azure DevOps and Azure Update Management Center. I’ve written about the risk associated with software updates and build systems many times over in this blog. Although I thought Azure DevOps policies were pretty cool, but then I saw the network requirements I had to nix the idea of using it. I’ve also been pondering network security issues with GitHub but at least you can run your own server. AWS CodeCommit can run entirely in your own network and a connection to the service that stays within the AWS network, but I’ve written before about why I’m still using both GitHub and AWS CodeCommit. For now.
- I like AWS VPC Endpoints — something that was on the feature request list from Capital One I used to manage and became a thing. Originally it was S3 endpoints. Then it evolved into endpoints to access any AWS service and keep the traffic off the Internet. Azure and GCP have similar services now.
- I like the way AWS IAM works. Yes it’s complex, but it’s also very powerful. There are ways to simplify it which I started demonstrating in the blog on automation above and I’m not done. Follow for more updates as I work to streamline and simplify my IAM implementation.
- I struggle to embrace certain aspects off AWS SSO, but I reckon some changes are coming as AWS works to merge SSO and IAM. I wrote about some potential changes that would put the power of trust policies back in the hands of customers. I noticed some pretty cool updates to the AWS IAM console in regards to trust policies recently, though I mostly do everything via code. I don’t really want to give up my SSO to Azure AD which is constantly in the news with some security problem, and in general I don’t like the way it works or how much it costs for the advanced security features. Additionally, I would prefer to have a neutral identity provider and require two-party access to my cloud account, which is why I am exploring Okta. I haven’t completed the exploration of that concept yet. I thought about using Google Workspace as an IdP but that also does not meet the 2-party requirement. We’ll see how Okta works out. One of the things I don’t like about Okta is no Private Link access to AWS. I’m still writing about it and pondering the implications. And things change over time, like my mind. :-)
- I really like being able to enforce MFA on role assumption when working on penetration tests and assessments for third parties. I like being able to enter a code with no browser interactions. Over the years Azure MFA has been very challenging, though it’s getting better. But they force you to do this weird browser thing for MFA with the CLI. Unfortunately AWS SSO adopted something like that. It’s not my preference. But you can still use AWS IAM instead so that’s cool. I’ve seen vulnerability after vulnerability with Azure AD, passwordless, and IdP related technologies which makes me wary of using it. I’m sure part of that is because — that’s what everyone uses. It’s a big target. The other problem I have with Azure is that if you want some of the security features you need an E5 license and maybe more which gets pricey. some things used to require an Enterprise account. Not sure if that is still true. One of the reasons everyone still uses it is because Azure won’t let you federate to a third party like Okta the way AWS and GCP will. Yes you can SYNC, but not FEDERATE. There’s a difference.
- I find setting up roles for a penetration test or security assessment very easy on AWS and I can enforce MFA when connecting to a customer account. I’ve had challenges with Azure IAM, especially when trying to set up roles that can see the entire organization and all tenants, no exception, for security auditing. Personally I thought GCP was easier to set up than Azure, but I did put a request into the GCP team for a default Security Auditor role. The first time I did a security assessment I had to create my own role to see everything. I wrote a custom script that looped through all the read-only policies — and I found very interesting things in read only policies by the way — and then I had to do various things to make my policy small enough to use with my role. Unless this has changed recently, I was not able to enforce MFA on my role from a third-party account when performing a GCP assessment.
- AWS Nitro. That’s it. I like AWS Nitro. A lot. It provides a very cool way to segregate customer compute resources, data, and processes on AWS hardware.
- Availability of resources — Azure ran out of VMs repeatedly while I was trying to prepare for a class. I simply could not start a VM. I figured out that if I used the East Australia region, that was the only way I could start one through trial and error. It made that preparation quite challenging! I’ve had very rare instances of that on AWS, typically resolved quickly. However, you should always have backups and images when working on a project in case something happens to your VM. One time, while teaching a class in Australia, I also had availability issues with GCP. That was about two years ago now, however. I also remember back when I was first teaching SANS cloud security classes there was an issue with AWS in Germany but that has long since been resolved. It was related to a new fraud protection feature. AWS has had a long head start on the other cloud providers which is why the platform may be a bit more robust in a lot of areas.
- I’m not a fan of the Azure licensing and cost model. I recently saw people complaining because they got hit with fines for being out of compliance on Azure licenses after someone at Microsoft told them they were OK. If you’d like to see those posts they are somewhere back in my Twitter (X) feed. I’m not going to go dig them up but I can if needed. I had issues in the past in classes where I taught all three clouds. People with paid Azure subscriptions were fine but people using free accounts were angry when things demonstrated in class didn’t work. This confusion rarely exists on AWS. There are only a few services where you can’t use the pay as you go model and test things out. There’s none of the complicated licensing you have to deal with on Azure and the subsequent surprise fees for being out of compliance with said licenses. I had some billing issues on GCP but they were quickly resolved. I can’t say that I’ve used GCP as heavily to fairly judge — but I can say that the billing account thing is confusing on GCP. Trying to merge a bunch of services or getting blocked from setting up a GCP account because your marketing team has a Youtube account set up somewhere can be confusing. Not sure if that has improved lately.
- I wrote about cloud outages like the Azure 5-hour BGP outage, for example. Microsoft Azure has the most outages as far as I can tell. It seems like there’s not enough governance when things get changed but I don’t know exactly what the problem is that is causing the Microsoft issues. And to be fair, I addressed outages from other cloud providers in this post as well. What I wonder, is if anyone is tracking the total downtime incurred by each provider and judging it fairly? I just know what I’ve read and when I’ve been blocked and Azure has blocked me most often over the years even though I use AWS more. I’ve had minor issues with Google Cloud services (calendar, mail) but I know others on the East coast (when I was on the West Coast) were affected with some calendar and mail outages a while back. If I had been on the East Coast, that would have hurt.
- When I use Microsoft Azure, I honestly like the UI. It’s very well laid out and organized and consistent. I like the risky logins feature. I like some of the governance and SEIM features. But behind the scenes I feel like it’s a very brittle house of cards based on the issues I’ve had. One time, I was getting blocked from doing something based on a network rule I created on a resource. The only problem was, that IP address was not mine getting blocked. It was a Microsoft IP address. I was later told it was a security issue but they were not allowed to talk about it. It took me weeks to convince them they had a bug in a new IdP solution in preview. I was getting an error but there was no error message anywhere. To be fair, it was a service in preview but the time it took to get someone to acknowledge the problem seemed a bit egregious. I just feel like AWS has a more robust architecture behind the scenes for most of the services — without even seeing it. This is based on years of reverse-engineering systems to understand how they work and make improvements or these days, report security problems.
- Although I’ve documented a lot of error messages on AWS recently, those messages were intentional, though perhaps a bit obscure in some cases, and I wrote them down along with how I solved the underlying problem. I would probably be writing a lot more about Azure errors based on the last six weeks I spent on it updating my Azure class — but I’ve been using AWS a lot more recently. Hence, more documentation of AWS error messages on my Bugs That Bite blog. I document error messages most of the time to explain how to fix them, not because there’s something wrong with the error message itself. But in Azure’s case, I did have some problems with the actual error. I had to prompt Azure support for what I was guessing the problem was and get them to agree with me because they generally didn’t know and had to go research it. In all but one case, they came back and admitted the message was wrong or it was a bug.
- On that note, I am not the typical user so I find support challenging from almost any vendor. By the time I’m contacting support, it’s often a bug or a documentation problem or something that you can only do by contacting support. I can reverse-engineer most things faster than I can get an answer from support. But when I’ve had to contact support, I’ve had the most challenges with Azure. AWS and GCP have been pretty responsive lately. I know GCP had some issues in the past but I think they are getting better, at least for the things I’ve asked them. But I’m not a heavy user of GCP at the moment. I’ve performed security assessments on Azure and GCP, but for all the reasons above I use AWS to build most of my pentesting tools and host applications. For the record, I’m pretty sure no one on the AWS support team knows what an AWS Hero is or cares if they do, but that’s ok. I don’t get special treatment. But I also know support is an incredibly hard job and I’ve written about that. I’ve stood up for AWS support teams when I felt like they were stalling for time (which unfortunately wasted my time) because I was guessing they were doing what they were doing to meet some kind of unrealistic quota. I’ve also been fortunate to have received support from some highly technical people at AWS, whereas I’ve not had that experience almost ever with Azure. Maybe enterprise accounts get better treatment. I can’t say that I’ve contacted GCP support enough to judge fairly but recently had good experiences when I had a billing issue. I also did not have a lot of reasons to contact GCP support because I could figure out how to do what I was trying to do based on the documentation and error messages for security assessments. I wasn’t doing quite as much as I’ve been doing in my recent AWS blogging so perhaps if I did that I would hit more issues. GCP denied my request for a security auditor role saying that request already existed. Not sure why they didn’t send me the link to the other request in the meantime or add my name to it.
- I think all three cloud providers are getting better and better with governance — and that’s one of the most critical aspects of cloud security so I’m very happy to see that. I like Azure initiatives. I love Just In Time access. Azure was the first to come out with it even though I wrote a blog post about JIT access for AWS prior to that. GCP has an organizations feature that’s pretty good, though I find some of the precedence in policies a bit error prone and possibly risky to keep in check. As for AWS, I like the way you can control AWS Service Control Policies to meet your needs and store them in source control, where you can monitor for drift. I like how you can architect an AWS Organization. I hope the quotas on SCPs will be lifted soon so organizations can write more succinct SCPs and in general, more of them per OU and account, but beyond that I think it’s a very powerful and underused feature. AWS has come up with a number of sample Service Control Policies which should be very helpful. I’ve been writing about that and plan to write a lot more about SCPs in the future!
- This is not a complete list. There are other things I like about AWS including Secrets Manager, KMS, and a bunch of other other security features. But those are some of the things I think sets AWS apart as a cloud provider from a security perspective.
So anyway, I got to be an AWS Hero not because I lobbied to be one nor did I try to figure out how to be one. I just truly like AWS. That’s it. I use it. I like it. I write about it and try to help other people solve security problems on AWS.
Not saying it’s prefect. There’s always room for improvement. But those are some of my takes on cloud security at a very high level and why I primarily work on AWS at the moment.
Follow for updates.
Teri Radichel | © 2nd Sight Lab 2023
About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight LabNeed Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for PresentationFollow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab
