Linux Foundation Certified Systems Administrator (LFCS): Learnings and Preparation Guide

W. Jenks Gibbons

Initial LFCS Attempt

If you have ever taken a Linux Foundation certification test (e.g. the LFCS, the CKA, the CKAD, or the CKS) you know they are not easy tests. They are practical exams with about 17 to 22 non-multiple-choice questions on various virtual machines or Kubernetes clusters.

I have taken, and passed, each of the four exams above. The most recent one was the LFCS. The difference with the LFCS was that I found the preparation material difficult to find, low-quality or non-representative of the skill-set required. How did I overcome this and successfully pass the exam and what did I learn?

Preparation

For the CKA, CKAD and CKS I took the following courses to prepare:

• Certified Kubernetes Administrator (CKA) with Practice Tests
• Kubernetes Certified Application Developer (CKAD) with Tests
• Certified Kubernetes Security Specialist (CKS)

All three of these were offered by Mumshad Mannambeth either in whole or in conjunction with another trainer. All three were excellent, were representative of the exam material and provided me the foundation to pass the exams.

When I decided to study for the LFCS it was a no-brainer to use another Mumshad Mannambeth course: Linux Foundation Certified Systems Administrator — LFCS. I may recommend the course but not as a complete preparation for the exam.

I had never heard of the LFCS when a colleague mentioned it. I thought it sounded fun, saw there was a preparation course offered through Udemy (I have a subscription through work, but courses are available on a one-off basis as well) and started to study.

I have over 20 years of experience on the Linux command line doing everything from building kernels, to file searching, text manipulation to git. I was in no way starting from scratch, but I am also not a system administrator. So, I took the course… I enjoyed the course… I learned from the course… I took the practice exams… I was able to ace them… I took the LFCS… and………… what was that? The level of difficulty was incomparable to the course practice tests and as you can see above, I did not pass it.

So, how to prepare? I looked and looked for material. I did not find much.

1. Linux Foundation study guide: while any candidate should be able to answer these questions, they are fundamental, are not representative of the exam and don’t have answers. They are also from 2019 (https://training.linuxfoundation.org › 2019/04), does the preparation/exam not change over years?
2. VMEXAM: I was desperate by the time I found this. I do not recommend spending money on this. The quality is about the same as that of the web site. While there are some good questions, most are not representative and only about half have answers. This seems to be reoccurring, why would I want answers to study questions? Hmmmmm….

After studying with VMEXAM, I failed the test again with a 64. To say the least I was disappointed and to be honest was just straight embarrassed. Now, I was not sure how to prepare: enter killer.sh If you are not familiar with killer.sh it offers practice tests for Linux Foundation certification exams. I don’t recommend them, but don’t not recommend them… I rather usually say to make sure you know they are harder to much harder than the exam. The night before the CKA I scored ~20% on the killer.sh test and then passed the CKA, but the experience left me in a negative mindset. By this point I was wishing there was a killer.sh exam for the LFCS as I barely missed passing it the second time.

When I was looking at registering for the exam again I saw this:

Access to a killer.sh Practice Exam

I may have seen that when I registered the first time, but do recall seeing a link to the “Exam Simulator” in my checklist. I knew what the exam environment was and what my experience with killer.sh was in the past so I didn’t give it any more thought. My views have changed.

The killer.sh tests are much harder than the certification exams in the same way that the LFCS preparation course exams were much easier. Harder is better when practicing however. Once I had access to the killer.sh test I studied what gave me issues in the exam, used my two sessions of killer.sh and passed the exam with an 85 a week later.

So, to wrap this part up. The LFCS course I would say is foundational. You need to be able to pass the practice tests or rather know the material in them. If you are not well on your way to having the skills to pass the LFCS then I recommend the course. Otherwise, use the killer.sh tests as a preparation guide and if/when you can pass those you can likely score highly on the exam.

Exam Tips

I will cover the basics first as they are foundational and include techniques I have used in all the Linux Foundation exams. At the beginning of the exam, review all the questions and mark them in a file with their number and percentage value. Begin with the questions where you feel confident from the highest to lowest value followed by any you are unsure of or need to return to. It is difficult, but if you get stuck on a question move on and return later… time may be your most valuable resource and it is easy to run out with unanswered/unattempted questions.

Those are foundational and these should be as well, but I had bring focus to them on this exam.

1. confirm you are on the right machine
2. make sure you carefully read the instructions. I found that it is easy to miss that the first machine you ssh to may just be a jump host. I also found that some very common commands I use were presented differently than how my mind frames them (e.g. arguments backwards for example such as map container port to host port).
3. slow down, read the documentation and keep from getting flustered… breath and do your best to relax
4. install tldr (apt install -y tldr) : simplified man page information with examples (e.g. tldr docker run)

Learnings

Did I learn new information about technology, yes I did. I learned new information about Linux, firewalls, find, proxies, LVM and more. Was technology the main thing I learned? It was not. Almost all of the material I had used to some extent at one time or another and I can look at documentation when I am doing something old or new, I don’t have to know it by memory.

When I reflect on what I learned I realized that I strengthened some of my biggest weaknesses. I tend to think these weaknesses are often areas for potential improvement for many technical people.

1. I learned to focus and slow down more…. Often when I am presented with documentation or a man page I become overwhelmed and impatient. I decide I will figure it out, have a hard time getting things working, go back to the documentation and resolve things quickly.
2. I learned how to search for documentation and hints towards needed information/solutions in additional ways:

• apropos search for man page information and descriptions. For example, if I don’t know how to en(dis)able an Apache module. This gives me a nice lead as I don’t see anything at a glance in the man page for apache.

# apropos "apache2 module"
a2dismod (8)         - enable or disable an apache2 module
a2enmod (8)          - enable or disable an apache2 module

• tldr as mentioned above

$ tldr docker run
docker run
Run a command in a new Docker container.More information: https://docs.docker.com/engine/reference/commandline/run/.

 - Run command in a new container from a tagged image:
   docker run {{image:tag}} {{command}}

 - Run command in a new container in background and display its ID:
   docker run --detach {{image}} {{command}}

 - Run command in a one-off container in interactive mode and pseudo-TTY:
   docker run --rm --interactive --tty {{image}} {{command}}

 - Run command in a new container with passed environment variables:
   docker run --env '{{variable}}={{value}}' --env {{variable}} {{image}} {{command}}

 - Run command in a new container with bind mounted volumes:
   docker run --volume {{/path/to/host_path}}:{{/path/to/container_path}} {{image}} {{command}}

 - Run command in a new container with published ports:
   docker run --publish {{host_port}}:{{container_port}} {{image}} {{command}}

 - Run command in a new container overwriting the entrypoint of the image:
   docker run --entrypoint {{command}} {{image}}

 - Run command in a new container connecting it to a network:
   docker run --network {{network}} {{image}}

• apt search search for packages that can lead me to what I need

$ apt search apache | grep -i apache | grep puppet

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

puppet-module-puppetlabs-apache/jammy 5.5.0-2 all

• find this is a tool I use often. I learned to use flags that I don’t usually use that are helpful such as -size, -perm, -newermt, -user

$ ls -lt $(find ! -newermt 20230901)
-rw-r--r-- 1 root root 3106 Oct 15  2021 ./.bashrc
-rw-r--r-- 1 root root  161 Jul  9  2019 ./.profile

or how to find another time daemon if it is not ntpd

find / -name *syst*times* -type f 2>/dev/null
/snap/core20/1879/usr/lib/systemd/ntp-units.d/80-systemd-timesync.list
/snap/core20/1879/usr/lib/systemd/system/systemd-timesyncd.service
/snap/core20/1879/usr/lib/systemd/systemd-timesyncd
/snap/core20/1879/var/lib/systemd/deb-systemd-helper-enabled/sysinit.target.wants/systemd-timesyncd.service
/snap/core20/1879/var/lib/systemd/deb-systemd-helper-enabled/systemd-timesyncd.service.dsh-also
/snap/core18/2745/lib/systemd/system/systemd-timesyncd.service
/snap/core18/2745/lib/systemd/systemd-timesyncd
/var/lib/systemd/deb-systemd-helper-masked/systemd-timesyncd.service
/var/lib/systemd/deb-systemd-helper-enabled/sysinit.target.wants/systemd-timesyncd.service
/var/lib/systemd/deb-systemd-helper-enabled/systemd-timesyncd.service.dsh-also
/var/lib/dpkg/info/systemd-timesyncd.postrm
/var/lib/dpkg/info/systemd-timesyncd.list

• man so many references to needed information, not just flags etc.

For example, searching for .conf or using SEE ALSO or EXAMPLES

3. options

There are often many ways to do something. Let’s take the example of forwarding incoming traffic from one port to another.

We could use nginx, squid, iptables, firewalld, or ufw. From what I saw on the killer.sh practice test and on the exam, for these types of questions you can often use any technology that accomplishes the task. NOTE, this is my understanding but if someone has seen differently please comment. For example, on the practice test the solution used iptables, but using firewalld and ufw I was able to successfully answer the question. For example, netfilter is used in various technologies such as iptables and firewalld.

4. Set a time limit for if you get stuck

This is a strong weakness of mine. I get caught up in a task and I can’t let go until I get unstuck. The opportunity cost for this is/can be very high here.

For example, there were two questions that I did not answer on the exam I passed.

a) port filtering (-7%): I knew how to do this, but got caught up on the language and thought I had to use a particular technology. I started using what I knew, got flustered and moved on. I did not have time to go back and finish it because of what I did with the lv task below.

b) logical volume task (-6%): I answered this correctly twice before, however for some reason could not get it to work. Because I had answered it before I was unable to let go and return to the port filtering question. Therefore, I did not answer either.

Now, because I was well prepared and knew the material I still scored 85% when I did not answer 13%. I was disappointed but ultimately happy with my score.

To conclude, the LFCS content is fun and a solid way to show your system administration skills whether that is your role or not. I hope the tips, exam practices and learnings were helpful.

Have fun :)