Create your own system monitoring tool in Linux

Umangshrestha
Jun 8 · 5 min read

Why create your own monitoring tools when there are so many better ones in the market? … Why not?

plotting uptime and cpu using plotly

Maybe you want to do performance testing. Maybe you are using low spec system and want to create your own monitoring tools instead of inbuilt tools such as htop to reduce impact of analysis on data being analyzed. Maybe you just want to learn for the sake of learning. In this article I will help you write the script to monitor system. I will primarily use golang as it is fast, simple to can be compiled natively.

Of course, we will not all will be discussed all the method of monitoring all the parameters in this article. For code reference on what we are discussing here as well as some extra parameters, you can refer to my GitHub page here: https://github.com/umangshrestha/system-monitoring

Before we start we need to learn little basics about pseudo files. We will try to keep the theory to minimum as possible.

Pseudo Files

In Linux, there are concepts called pseudo files. As the name suggests they are not a real files but look as such. They primarily reside in ram and store information about current system set. As such, they don’t pertain the data like other files after system restart. For system monitoring we mostly care only about /proc/ files. Learning the psudo files will take you one step closes to better understanding of linux.

/proc/cpuinfo -> nproc

nproc prints the number of CPU processing units available in the system. In normal programming this is the max number of threads that kernel allocates.

$ nproc
8

We can read the /proc/cpuinfo files to number of cores in the system.

$ cat /proc/cpuinfo |awk '/cpu cores/'|uniq
cpu cores : 8

Now we know the the number of process is 4. Each cores can do two tasks in parallel.

proc = number of cores * 2

Steps:

  • Read the file /proc/cpuinfo till you find the line with “cpu cores” in string.
  • Split the line based on white space.
  • The last element of the array will be number of cores.
  • Multiply it by 2 to get the number of procs.
Reading nproc in golang

/proc/stat -> cpu (%)

In Linux, you can get the cpu using htop , top, etc. I generally prefer the following way.

$ echo $[100-$(vmstat 1 2|tail -1|awk ‘{print $15}’)]

To get the overall CPU we have to read the /proc/stat contains many data. I am filtering it to show only the relevant information.

$ cat /proc/stat |grep cpu
cpu 374372 5761 145972 9411508 137412 0 2559 0 0 0
cpu0 43015 760 18354 1178386 14299 0 1488 0 0 0
cpu1 47073 257 16105 1176409 21069 0 169 0 0 0
cpu2 46042 380 15558 1183634 15595 0 75 0 0 0
cpu3 44937 218 16184 1184143 15076 0 79 0 0 0
cpu4 58434 69 35890 1147711 12275 0 178 0 0 0
cpu5 43949 3015 14509 1172134 26881 0 412 0 0 0
cpu6 46050 388 14534 1182711 17714 0 117 0 0 0
cpu7 44869 673 14835 1186378 14499 0 38 0 0 0

The top one is average cpu stat followed by individual procs in your system. They are in following format:

$ head -n 1 /proc/stat 
cpu 400260 5764 154072 9815486 139611 0 2740 0 0 0
# user -> 400260
# nice -> 5764
# system -> 154072
# idle -> 9815486
# iowait -> 139611
# irq -> 0
# softirq -> 2740
# steal -> 0
# guest -> 0
# guest_nice -> 0

The time used by each process used the given user or system is stored in their own array. Here we just care about the idle time which is total time used by cpu doing nothing. And total time which is the sum of all the array values. From there we get the total percentage of time cpu did nothing. If we want total time when cpu did somethings

We will only be talking about the average cpu. The remaining cpu follow same logic.

Steps:

  • Read the first line of /proc/stat.
  • Discard the first word which is always cpu.
  • Add all of the values in the line to get the total time.
  • Take difference of change in total time and idle time at 1 second interval.
  • Divide the idle time( fourth value) by total time to get fraction of idle time
  • Subtract 1 with the idle time to get percentage of non idle time.
  • Multiple by 100 to get a current cpu percentage.
Calculating cpu in golang

/proc/meminfo -> memory usage (kb)

When you see memory calculation done by free, htop, vmstat they are reading the data from /proc/meminfo. It is a big file. So I am only showing the relevant information from there

$ cat /proc/meminfoMemTotal: 16230988 kB
MemFree: 7264300 kB

Buffers: 342284 kB
Cached: 2892272 kB
SwapCached: 0 kB

SwapTotal: 2097148 kB
SwapFree: 2097148 kB

Shmem: 697432 kB

SReclaimable: 254032 kB

Steps:

  • Read the /proc/meminfo file.
  • Find all the parameters that I have mentioned in given file.
  • Do calculation as:
Mem: total -> MemTotal
used -> MemTotal — MemFree — Buffers — Cached — Slab
free -> MemFree
available -> MemAvailableshared -> Shmembuff/cache -> Buffers + Cached + SlabSwap: total -> SwapTotal
used -> SwapTotal — SwapFree
free -> SwapFree

note: The results are in terms on Kb by default.

Calculating memory in golang

/proc/uptime -> uptime (s)

$cat /proc/uptime
34903.79 224873.38

The 1st element uptime is the duration in seconds that system has been on. The second is t idletime is the duration in seconds when processor was not being used by any program.

Steps:

  • Read the first line of /proc/uptime file.
  • Split the line and read the first element.
  • The data is in seconds.
Reading uptime in golang

Geek Culture

Proud to geek out. Follow to join our +500K monthly readers.

Umangshrestha

Written by

I like computer and programming. I am currently trying to learn how to write articles for sharing what I know.

Geek Culture

A new tech publication by Start it up (https://medium.com/swlh).

Umangshrestha

Written by

I like computer and programming. I am currently trying to learn how to write articles for sharing what I know.

Geek Culture

A new tech publication by Start it up (https://medium.com/swlh).

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store