A journey for a better log common ecosystem for Go (Part 1)
When we are developing an application or library, we rapidly need to log somewhere what happens inside it, or who does what. It is what we call application logs and access logs. We can even do this with a certain level of granularity depending on the section of the program, whether we have to restart the application or not.
For this, whatever the languages, there are logging libraries ensuring the management of logs according to a given level. For the Go programming language, we have the basic package (log) which allows to output logs to an output (io.Writer).
Besides this, other logging libraries have been developed to meet more advanced needs (such as the level of logging, the destination of the logs, their formatting, …). Here are a few:
and all others…
Instead of making a choice, it would be great to just switch between them, as needed and without rewriting everything but for that we need to have a library defining a common interface, and maybe some methods or values.
This library should meet the different needs:
- application developer: switch from one library to another without rewriting all the code, common leveling values and type, concurrency safety, easy logger access (context access, logger contextualization), simple to use, …
- library developer: use only a common interface, and not have to choose one implementation, …
- logging library developer: have maximum freedom and as little limitation as possible for library implementations (custom log levels, level matcher, synchronous or not, …), possibility to do their optimization, have a base code to not rewrite/think about (level type and basic values), …
But this library does not exist … yet. To make it a reality, so that it can be integrated as a standard Go log2 library and by the different logging libraries, we need to define a common leveling, a logger interface and maybe a loggers accessor…
OK, let’s Go !
The first step is to define levels. Logging libraries have been around for years and we are used to the classics FATAL, ERROR, WARN, INFO, DEBUG, TRACE. So it may be interesting to be able to define our own levels. But which type to choose ? 2 types are possible, integer and string.
integer
string
integer type could be a good choice. Simple type, rapidly copyable when passing method arguments, low memory cost…
But as they say let’s beware of appearances, so let’s look at it in detail … Fight!
Round 1: Using a level to log
Use a level when calling logger is good in the two cases
because it is readable for the developer. We have a little advantage for string, as we can use string literal name 😉
You will have as result a beautiful message… and a happy developer 😁
2020/09/19 21:32:32 WARN [300] my msgOK, ex aequo but still a slight advantage to type string.
Round 2: (Re)Defining level value
The developers of the logging library may want to give the possibility to define new levels, or redefine an existing one, with a method like SetLevel for example. In this case, an application developer using it will do something that might look like this
That means that in integer case you need to change the definition of the WARN value 300 to 3 which is … const. It will be a little tricky 😅.
As we can not use const anymore and with the concurrency context (goroutine), the best approach to allow a developper to logging library to redefine classic value will be getter/setter methods with atomic access value. The definition of custom levels can then be done by the logging library with an atomic.Map for example. Moreover we are compelled to define the type of integer to use for all.
Giving the possibility to some logging libraries to redefine the default values (classic levels) will result to a performance impact for those that do not need it (atomic cost).
It’s not really fun to use really 🤔
And so for our string challenger…
Well as you can see here, log2 library is kept simple, because everything is handled by the implementation of the log library. No level (re)definition needed? ok no atomic or anything else. Need it? ok well choose your best implementation 💪.
It’s even better because it gives to developers of the logging library the freedom to choose which type will be used (uint8, int, …), to use mutexes or not (and which), and therefore to have better performance.
This round goes without hesitation to the string type !!
It’s interesting, but the purpose of logs anyway is to be able to display or send messages!
Round 3: Displaying log level
Let’s take our example
We expect to have format that looks like this
2020/09/19 21:32:32 WARN [300] my msgFor the type integer, the translation role to transform log2.WARN() (here the integer 300) into a string of character is provided by the implementation of the logging library. For the type string, it couldn’t be easier … because we already have what we want to display!
Let’s go a little further. In the case where the implementation of the logging library allows to define custom level, let’s say an application developer add a FOO level with the value 300.
And let’s use it with integer
What do you think will be the result? 🤔… unexpected, because the value 300 is associated to WARN and FOO strings of character… So 300 has two possible display choices, and we can not determine the good one.
OK…., and now for string
You will have…
2020/09/19 21:32:32 FOO [300] my msg for level FOO
2020/09/19 21:32:32 WARN [300] my msg for level WARN
2020/09/19 21:32:32 FOO [300] my msg for level FOOIt’s so easy because FOO (who is a string of character) will display FOO, and log2.WARN (who is also a string of character) will display humm let’s me think… 🤔… WARN, even if they share the same value 300.
We know the winner of this round… string 💪.
Conclusion
Finally, we see that type string is the good choice, because he has the advantages of:
- being more readable by the developer.
- giving more freedom to the implementation of the logging library (which type for the level values, redefine level or not) without loss of performance.
- giving coherency between what we want to log and what we see displayed.
- defining a simpler common base for implementing logging libraries.
You can find an implementation of all this cases here https://gitlab.com/SylvainDumas/log2_article_part1
