How to create useful logs
If you log well, your system will recover fast from errors
With experience as a developer, one area you get better in is logging. Paying attention to logging. Capturing the right amount of detail. Knowing where in a call stack to log something, so you don’t have duplication, and you log enough context to be useful. What to log at what level — info, fine, finer, …
- Make the log file human readable — Production issues will require you or your support team members to scan through files. So keep each log entry crisp, easy to scan, so know what it’s about. This means keeping the initial parts short, so the type of entry is visible when the developer is seeing it in a small window or a text file viewer without word-wrap. Use at most 30-40 characters for reference info, like timestamps, source, thread ids.
- Start your log entries with something unique like >>> or ??? — Your log messages will contain text with line breaks. Like stack traces. When trying to process these files, if you have a delimiter, you will be able to read each entry.
- Log context — What was the user asking the system to do. With what information. If there’s an error, what was the data which caused the error. You will log the entire request when you get a web hit, but what specific data caused the problem. The context might be “Sending confirmation email to user email@example.com”, the error could be “unable to connect to email server”, and the relevant data would be “server=mail.mycorp.com;port=456”.
- Log reference info — Remember the person using the log file isn’t going to know what server it was generated on, what node it was for. Try to log as much in each entry, so reading the file is easy. At a minimum, log the following: entry delimiter (ex. ‘>>>’), timestamp (ex. ‘12/30/14 14:35:40'), thread name and id (ex. ‘S89837937987-R8792878998-T28’), log level (ex. ‘INFO’, ‘CRIT’, ‘FIN1', ‘WARN’), module or class and method, context + msg. If possible, try to set the thread name as a combination of the session id and request id. Log such that the reference info has a fixed length.
- Log as INFO for anything which has external calls — any call you make to another system is liable to fail. When calling a database, it will fail. When sending a message, it can timeout. You don’t want a second chance, so log external system interactions with INFO. Internal calls don’t require logging, since you should log the context with the error.
That’s it. Remember the audience for the log — it’s you or the co-workers you eat burritos with. Once you learn to use your logs and not guess at possible errors in production, you will start quickly refining your log entries so they are useful. Logging too much creates issues just as logging too little, but fewer.
It’s a journey, but you will get better and at some point you will start on a positive spiral towards improving this aspect of your development prowess. Good logs recover systems fast and help improve error handling.