Open Sourcing Tools to Help Read Hospital Log Data

Pavlo Dyban
Feb 24 · 3 min read

I joined Doctolib as a technical project manager a few months ago. Among others, my responsibilities include integrating Doctolib software into the IT infrastructure of the various hospitals and making sure the interfaces with hospital information systems (HIS) run reliably.

Behind the scenes at Doctolib

Doctolib offers a multitude of standard-conform interfaces to connect our fantastic booking software to a variety of HIS. Normally, the communication between medical software in a hospital is regulated by a central communication server (aka enterprise integration application, EAI).

Hospital information system
Hospital information system
A hospital information system consists of a manifold of components and interfaces (source: https://existek.com/blog/hospital-managment-system/)

In the past few months, I had multiple occasions on which I had to debug inexplicable behaviour of a medical software interface in the hospital and my only point of reference were plain-text log files produced by a third-party communication server. I spent long hours scrolling through anonymized log files (small green letters on a black background of the terminal command-line window!), reading through UNIX timestamps and cryptic HTTP error codes.

Reading plain-text log files is not an activity that excites me much, to say the least. Anybody who has debugged third-party software by reading through log files knows how painfully cumbersome it can be to find an error in hundreds and thousands of lines of timestamps and error codes. Now this is where I took a break from reading and developed a standalone log file parser — an application that converts plain-text log files to structured data.

I developed a Python application that parses plain-text log files, structures them into a dictionary, filters unnecessary information and writes the result into a JSON file. I can now visualize a plain-text logfile in pandas or any other JSON-to-table viewer and recognize the error sources at first glance. I can even sort log files by error codes or content strings. My new tool saves me a deal from my work time!

Input: plain-text log file
Output: a pandas data frame in a Jupyter notebook

How to use it

hospital_logfile_analyzer is available via Python’s Pypi package index.

Install it in your (virtual) Python environment:

pip install hospital_logfile_analyzer

Use it as a standalone executable:

python -m hospital_logfile_analyzer — help

Or use it as a Python package:

from hospital_logfile_analyzer.parsers import parse
parser = parse(‘hospital_logfile_analyzer/test/test.log’)
print(len(parser.events))

This tool has made my job so much more effective, I thought I should share it with the open-source community. Currently, only one format of log files is supported (Infor Cloverleaf), but I expect to see more parsers added in the near future. Clone it, enjoy it, contribute to it and shape the future of healthcare together with Doctolib!

We at Doctolib believe that this tool will be a valuable aid for IT administration of hospital networks and beyond. While following our mission to revolutionize healthcare, we support not only patients, doctors and assistants with easier and better access to healthcare, but we also contribute to the well-being of the IT personnel by removing repetitive tasks and making their everyday life a little bit more effective and fun!

Read more on how to install and use the Python package in Pypi and on Doctolib’s Github. Under MIT license this project is available for unlimited (for and non-) commercial usage and is now open to your contributions!

Sign up for weekly updates from the tech team at Doctolib and more awesome Open Source stuff like this.

Disclaimer: This application can only be used with anonymized data. Anonymized is data that does not contain any personalized information in accordance with GDPR.

Pavlo Dyban

Written by

Driving the future of healthcare!

Doctolib

Doctolib

Pour un système de santé plus humain, efficace et connecté

More from Doctolib

More on Open Source from Doctolib

More on Open Source from Doctolib

More on Open Source from Doctolib

Helping scientific research @ Doctolib

1K

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade