PDF reports in Python web applications — Part 1: Server side report generation
When starting developing a B2B web application for simplifying time tracking and invoicing in 2011 we had to decide on the reporting tool we were going to use for PDF and Excel exports of invoices, statistics, time reports and so on.
Besides standardized reports we wanted to customize reports to our clients individual needs by adapting the layout and content (e.g. of an invoice). Based on these considerations we were looking for a report generation tool that includes a design tool allowing us to visually create and adapt reports whenever needed.
Our application is written in Python and the obvious tool to use would be ReportLab. However, while ReportLab provides the functionality we needed it lacked the design tool. That’s why we started using JasperReports Server together with Jaspersoft Studio as the design tool. Even though our application had no need for data warehousing and wouldn’t use most of Jasper’s reporting capabilities for complex data visualisations it best met our requirements by that time.
After using Jasper for some years we felt it was time for a change
We decided to develop our own reporting framework to fulfill our needs: Written in Python, lightweight, fast and optimized for small reports and, most importantly, a visual designer which can be embedded into a web application.
Why would we do that?
To start with I’ll focus on the server side reasons by comparing different aspects between the well-established Java-based reporting framework Jasper and ReportBro. I’ll discuss the client side aspects in another article.
With regards to a web application with a Python server backend we identified 5 major drawbacks using a JAVA-based reporting framework:
1. Installation
Jasper requires a Java Runtime Environment (JRE) plus corresponding Tomcat server configuration. Besides the necessity of adopting the appropriate know-how regarding its setup and optimization this means to reserve designated resources for running the reporting server. The JasperReports Server needs to be installed and started so that reports can be generated with REST requests (see Request handling below).
ReportBro is installed from PyPI just like other Python packages:
$ pip install reportbro-lib2. Resource allocation
Running and servicing a resource-intense JAVA server for the purpose of reporting only is a major disadvantage when the main application intends to be a lightweight and process-optimized Python web application.
With ReportBro we can directly generate the reports in our server backend, everything runs in the same thread. There is no additional overhead.
3. Request handling
Users of web applications expect quick results and low latency. Especially when creating hundreds of relatively simple reports at once — all of them with a fairly manageable amount of data (like invoices or monthly analysis) — overall application performance shouldn’t be interfered by report generation.
We call a controller in our application to access a report. This way we can use our already existing authentication/authorization, access data and perform calculations as needed.
When generating a report with Jasper we perform a REST Request to the JasperReport Server in the controller. We use the jasperclient lib for a convenient way to perform the request. Initially we faced some minor hurdles, e.g. howto encode date parameters, encoding <, > and & in strings and so on. In our web application it is possible to download multiple (up to 100) reports at once — they are generated individually and afterwards merged into a single PDF file. This can take some time because each report has to be created by a separate REST request. With ReportBro we can omit this overhead because the reports can be directly created in the application controller itself. We also do not have to convert any data in our server backend — we can simply pass all native Python objects to ReportBro as report parameters.
4. Data processing
Jasper is specialized in processing huge volumes of data. Therefore, the request for data is usually done within the report via data source (e.g. a database query). This also implies the risk of inconsistencies due to separated data queries and data processing within the application (for the web view) and the report. We can eliminate the risk of data inconsistency in ReportBro by using the same functions for data requests both in the web application and for report generation. This is possible because both the controller for the web view and for the report are executed in Python.
5. Creation and maintenance of reports
We have a separate instance for each of our clients, this means we also have separate databases and individual reports. Therefore, each time we setup a new client we also have to configure the client reports. In Jasper this includes creation of a data source (for the database query), folder (for the customer), report objects (containing the report itself) and report units to access a report with the correct data source. We also have to update the report on the server everytime the report is changed. These tasks are quite cumbersome which led us to write a script to automate the repeating jobs as much as possible.
In ReportBro the report template is passed to the lib when creating a report. There are no additional dependencies like data sources. Therefore, we can easily integrate report maintenance in our web application by storing the report template in our application database. This saves us from a lot of annoying administrative work, especially when we have to maintain different versions of a report.
This article is the first of a two part series. The second article, PDF reports in Python web applications — Part 2: Browser-based report template design, covers client side aspects that led us to developing our own reporting framework.
See and try ReportBro: https://www.reportbro.com
Python package on github: https://github.com/jobsta/reportbro-lib
JavaScript library on github: https://github.com/jobsta/reportbro-designer
