The ECALab — our in-house incubator for applying data analytics, data visualisation and process mining to audit
In 2017, the ECA launched the ECALab as a centre for exploring ideas and technologies and building on the new possibilities offered by digitalisation. In this article Emanuele Fossati, ECALab coordinator, Clare Reilly, a former ECA trainee who is currently employed at Deloitte in Dublin, and Carmen Schnell, assistant in the private office of Annemie Turtelboom, ECA Member, zoom in on the role of the ECALab.
By Emanuele Fossati, Information, Workplace and Innovation Directorate, Clare Reilly, Deloitte, and Carmen Schnell, private office of Annemie Turtelboom, ECA Member
Technology and audit
Technological advances provide a great opportunity for audit. New tools could see a shift from traditional sampling techniques to real-time whole population audits, allowing for results that are more accurate as well as greater accountability and transparency. Automating repetitive tasks could free up valuable audit time, while data visualisation and data analytics may facilitate a more investigative analysis in the early stages of an audit. The potential of predictive auditing is equally vast.
However, these opportunities also entail major challenges for auditors. Testing entire populations and managing a greater number of outliers will undoubtedly change the role of the auditor. The ECALab serves as an in-house incubator for exploring new ideas in audit and offers auditors a variety of techniques that can assist in achieving individual audit goals.
Box 1 — Why the ECALab?
Emerging technologies not only allow auditors to efficiently manage the changes prompted by digitalisation, but also improve processes and create rewarding new opportunities for audit. For many years, the private sector audit firms have been exploring how to make best use of data-driven audits. The Big Four audit firms have embraced the opportunity to explore areas such as artificial intelligence (AI) and blockchain, so why should supreme audit institutions not follow the same path? In 2017, the ECA responded to this question by launching the ECALab, an in-house centre for research and innovation. This space for sharing ideas, exploring, testing and implementing technologies in the audit process is part of the ECA’s digital transformation initiative. The ECALab is comprised of data science enthusiasts and expert auditors who cooperate on finding tailored solutions to audit tasks and audit-related projects.
Staying up-to-date with innovation
The work of the ECALab is crucial in ensuring that the ECA will stay up-to-date with innovation, as technological advances are occurring at an unprecedented rate. This involves a continuous technology watch, internal experimentation, learning from other EU institutions and supreme audit institutions (SAIs) and the transfer of knowledge (see Figure 1).
Figure 1 — ECALab helps to stay up-to-date with innovation
The ECALab therefore helps our auditors to make practical use of innovative approaches like data analytics, data visualisation and process mining in their audits. Other new technologies, such as blockchain, text mining, etc. are also tested and applied in the ECALab.
Data analytics — techniques for identifying patterns and outliers
Data analytics is the science of analysing raw data and drawing conclusions from the process. It is all about discovering and examining patterns. Not only does it help auditors manage and interpret an ever-increasing amount of data, but it also allows them to gather further audit evidence by analysing large populations rather than small non-representative samples. By identifying patterns, clusters and outliers, auditors gain an enhanced vision, allowing them also to detect and judge the relevance of causality during data analysis.
Technology can help identify previously unrecognised patterns or clusters of interest and therefore raise questions which would have otherwise been overlooked. Such an open approach is valuable in that it permits the exploratory analysis of audit evidence and has the potential to increase the audit scope. Data analytics could bring further added value to audit by enabling the projection or forecasting of future events. However, this would see the role of audit shift from reliable historical cost accountancy to estimations and forecasts and should therefore be treated with utmost caution to avoid separating audit from verifiable facts (1).
Data visualisation — translating data analytics into visual format
A more advanced area of data analytics is the visualisation of both structured and unstructured data. By extracting, processing and translating raw data to visual formats, data visualisation software makes content more accessible and easier to interpret. Just like data analytics, data visualisation allows auditors to analyse entire data sets and thereby identify outliers, patterns and irregularities. While both techniques can help raise further questions and thereby expand the audit scope, data visualisation also acts as a valuable communication tool. Intertwined with text, it can help to convey essential information. Informative data-driven visuals can assist auditors in writing reports that tell an engaging, coherent and conceivable story.
Figure 2 — Example of a data-driven visual
Data visualisation not only enhances reports visually, but also contributes to making them more memorable. At the ECALab, data scientists are already creating interactive visuals by using tools, such as Tableau, PowerBI, Gephi (visualisation of networks), Maltego, and Plotly , which allow for an even more dynamic and engaging reading experience and deeper analysis. These provide an overall picture of the data and enable the reader to zoom in on specific statistics of interest. This technique can be applied to any kind of visual, be it maps, timelines, bar charts or others.
At this stage, interactive visualisations are particularly interesting for communication purposes following the publication of a report. However, as the audit world sees a shift towards electronic reports, the role of interactive data visualisation is set to become more prevalent in the future.
Process mining — combining data analytics and data visualisation
Process mining is one of the technologies tested at the ECALab which could play a significant role in the digital transformation of audit. It combines data analytics and data visualisation to allow auditors to analyse and detect patterns and irregularities in both static data and entire procedures. It works by extracting event logs to reconstruct and visualise processes. Due to the increasing digitalisation of auditees’ processes and records, log data has become more accessible, and auditors can now examine very detailed audit trails.
The ECALab was established as a place where people can experiment with new ideas and technologies, organise demos and discuss their results with colleagues. As testing always works best on real cases, why should the wealth of real information found at the ECA not be used for experimenting and learning? Box 2 provides two recent examples of how this can be done.
Box 2 — Two examples of the ECALab’s contribution to recent audit tasks
In September 2017, in the context of an audit task which resulted in the Landscape Review ‘Putting EU law into practice,’ a mixed team of ECALab data scientists and auditors worked together to merge data from various sources in a single integrated model. This allowed for innovative ways of presenting data graphically and greatly facilitated the task of data analysis.
As auditors often have to handle hundreds of documents, a tool allowing them to navigate through these by extracting summaries, prioritising by relevance, connecting similar paragraphs from different documents, etc. could help save time and resources. This kind of AI solution was exactly what another audit team had in mind when it recently approached the ECALab. Text mining or, more generally, natural language processing can be one of the major assets of AI. While the ECALab was not able to provide immediate added value for the task, work it had done previously contributed to the prototype of a useful new tool, the ‘Document Navigator’, which will be made available to all auditors in future.
The ECALab — key activities ahead
The ECALab will have to focus on the following priorities in the next few years:
- Further increasing internal capacity. This will mean training staff volunteers, creating a network of proven external experts and hiring new staff with specific skills in data science. One issue in this context is that data scientists tend nowadays to come from various other business disciplines and therefore cannot be expected to master IT security and/or system administration.
- Further developing interinstitutional cooperation. In view of the significant investment needed to keep up with technological developments, it is immediately obvious that a single (audit) institution cannot go very far. For this reason, the TINA (Technology and Innovation for Audit) initiative was launched during the recent Big and Open Data conference for EU SAIs (see also p. 108).
As regards the ECALab’s continuing role as the in-house incubator and consultancy service for innovative approaches in audit, we will need to cautiously manage our customers’ expectations. Auditors usually bring interesting needs to us and are willing to experiment with new ways of doing things. In some cases, while technology and methodology are heading in the right direction, they will not immediately be productive. This may lead to disillusion and prevent auditors from expressing good ideas in the future. To avoid this kind of scenario, the ECALab only rarely dismisses ideas as ‘not possible’, preferring to adopt a ‘try it and see’ approach. Even if this does not bring the hoped-for results, it will still provide a valuable learning experience for everyone.
Most importantly, however, maintaining close cooperation through different teams of auditors and data scientists will remain an essential factor that will determine the success of the ECA’s digital transformation.
(1) Jonathan Ford and Madison Marriage, ‘The big flaw: auditing in crisis’, The Financial Times (London:August 1, 2018) <https://www.ft.com/content/29ccd60a-85c8-11e8-a29d-73e3d454535d> [accessed 1 June 2019].
This article was first published on the 1/2020 issue of the ECA Journal. The contents of the interviews and the articles are the sole responsibility of the interviewees and authors and do not necessarily reflect the opinion of the European Court of Auditors.
