Making digital audit part of the ECA’s DNA
Interview with Magdalena Cordero and Mariusz Pomienski, ECA directors
Digital transformation is one of the key objectives of the ECA for the upcoming years. But how to put such objective into practice? Where do you start and who do you need to involve? And how does it affect work processes, auditors’ skills and relations with the auditee? Magdalena Cordero is Director of Information, Workplace and Innovation and has been working for years on innovation and digital transformation at the ECA. Mariusz Pomienski is Director of the Financing and Administering the Union Directorate in charge of the financial and compliance audit work that underpins the ECA’s annual Statement of Assurance and he is closely involved in the project ‘ECA audit goes digital.’ We interviewed them to discover their views on what digital transformation means for ECA auditors and what the two directors consider to be the key ingredients to make the transformation happen.
By Derek Meijers and Gaston Moonen
The ‘ECA audit goes digital’ project as stepping stone
Magdalena has been working on digitalisation and audit for quite some time. ‘I started facilitating international working groups on the use of digital tools in audit and, little by little, you see things moving, with a take-up in several audits and in several countries. But there comes a moment when further progress requires clear decisions, and I think this moment has arrived for the ECA. To make things happen pressure is required from different positions: from the top of the organisation, bottom-up, and also from the sides, from peers and facilitating services.’
For Mariusz, digital transformation has so far not been his daily ‘bread and butter’: ‘I got more involved in digital transformation through the project ‘ECA audit goes digital,’ the project that is led by the ECA Digital Steering Committee consisting of six ECA Members and chaired by one of the, Eva Lindström. This Committee was set up in Spring 2019 and is supported by three directors — Magdalena, Geoffrey Simpson, the director responsible for quality control, and myself. The project manager is Julia Pliarczyk, an auditor from the Investment for Cohesion, Growth and Inclusion Directorate.’
Mariusz explains that the project’s objective is the digitalisation of audit at the ECA. ‘People all around in the ECA are faced with digital reality on a daily basis. We live in a digital world — whether we like it or not — and auditors, naturally and intuitively, often try to get a grasp of it. Quite a few initiatives have been developed already, many of them channelled through ECALab with the help of Magdalena’s directorate.’ He adds that now they want to take good stock of what we already have at the ECA before setting up a development phase, and of what else needs to be initiated to achieve this digital transformation in the ECA’s work. ‘We are currently at stage 1 of the project, which is the Status Quo Analysis, meaning that we first identify what has been happening until now within the ECA. An important part of this first phase is also looking at the Commission — our main client, our main auditee. Because we foresee that quite a lot will depend on how they react to our digital progress, as auditors. If they do not go digital, or digital enough, we might find it difficult to become as digital as we would like to be.’
“If they [the Commission] do not go digital, or digital enough, we might find it difficult to become as digital as we would like to be”.
Mariusz underlines that none of the auditors are excluded from the ‘ECA goes digital’ project. ‘But for now, the main focus in the early phase will be financial and compliance audits where the audit procedures are more standardised.’ His hope is that focusing on this audit domain will bring recurrent benefits. ‘Such as improving the processes, also from an automation point of view.’ In this connection he also refers to an ongoing pilot project at the ECA in relation to auditing executive agencies. ‘They have identified processes that could be subject to automated audit procedures. I am sure their results and experiences will show real benefits and feed into our financial and compliance audit work.’
Magdalena explains that the bottom-up instrument for her was the ECALab. ‘It was my dream years ago and I’m very proud we have got to where we are now. It was very important to have this instrument in place, combined with what I call the ‘lateral’ pressure.’ For her this means information on what the ECA’s peers are doing: talking with others, taking initiatives to facilitate exchange. ‘From one year to another we see the progress other supreme audit institutions [SAIs] have made. This exchange with the outside world is very important, and it will contribute to the realisation of my new dream: creating a centre of excellence and knowledge on public audit, linked to the ECA.’ She conveys that this is linked to another dream of hers: to create in Luxembourg — preferably in the historical Schuman building — a centre of knowledge on the construction of the EU. ‘But coming back to digitalisation: for me that is already a reality, it will happen with or without me, also for the ECA. This is not always easy, but we will get there, we are now taking concrete steps in that direction. In the end it is about cultural change.
“But coming back to digitalisation (…) In the end it is about cultural change”.
Searching and finding inspiration…
It also became apparent during the conference the ECA organised on Big and Open Data, that others have become aware that the ECA is taking concrete steps. When discussing what inspires the ECA to take these steps Magdalena is clear: ‘You need to be inspired by those who were there before you and since it is very unlikely that you have a genius who sees things and can convince you that you should do them, you often look outside.’ She adds that there are many different ways to innovate. ‘You can innovate because you change your process; you can innovate because you change your product; sometimes there is a way of innovating that is called positioning — you take what others are doing in another domain and you apply it to your domain. When you do that, you see that, suddenly, you have a new line of innovation.’
She gives an example. ‘Take the ‘smart data’ approach Eurostat has launched. We can learn a lot from it and bring that knowledge inside our house and see how we can apply it to our own processes, to our own organisation.’ For her, digital audit, a term used by her directorate since 2017, contains three core elements. ‘One is automation, as was also mentioned by Mariusz: automating things that were or are done manually.’ An important aspect here is to optimise this automation by bringing in ‘control by design,’ meaning that some key control issues are already embedded in the automated process. ‘The second one is data analysis. This is not only relevant for financial and compliance audit but also for performance audit.’ She argues that too often performance auditors think of data as numbers, as structured data. ‘Data includes text, images, data is all kinds of information that is digitalised, for example, social media are also data.’
On this issue, Mariusz continues that the environments of private auditors and public auditors are not necessarily the same. ‘Private audit firms mainly focus on financial audit. Many of our audits, on the contrary, are mainly focused on non-financial information. In our compliance audits, checking the legality and regularity of transactions, we compare the reality with what the Commission, or other management authorities, tell us. In this picture the accounts are only a small part of this reality.’ He explains that most of this information is non-financial, and often non-structured. ‘This is also why private audit firms may advance differently than we in public audit do, and is what makes our auditee a different type of auditee in digitalisation than their clients. And what makes the challenge for us regarding digitalisation much greater.’ Magdalena concurs: ‘In a way our audit questions and our audit environment are more complex.’
For Magdalena the third core element of digital audit is the audit of digitalisation itself. ‘In the past we would speak about IT audit and this third element builds on that. As auditors, our role is to contribute to trust regarding the systems and processes used. And since the system is more and more digital, then the role of auditors is to provide trust in that digital environment.’ She refers to an audit case brought forward by the French SAI. ‘They were auditing an algorithm that was used when assigning people to universities. As auditor you need to verify and ensure that there is no bias in the tool used.’ She underlines that, unlike the former role of the IT auditor, who was auditing mainly IT governance and project management, the audit of digitalisation goes further. ‘We need to assess whether the system has any bias, we need to audit the algorithms, and assess whether the system is secure so nobody can modify the data at some moment in time, and if they do, then with clear trails; cyber-security is one of the big risks of digitalisation. All this relates to trust, the core of an auditor’s profession.’
“As auditor you need to verify and ensure that there is no bias in the tool used”.
… to strengthen the foundations of audit
Mariusz adds that he could not agree more. ‘The bottom line is that the foundations of audit will not be changed by digitalisation: we will continue to provide our judgement, applying professional scepticism. But with this ever growing component of IT, of digital, in our life, it will also be a growing component in our work. With the data we get from our auditee, every stage of our audit activity becomes more digital. It will impact our risk assessment, our planning, our preparations, our audit ideas, audit implementation, and our reporting and communication about it.’
On this issue Magdalena explains further that, with digitalisation, analysing reality as it was when it was created is crucial. ‘With an information system you can ‘build’ information in any way you want, even on wrong data. And you can try to prevent others from following the right ways to find out. Creation and modification of data needs to be recorded so that auditors can analyse what happened. We need this extra layer of security and blockchain can help in this context.’ She refers to Estonia, where a lot of personal information is used for public administration purposes. ‘People can access this information — or not — but all this is registered in a blockchain, the logs of those transactions are there and immutable.’
For Mariusz the prerequisite for digital audit remains the same as in the past: the data must be reliable. ‘But for me the most tempting thing about digitalisation is that we simply give better assurance, better information to our stakeholders. It enables us, hopefully, to provide assurance on testing the whole population. Which is an auditor’s dream, in the sense that you are close to 100 % sure.’ He explains that in this way you can elminate certain risks involved in sampling or extrapolating.‘
“… testing the whole population. Which is an auditor’s dream, in the sense that you are close to 100 % sure”.
Magdalena presents another advantage. ‘With sampling, a transaction containing an error is considered as an error. But if you analyse the full population you may discover that there are 500 or 1000, or even more transactions that follow the same pattern. So what before was labelled an error can then be identified as possible fraud’ Another example she gives relates to process mining. ‘It is about processing the logs. Any activity in an information system is registered in the log. The analysis of the logs allows you to easily identify transactions that didn’t follow the “normal” path, and the auditor can concentrate the analysis on these cases. This analysis can produce interesting results from a performance perspective.’
Assessing the digital architecture
Auditing digitalisation means for Magdalena that auditors have a role in looking at the digitalisation process since the starting point. ‘The construction of new information systems represents an enormous investment for the public sector that must demonstrate performance and value for money. A solid IT architecture is the foundation for building a real transformation.‘ She refers to the 2017 Ministerial Tallinn Declaration on eGovernment by which EU Member States committed themselves to building user-centric digital public services, highlighting interoperability and openness. ‘The new European Commission has committed itself to a digital Union and will invest a lot of money in it. As auditors we need to see that the systems work and that they protect EU citizens’ interests. This includes looking at the architecture of these systems.’
“As auditors we need to see that the systems work (…)This includes looking at the architecture of these systems”.
Mariusz is more cautious as to what extent auditors can play such a role. ‘Regarding the external auditor’s role in the whole building process of the IT — or broader — internal control systems, I think it might be quite a challenge to reconcile this with our independence and our ability to then audit ex-post, the way we traditionally work. There is this fine line that we need to be careful not to cross, becoming a type of consultant.’ An important argument for him is that the auditor’s role is later to provide assurance on what is being produced with the system. ‘I think there is a bit of a grey zone. ‘Our tool is audit; we can audit the design, we can look at it, but guard our distance vis-à-vis the auditee and also not become part of the management.’
Failure as part of the learning process
As the Director of Information, Workplace and Innovation, Magdalena has the role of facilitator, innovator and enabler for auditors with regard to digital audit. One of the core elements in doing this has been the ECALab, launched in 2017 (see p xx for more details on the ECALab). She explains that with the ECALab the intention was to experiment, to raise awareness and to create an appetite, not only for technology, but stimulating the right questions on what can be done with the digital means available. ‘Now the appetite is there and when there is a new audit task several ECA Members — but not all of them yet — and directors will consider using the ECALab. It is “à la mode,” quite different from three years ago. Now everybody talks about digitalisation and about digital.’
However, she underlines that the ECA needs to move forward and perform a real organisational transformation. This means designing structural training programmes for auditors, going beyond what we did in Pisa. She underlines that the idea is not to create an enormous ECAlab. ‘Auditors need to build their own capacity, the audit chambers need to understand that they need to train their staff and recruit new profiles. This requires a cultural change, an understanding that moving to digital audit also means allowing room for experimentation. And if people experiment this means that in some cases they will succeed, and in some cases they are going to fail.’
For Magdalena, it is clear that this concept of failure is not easy for an audit organisation. ‘The feedback I get is sometimes, “We cannot make the same mistakes our auditees are making.” But why should we assume we are better? Of course, we are going to make some of the same mistakes as they do. As our chair of the Digital Steering Committee, Eva Lindström, has indicated, we need to allow failure. Otherwise we cannot progress as fast as we should.’ In her view, auditors should not be afraid to fail and management should allow room for that. ‘You need to try. We can lead by example by allowing room for that. For the EU and digitalisation, to make it happen, you have to experiment…and fail now and then. Otherwise, we will lag behind the rest of the world. In some elements of digitalisation we are already lagging behind.’
“For the EU and digitalisation, to make it happen, you have to experiment…and fail now and then”.
Magdalena underlines that the cultural change needs to start at the top. ‘Senior management needs to be generous, meaning that they need to allocate permanent resources to experiment for digital audit.’ She quotes an example. ‘The Finnish Auditor General, Tytti Zli-Viikari, said: “Any financial auditor must be an IT auditor,” thereby underlining the importance of digital.’ She continues with another example from the same audit office. ‘She put together a number of people, not auditors, and asked them to come back with an example of use of AI in audit, a team with expertise in different fields.’ She points out that if you want to transform — in this case transform digitally — you need people who have an understanding of what it is about, you need people who have the skills, and people who have vision. ‘This transformation doesn’t necessarily require a deep change in current organisational structure, it can work just by allowing for organic growth. This can work well. But there are also other ways, perhaps more disruptive, like the approach we see in the Finnish SAI.’
“Senior management needs to be generous [and] allocate permanent resources to experiment for digital audit”.
As to whether the auditor can go faster on digitalisation than the auditee, she comes back to the digital architecture and controls by design. ‘I was working at the statistical office in Spain in 1986 when Spain joined the EU. At that time, Eurostat was receiving all the data on paper. I was assigned to an international working group on telematics networks, aiming to create a network allowing any Member State to send information already in digital format so it would be filed automatically when arriving at Eurostat. Imagine, this was 1986/87. Eurostat, the European Commission, provided equipment to every Member State, if needed, to achieve connectivity, including paying for the line. Because they wanted to make it happen.’
She points out that the ambitions of the new European Commission for a digital Europe offer new opportunities to build in controls by design. ‘With Digital Europe…big investments will be made and we should use that opportunity to stimulate controls by design. ‘Some organisations have taken the initiative to contribute to the digitalisation of the auditee. The World Bank is building a system on blockchain to help the auditee to register all financial documentation, including smart contracts and even a full financial process. When the ECA launched its pilot on blockchain, our intention, although less ambitious, was similar: to simplify the process for the auditee by means of digitalisation.’
Outlook for digital audit
When it comes to where the ECA would be five years after the start of the ‘ECA audit goes digital’ project, Mariusz takes a modest view. ‘I think, of course, it is progressing well. But my experience with the pace of change is such that, whatever I think of what is going to happen, or where it is going to be in five years, will probably be obsolete by the time we get to that point.’ In his view, it is clearly a moving target. ‘But I do think that in five years from now, for sure, we will be auditing differently. How differently will depend, on the one hand, on how successful we are in changing our organisation, in following-up the successful pilots, extending their scope, going step by step.’
However, he also believes it depends on how the Commission will move forward. ‘In the private audit world, obviously, the auditee is very much interested in helping their auditor to become digital. Simply because they will not only get a better product, but also it will be faster and presumably cheaper. I hope the Commission will have the same incentive, namely that they are interested in us, as a public auditor, being faster, better, and cheaper — of course that remains to be seen. But I do hope that five years from now both of us, the Commission and the ECA, will be in a changed reality and that audit will look profoundly different in practice, but fulfil the same basic objective as now.’
“… five years from now both of us, the Commission and the ECA, will be in a changed reality and that audit will look profoundly different in practice…”
Magdalena shares Mariusz’s sentiment on the pace with which changes can take place. ‘So I would not like to look further than five years from now. For sure, digitalisation is going to happen. The process has started already and it will not stop just look at the world around us and our audit peers.’
She summarizes her perspective around three key aspects. ‘First of all we need to ensure that the results of our audits are relevant. This is not easy, because the world is evolving and we need to be sure we keep up with that.’ The second aspect for her is trust. ‘Trust in the digital world will be essential. This is where the auditors need to work a lot. I would like to see that the ECA is able to contribute substantially in the digital world of the EU, in any context.’ The third and last aspect for her is speed. ‘We will need to work at a certain speed and that must take into consideration the speed of the auditee. This means that in some cases we need to run because we are behind. But in others we need to provide help to the auditees in order to keep them up to our level. Because we are the ones who want to work in a different manner, providing more added value.’
“Trust in the digital world will be essential. This is where the auditors need to work a lot”.
For all of this to happen Magdalena reiterates the need for cultural change. ‘With the — as I like to call it — generosity of senior management. Digitalisation of audit is a dream for me any longer. My dreams are always about all those initiatives where I need to talk and persuade people at different levels, and share my ideas and convince them! And the digitalisation of audit is already a reality, it only requires hard work.’
This article was first published on the 1/2020 issue of the ECA Journal. The contents of the interviews and the articles are the sole responsibility of the interviewees and authors and do not necessarily reflect the opinion of the European Court of Auditors.
