IT audit at the ECA — assessing the digital environment in policy areas
In our modern society, it is difficult to imagine a policy initiative or programme that would be implemented without the support of IT systems. So, auditing any policy often means assessing the quality of the underlying IT systems. But how aware are auditors of this fact? Even more importantly, how well equipped are auditors to make such assessments? Franck Noël is a Principal Manager with extensive experience in working with IT systems at both the European Commission and the ECA. Here he presents some key aspects of the IT audit assessments done at the ECA and the different domains that making up the digitalisation of audit.
By Franck Noël, Audit Quality Control Directorate
The world is already digital
These days, many discussions refer to ‘digitalisation’. Digitalisation of our society, digitalisation of policies, digitalisation of audit. In fact, though, western societies and economies have already been digital for a while. Bank accounts are simply records in databases; our favourite songs are electronic files stored on servers in the cloud; university courses are accessible online.
Back in 2007, Wolfgang Schäuble, then Germany Minister of Interior, said that any political project involved or would result in an IT project. This was 12 years ago. Indeed, the free movement of goods within the European Union relies on the new computerised transit system; security policy is implemented with the help of systems such as the Schengen information system (see Box 1); carbon emissions are traded via the Emission Trading System.
Against this background, auditing means auditing in a digital environment and, increasingly, auditing digital evidence in a fully digital environment. Whether it is for a financial or a performance audit, auditors collect data and documents that are natively digital. Determining the reliability of such data and thus the trust that can be granted to such data, depends on our capacity to fully understand how these data are loaded, processed and extracted by the systems storing them. This is where IT audit plays a role.
What is IT audit?
The International Standards of Supreme Audit Institutions (ISSAI) 5300 Guidelines on IT Audit define such audits as ‘an examination and review of IT systems and related controls to gain assurance or identify violations of the principles of legality, efficiency, economy and effectiveness of the IT system and related controls’. Such audits may be carried out either as a stand-alone exercise to assess the performance of an IT system, or as part of a financial, compliance or performance audit of an entity (its integrity and capacity to deliver) or subject matter where an IT system (or a component of that system) plays an important role.
IT systems belong to information systems, which also comprise processes and the people operating them, and are controlled by the overall IT governance framework in place at the audited organisation. Therefore, an audit of an IT system encompasses not only technicalities but also the decisions and decision-making processes impacting their lifecycle. Operational aspects must also be considered to ascertain the reliability of the data of these systems. The scope of an IT audit may therefore cover fields as varied as cybersecurity, data protection, governance and business processes.
Box 1 — ECA special report 20/2019 on EU information systems supporting border control
The creation of the Schengen area, which abolished border checks between 22 participating Member States and four other European countries, increased the importance of effective control and surveillance of the area’s external borders to prevent crime and terrorism and to control migration. According to estimates, the EU budget provided over €600 million to set up the IT systems to facilitate the work of border guards.
The ECA examined how well the main IT systems for border control allowed border guards to check individuals entering the Schengen area at authorised border-crossing points — land, seaports and airports. The audit aimed to identify aspects in the design and use of these systems that helped border guards do their job more efficiently. The IT systems concerned checks on people and objects, visas and asylum applications, fingerprint comparisons and passenger records.
The ECA concluded that the IT systems were a strong tool increasingly used by border guards performing border checks. However, some data was not yet included in the systems, while other data was either incomplete or not entered in a timely manner. This reduced the efficiency of some border checks. The border control authorities should focus more on entering complete data promptly in the EU’s information systems involved in supporting surveillance of the Schengen area’s external borders.
IT audit at the ECA
Many audits we carry out have an IT dimension to a greater or lesser extent. IT audit is therefore an important field for us. In recognition of this, and eager to improve related practice, the ECA launched an IT audit self-assessment in November 2018, using a method developed by the IT Working Group of the European Organisation of Supreme Audit Institutions (EUROSAI).
The IT audit self-assessment involved a group of auditors filling out a questionnaire to assess the IT audit function of their SAI, and then attending a workshop to discuss the results and define steps for improvement. External moderators from Switzerland and Malta facilitated the process. Overall, participants in the IT audit self-assessment perceived the maturity of the ECA to be moderate, with an average score of 2.2 for IT audit aspects on a scale from 1 to 5.
Table 1 shows that participants aim for around a level 4 in all aspects of IT audit, thereby demonstrating the appetite for IT audit at the ECA, as well as the need to take concrete action to develop this field internally. The disparity between the maturity and ambition assessments clearly shows there is still some work ahead of us.
Table 1 — IT audit aspirations: results of the 2018 IT audit self-assessment at the ECA
This assessment pinpointed areas where our procedures could be improved. As a starting point, considering IT risks inherent to a policy area or a specific audit is not yet part of our standard practice. One of the axes for improvement is thus to raise awareness about IT risks, including in policy scans, in the programming cycle and, of course, during the planning phase of an audit. In this respect, knowledge of the European Commission’s information systems landscape appears to be of utmost importance.
To assess IT risks in a systematic manner, audit guidance and methodology should be adapted to make sure that our audits consider IT elements wherever necessary and relevant. The review and elaboration of such guidance will of course require the attention of a dedicated team. It may also result from collective intelligence work conducted by an internal network of IT auditors.
Several auditors at the ECA hold an IT audit certification. They should form the kernel of a new internal network — or ‘knowledge node’ to use the ECA jargon — whose role would be to contribute to and review methodological guidance on IT risk assessment and IT audit, and to support specific audits should the need arise. This network should of course be open to all interested ECA staff, in particular auditors but also IT professionals.
Such a network should also identify emerging training needs in the IT audit field. Training courses on IT audit may need to be reassessed in the light of the IT audit self-assessment, with the possible addition of new modules. All aspects of IT audit should be tackled, for instance, cybersecurity, auditing IT in the cloud, and IT governance frameworks such as COBIT (Control Objectives for Information and Related Technologies) — a framework for IT management and IT governance.
IT audit, digital audit, IT support to audit
The world and modern organisations, such as the ECA, have gone and are increasingly digital. When it comes to the digitalisation of the audit discipline, we must distinguish between three separate domains: IT audit, digital audit and IT support to audit.
IT audit refers to the examination and review of IT systems and possibly of the wider IT environment, as described in this article. Digital audit involves using advanced techniques to perform an audit, for instance, data mining or software robots. So, the former concerns the object of the audit, the latter concerns the audit technique. IT support to audit, by contrast, relates to the set of IT tools that are built to facilitate and document the work of auditors. It of course covers audit documentation systems, but could also encompass digital exchange platforms between auditors and auditees to, for instance, securely collect audit evidence or share findings with a view to implementing an online clearing exercise.
As a modern public audit institution, the ECA strives to progress in all three domains by understanding and listening to the needs expressed by its stakeholders and staff, and by identifying and embracing the possibilities offered by technology.
This article was first published on the 1/2020 issue of the ECA Journal. The contents of the interviews and the articles are the sole responsibility of the interviewees and authors and do not necessarily reflect the opinion of the European Court of Auditors.
