Italy in the eGovernment Benchmark 2018: a problem of digital maturity of services or country?
Methodology, measurement objectives and evaluate the results, without neglecting to take into account the structural characteristics of each country
Questo articolo è disponibile anche in italiano
In Fall, every year, we “must” deal with the results of the eGovernment Benchmark: the 2018 edition is just out. I use “must”, since a useful and informative reading is often turned into an opportunity for criticism and recriminations, mostly due to the not always brilliant performance of our country. “Who should have done what, and why it wasn’t done?” “Whose fault that was: the competent Minister or the lethargic and indifferent public administrations?”. And again, “Is the central administration totally incapable or are the local administration totally irresponsible?” “Is it AgID or the High Commissioner for the Digital Agenda – last entry in the list of possible culprits??”. Ok it’s my fault, I admit it; in the last three weeks, after Diego Piacentini left me alone, I did more damage than in the previous twenty years. But now let’s look forward. The benchmarking exercise does not belong to the mystery/detective genre (otherwise Maigret or Montalbano would have been in my place). There are no culprits to identify. Its intended goals are understanding, taking the necessary corrective measures, learning from the best and, if possible, copying and taking them as a model. For a savvy and not resentful reading, it is important to understand the methodology, measurement objectives and, on the basis of these parameters, evaluate the results, without neglecting to take into account the structural characteristics of each country.
In my opinion, this useful interpretative note – written by Daniela Battisti, head of the International Relations of the Digital Transformation Team – wishes to achieve precisely this objective: we are not looking for the culprit, but rather trying to better understand what it’s working and which areas are to be improved.
P.S. Do not be discouraged by the apparent length: the text flows easily and runs fast. And, there are many graphs and pictures!!
Luca Attias, Government Commissioner for the Digital Agenda
Methodology: eGovernment Action Plan Benchmark
The eGovernment Benchmark is an annual report, published by the European Commission – DG CONNECT, which measures the progress of individual Member States in the implementation of the eGovernment Action Plan 2016–2020 road map to the public sector digitisation.
Progress on the priority areas are measured by one or more indicators (top-level benchmarks), grouped in:
- User centricity – Centrality of the user: the characteristics of mobile friendliness and usability of the service (in terms of available online support and feedback mechanisms);
- Transparency – It indicates the level of transparency in the provision of the service i.e. responsibility for the provision and use of personal data;
- Cross-border mobility: to what extent public service users can use online services in another European country;
- Key enablers – Key enablers: the technical pre-conditions for the provision of digital services: electronic identification or authentic sources (ANPR, for example, is an authentic source).
In order to evaluate all the indicators, the Benchmark uses the so-called mystery shoppers,[1] who are two citizens of each Member State, in possession of an Electronic Identification -eID-, which, after receiving specific training to test and measure specific digital public services, behave exactly as potential users; the verification of the mystery shoppers is guided by a checklist covering the following areas:
- services’ availability;
- services’ usability;
- transparency of service delivery of public organisations and handling of personal data;
- integration of IT enablers in the service delivery chain.
Any discrepancies in the evaluation of the two mystery shoppers are evaluated and resolved by a third party. The evaluation of the mystery shoppers focuses on the so-called “life events”:
Life events are package services that are usually provided by the government. The IT systems of the participating government agencies then co-operate (ie interoperate) for the seamless delivery of the e-service.
The Benchmark includes a set of eight life events. Every life event represents a set of public services that citizens or businesses commonly use. Each year, alternatively, four life events are measured; the two-year cycle allows Member States to put the appropriate corrections after each measurement.
In addition to measuring the level of digitisation (digitisation) through the evaluation of mystery shoppers, the Benchmark is complemented by a Benchlearning Exercise.
The Benchlearning measures the maturity of the eGovernment in individual countries in terms of the citizens’ actual usage of digital public service, and in terms of the public sector’s ability to provide the population with efficient and effective procedures and services. This performance is evaluated on the basis of two indicators: the digitisation, already mentioned above, and the penetration, or the actual use by citizens. The latter is measured on the basis of the EUROSTAT source [2].
From a methodological point of view, the following aspects warrant attention:
– the evaluation of the mystery shoppers, although guided by the checklist and precise criteria, remains subjective, making the methodology not statistically robust;
– the national specificities and, therefore, the actual needs of users in each country are not sufficiently taken into account;
– too much emphasis is placed on the availability of the online service (digital-by-default) and less attention to a multi-channel approach, although recommended by European legislation. In fact, “penetration” only measures the interaction with the PA through the Internet, but does not take into account possible other alternative channels such as public access points (for example, tobacconists or banks) and intermediaries (for ex., CAF) that quite popular and widespread on the Italian territory.
e-Government Benchmark 2017–2018: the performance of the EU
According to the e-Government Benchmark 2017–2018, the EU countries are moving in the right direction: as many as 11 countries provide quality digital services. The top level benchmark on which on average all the EU countries reach the highest score is the User centricity.
eGovernment performance compared to the top-level benchmarks (average 2016 + 2017)
Performance of individual countries compared to life events (2017–2018)
EU performance in individual top-level benchmarks
1. User centricity by indicator and compared to the single life event
Indicators
▪ Online availability
▪ Usability
▪ Mobile friendliness
Conclusions
▪ The top-level User centricity benchmark achieves the 82% score for the EU (bi-annual average 2016–2017)
▪ Out of 10 digital public services, 6 are mobile friendly.
2. Transparency by indicator and compared to the single life event
Indicators
▪ Service provision transparency
▪ Transparency of public organizations
▪ Transparency of personal data
Conclusions
▪ The top-level Transparency benchmark achieves a 59% score for the EU 28+ (bi-annual average 2016–2017)
▪ Only 5 out of 10 services (52%) provide information on timing, progress and service performance
3. Cross-border mobility for indicator and business / citizen life events
Indicators
▪ Online availability
▪ Usability
▪ Electronic Identification (eID)
▪ Electronic Documents (eDocuments)
Conclusions
▪ The top-level benchmark Cross-border mobility reaches a score of 54% for the EU28 (bi-annual average 2016–2017)
▪ Cross-border public services are much more often available for business (72% two-year average) than for citizens (59% bi-annual average)
4. Key enablers: building blocks for user-centric digital public services
For indicator and life events 2017/2016
Indicators
▪ Electronic identification (eID)
▪ Electronic documents (eDocuments)
▪ Authentic sources
▪ Digital post
Conclusions
▪ The top-level Key enablers benchmark achieves a score of 54% for the EU 28+ (bi-annual average 2016–2017)
▪ eDocuments (equivalent in Italy: SPID, CIE) are the most common key enablers (63%)
Overall Conclusions
– The EU progresses in all 4 top-level benchmarks;
– User centricity, availability of cross-border online services and usability have improved significantly;
– The difference between the best performers and the worst performers is progressively reducing;
– The Penetration (actual use) is related mainly to users’ characteristics and generally to each country’s population; the Digitisation is determined by the spread of the broadband roll-out; broadband is in fact the condition enabling the adoption and provision of more sophisticated services and applications.
Italy’s performance in the e-Government Benchmark 2017–2018
From the eGovernment Benchmark 2018 (data average 2016–2017), it emerges that Italy is characterized by a dichotomy.
As for digitisation or the availability of digital public services, Italy with 58% is just below the EU average of 63% (2016–2017).
ANPR, PagoPA e Public Service Design Kits are three Italian good practices. The first two projects are examples of Key enablers while the Public Service Design Kits are an example of the application of the User-centricity principle.
As for the penetration or the effective use of services by citizens, it remains very low, only 22% of individuals interact online with the public administration, compared to the EU average of 53%.
The effectiveness of the digitalisation strategy of public services seems to be strongly influenced by factors such as the use of the Internet and the digital skills of the country.
The improvement in the availability and quality of public services is in fact slowed down by a social context that is marked by a strong digital divide and a lack of interest in the use of digital technologies. In the EU DESI 2018 Italy ranks 25th on 28; the worst performance is in the category of eGovernment users, which sees Italy in last place among the EU countries; it is a result even worse than that recorded for the use of other online services.
Furthermore, the latest EU figures for Internet use still show a low percentage, 69% compared to the EU average of 81%.
In addition, 23.2% of the Italian population has never used the Internet, compared to a European average of 12.9%.
This lack of interest in the Internet is also confirmed by the Knowledge Report 2018 by ISTAT; the Italian delay in the use of the Internet crosses all age groups, including that of the youngest, and is more marked for the 65–74 year-old class of people (27%) with a lower level of education, while among the 65–74 year-old graduates, regular users are 73.7%. Therefore, a correlation between the level of education and the effective use of the Internet seems evident. Without the reduction of this gap, it is impossible to ensure an inclusive digital transformation.
Mobile friendliness
The aforementioned ISTAT Report 2018, highlights that in Italy, as well as in other countries, the Internet use gap has been progressively reduced by smart-phones which made access much easier for everyone.
The use of mobile devices has increased significantly in recent years; thus, it is necessary to continue to develop mobile solutions for eGovernment services and ensure ease of use and accessibility for all; indeed, digital services which are optimized for smartphones, represent an additional channel of communication especially for people who live in disadvantaged areas, do not have a computer at home or have physical disabilities.
The approach mobile first remains one of the main features of the Digital Transformation Team strategy.
The results begin to be seen. In the e-Government Benchmark 2018, the mobile friendliness of Italian digital services grew significantly with an average of 88%, above the EU average of 69% (2017).
In conclusion, a greater diffusion of digital public services requires a serious intervention to reduce the digital divide that characterises our country.
As it has been appropriately noted, the digital divide is a multidimensional, multidisciplinary and constantly evolving concept, the definition of which must be dynamic and flexible, since it should adapt to and absorb the continuous changes and numerous innovative opportunities which are offered by the enabling technologies for the digitisation.
Several countries have adopted strategies (OECD Digital Economy Outlook 2017, pp. 60–72) both for the reduction of this moving target and the development of digital skills which are aimed at various segments of the population and at different levels of specialization: from general skills to specific training in emerging technologies.
Helping the Italian population to develop the necessary skills is a necessary condition to make them an active part in the process of the digital transformation of society and the economy; at the same time, it is essential to give citizens a strong motivation to go online through a range of digital services that are simple to use and meet citizens’ real needs: initiatives such as IO or Designers Italia go in this direction.
Giving continuity to the present digitisation strategy is essential. But it is not enough.
The challenge remains very complex and requires a triple action aimed at increasing the quality of digital services, fostering the dissemination of adequate digital skills in the public administration as a whole and among citizens, and finally, ensuring the complete rollout of broadband infrastructure.
[1] The mystery shopper is an independent evaluator who, incognito, tests and verifies the quality of products and services; the methodology is used in different sectors from sales to banking, insurance, etc.
[2] The Penetration is measured by EUROSTAT see https://digital-agenda-data.eu/ Indicator: individuals (aged 16–74) submitting completed forms to public authorities, over the internet, last 12 months; Source: Eurostat, (Community Survey on ICT Usage in Households and by Individuals- Module C: Use of e-government): Individuals who used the internet for interaction with public authorities; Unit of measure: Percentage of individuals who used the Internet within the last year. See also eGovernment Benchmark 2018, p. 101 and note 12.
Join Us| Follow us on Twitter and LinkedIn | Visit our Website | GitHub
