Global Goals, AI Solutions

Unlocking Insights from the Voluntary National Review Process Using Machine Learning

Data scientists, statisticians, policy makers, and stakeholders all love the High Level Political Forum (HLPF) on Sustainable Development for the breadth of information it provides on the status of the Sustainable Development Goals (SDGs). Hundreds of hours of in-person and virtual sessions supplement the literally thousands of pages of documents outlining the progress and challenges facing the Global Goals.

How do experts and on-the-ground practitioners process all this valuable content in a way that allows them to glean important insights without getting bogged down in the data deluge? We propose that recent advancements in Artificial Intelligence may hold the answer.

• Understanding the Data Deluge
• AI Process for VNR Summarization
• Findings
• Identifying Overarching Themes from the VNRs
• What’s the Data on Data?
• Final Thoughts

Understanding the Data Deluge

Last week, our team published a list of the 10 most significant reports from the HLPF, but this is just scratching the surface. In terms of both the time spent presenting and also quantity of material submitted, the Voluntary National Review (VNR) process is by far the most significant element of the forum. In short, VNRs are an opportunity for countries to share the progress and challenges they are facing in achieving the SDGs while allowing these nations the chance to frame their work in their own individual contexts.

In 2023, thirty-eight countries and the European Union presented their VNRs. This resulted in 6,625 pages of information across 175 documents, the longest of which was over 400 pages!

Documents from this year’s presentations as well as past submissions are available on the HLPF’s VNR website.

Wrangling this much content is a nearly impossible task even for the most dedicated student of development data. As we have written about before, challenges related to having too much information is a known problem and experts are increasingly turning to artificial intelligence and machine leaning algorithms to improve the uptake of essential messages from myriad documents.

Our team here at SDGCounting has set out to leverage our experience with AI and familiarity with SDG-related documents to help make sense of the information contained within this year’s VNRs.

AI Process for VNR Summarization

Our team utilized the following process to better understand the content found in the thousands of pages of VNR documents:

• Using automation tools developed in house, our team downloaded all the documents available through the VNR website
• All PDFs were converted to text documents for easier processing
• In cases where documents were available in multiple languages, English was preferred since AI tools have been trained primarily in this language
• Priority was given to “Main Messages” documents since these serve as a platform for countries to highlight their own priorities
• Documents were submitted to OpenAI’s GPT-4 Large Language Model (LLM) with a prompt requesting summary bullets that focus on findings related to data, innovations, statistics, challenges and successes (NOTE: These are our priorities as a team, other’s interests may vary)
• These summarizing bullets were then reviewed and fed back into the LLM asking for themes and specific examples
• Two passes were used to identify both overall thematic areas as well as data specific insights
• Results were formatted into HTML by the LLM to make reading easier
• Final outputs were reviewed for accuracy and specific claims were fact checked to ensure “hallucinations” were not present
• All code was written in the Wolfram Language utilizing Mathematica software and accessing the OpenAI models through their API protocols
• The entire process was coded in roughly 30 minutes and processing the data required about 90 minutes

Findings

Below is a summary of the main insights uncovered through our process. Content generated by AI has been notated below with only minor human edits to reduce redundancy.

Identifying Overarching Themes from the VNRs

The following themes were found to permeate the 39 VNRs submitted to the HLPF in 2023:

*** START OF AI GENERATED CONTENT ***

Commitment to Sustainable Development Goals (SDGs)
A common theme that cuts across all VNRs is the unwavering commitment to the SDGs. Countries have shown strong dedication towards the integration and realization of these globally agreed development goals. Some have even gone the extra mile by creating ministries dedicated to sustainable development or aligning national development strategies with the 2030 Agenda.

Impact of Global Challenges: COVID-19 and Others
The reviews reflect that global challenges, primarily the COVID-19 pandemic, have put a dent in the steady progress toward accomplishing the SDGs in most countries. The health crisis has caused significant social and economic repercussions, impacting nearly every sector. However, some countries, used their VNRs to showcased resiliency and development resulting from the challenge.

Environmental Sustainability and Climate Action
Most countries are cognizant of the urgent need to address environmental concerns and climate change. They have incorporated strategies that focus on reducing greenhouse gas emissions, achieving carbon neutrality, maintaining forest coverage, and increasing reliance on renewable energy sources.

Poverty Reduction, Social Inclusion, and Equality
Countries have highlighted their continuous strides towards promoting social equality, reducing poverty, and enhancing inclusion. This includes efforts to improve social protection systems, provide universal health coverage, promote youth employment, and champion gender equality, contributing significantly to the eradication of poverty and promotion of social inclusion.

Quality Education and Gender Equality
Countries have made achievements concerning quality education and gender equality. With increased rates of secondary and post-secondary education and investment in the health and education sectors, countries are making headway in these critical areas.

Economic Diversification and Growth
A primary pursuit of various countries such as Bahrain, Cambodia, Mongolia, Saudi Arabia, and Saint Kitts and Nevis, is economic diversification and growth. The focus is to reduce dependency on single-sector economies or commodities and expand into more sustainable sectors. Strategies encompass recovery plans, finance reforms, and development visions.

Challenges and Room for Improvement
Nearly all countries highlighted formidable challenges and areas to improve in their pursuit of SDGs. These range from securing transformative investments, health inequalities, data collection for SDG tracking, to socioeconomic issues for vulnerable populations.

Stakeholder Engagement, Partnerships, and Collaboration
The importance of stakeholder engagement, partnerships, and collaboration is evident in the VNRs. Countries value comprehensive cooperation at the local, national, and international levels. They recognize that succeeding in the journey to SDG realization requires the collective effort of government institutions, NGOs, civil societies, private sectors, and the broader international community.

Advancements in Infrastructure and Services
Improvements in infrastructure and public services remain a central theme in the VNRs. With advancements in sectors like transportation systems and increased prioritization of infrastructure investments, countries are setting the stage for robust economic growth and development.

Adoption of Digital Technology
The reviews also indicate the increased reliance on digital technology, with countries like Tanzania and Zambia leveraging technological innovations in fields like tax collection systems and land management. This underscores the vital role played by technology in accelerating progress towards the SDGs.

International Cooperation and Solidarity
The recurring appeal for international cooperation and solidarity is evident throughout the VNRs. Countries like the European Union, France, Liechtenstein, Saint Kitts and Nevis and Singapore put forth the importance of a unified global response to the unique challenges facing humanity.

Conclusion
Several key themes have emerged from the Voluntary National Reviews (VNRs) of progress towards the Sustainable Development Goals (SDGs). Despite facing multifaceted challenges, countries remain resolved and committed to achieving these all-encompassing and globally significant development goals. Their commitment, collaboration, innovation, resilience, and shared vision of a sustainable future, as reflected through the VNRs, provide much-needed optimism for the journey ahead towards realizing the 2030 Agenda.

*** END OF AI GENERATED CONTENT ***

What’s the Data on Data?

Because our team’s focus is primarily on how the SDGs are being counted, tracked, and information shared, we asked the LLM to identify information contained in the VNRs that was specifically related to the acquisition, use, and sharing of data. Here are the results:

*** START OF AI GENERATED CONTENT ***

The Impact of COVID-19
The global pandemic has had a profound influence on the progress of SDGs. Countries reported substantial challenges due to the pandemic which, coupled with geopolitical tensions, has significantly slowed down data gathering and processing. Some countries highlighted the importance of public healthcare in the context of the pandemic, further underscoring the crucial role of health data in understanding and addressing the crisis.

Environmental Concerns and Climate Action
Countries like Bahrain, Guyana, France, Iceland, and Poland have a strong focus on tackling climate change and promoting environmental sustainability. Data is being extensively used in monitoring emissions, tracking progress towards environmental targets, and formulating strategies for climate action.

Poverty Reduction, Social Protection, and Inclusive Development
Reducing poverty and promoting inclusivity are significant objectives for countries like Barbados, Portugal, Romania, Saudi Arabia, and Comoros. By leveraging data on socio-economic indicators, these nations are tailoring strategies to meet the specific needs of diverse populations.

Economic Diversification and Stability
Countries like Mongolia, Saudi Arabia, and Saint Kitts & Nevis are prioritizing economic diversification. Similarly, Zambia and Timor-Leste have embarked on ambitious programs of economic restructuring, using financial data to inform efforts to reduce dependence on specific sectors. In a clear illustration of data-led economic planning, Zambia is using robust financial metrics to guide its IMF-backed debt restructuring process.

Data Availability and Utilization
Data availability and utilization forms a critical part of the VNRs, with countries making notable strides in this area. For example, Iceland has increased its data coverage to 70% of the UN SDG indicators, demonstrating the power of comprehensive data coverage in assessing the progress towards SDGs. Similarly, robust datasets have enabled Fiji to track its progress more accurately, reinforcing the significance of reliable and high-quality data.

Partnerships, Support, and International Cooperation
Partnerships and international cooperation are viewed as essential components of SDG implementation. Nations are actively promoting collaboration at various levels. Collaboration extends to data sharing as well; countries are making concerted efforts to share data and insights, thereby strengthening global efforts toward achieving SDGs.

Data Challenges and Innovations
While data plays an indispensable role, substantial challenges remain. For example, Barbados grapples with data collection issues that hinder its measurement of SDG progress. Similarly, Canada and other nations face difficulties in obtaining granular, disaggregated data to address the specific needs of different communities. Despite challenges, innovations are burgeoning, such as Zambia and Tanzania leveraging digital technology to enhance their economies and public services. These advances signal a positive trend in leveraging data to drive sustainable development, promising an exciting future marked by data-led insights and decision-making.

Overall, the Voluntary National Reviews underscore the importance of data in achieving Sustainable Development Goals, be it in guiding strategies, tracking progress, or catalyzing innovations. As we move forward, the symbiosis of data and sustainable development promises to yield profound benefits for societies worldwide.

*** END OF AI GENERATED CONTENT ***

Final Thoughts

Without doubt, the best way to understand the progress and setbacks of individual countries in their implementation of the SDGs is to spend the time reading and understanding the detailed information available in their Voluntary National Reviews. These documents were created over many months by some of the best data experts in the world and the full insights available should not be discounted. However, in a situation where key conclusions may be overlooked due the sheer quantity of information available, we believe that AI may hold the key to unlocking important conclusions. In particular, tools like OpenAI’s Large Language Models have the ability to hold and process significantly more discrete points of data than any human can and thus can offer summaries not susceptible to the biases and limitations of a human reader processing thousands of pages of content.

We understand that the findings presented above do not reflect the entirety of important information presented at this year’s HLPF, but we do believe it is a stepping stone to learning more and identifying areas worth further investigation. Additionally, once initial processing of these documents is complete, it is possible to tailor individualized prompts to identify more specific information. For instance, if a stakeholder group is interested in progress around gender equality, it would be possible to ask the LLM to only return references on this topic and then find the corresponding original sources to back up the conclusions.

In addition to the automated process outlined above, our team spent hours reviewing actual VNR documents and listening to presentations from the HLPF. In our opinion, the LLM tools were able to capture the highlights of the content even if some of the nuance found in the full documents was lacking. The AI findings were faithful to the documents analyzed, but we recognize additional fact-checking and verification is necessary when using these tools.

AI and Machine Learning is not a replacement for the expertise of data scientists, statistician, and stakeholders. If anything, these new tools point to the increased importance of these individuals in gathering and analyzing data. However, AI promises to be a useful tool in understanding vast quantities of information and will only get better as people continue to explore its implementation.

If you are interested in discussing our methodology or findings, you can reach us at info@sdgcounting.info.

Disclaimer: SDGCounting utilizes AI tools such as GPT in the planning, research, and drafting process. This article was entirely written by hand except for content explicitly identified as AI generated.

SDGCounting is a program of StartingUpGood and tracks the progress of counting and measuring the success of the SDGs. Check us out on Twitter.