India needs to focus on Gender-disaggregated data to reduce the gender data gaps

The pursuit of gender equality continues to be hampered by a lack of reliable, comprehensive gender data. The United Nations states that 44% of the necessary data to monitor 51 gender-related Sustainable Development Goal (SDG) indicators, including SDG 5 on gender equality and women’s empowerment are incomplete or lacking. UN Women has projected that it will take 22 years to close the SDG gender data gap, making it difficult to assess and monitor the direction and pace of progress for women and girls.

Gaps in gender data and the lack of trend data make it difficult to monitor progress for women and girls, which in turn result in ineffective policy reform and budgetary allocations. Unless gender data is mainstreamed into national statistical strategies and prioritised in data collection, the scarcity and gaps will persist.

Investment in national statistical capacity is central to improving the coverage, quality and timeliness of data for monitoring gender equality and the SDGs targets. But beyond this, making sure that data represent the lived reality of women and girls in all their diversity by addressing deep-seated biases in definitions, classifications and methodologies, is essential to making women and girls visible.

Missing Numbers

“Women and Men in India 2023” report by the Government of India’s Ministry of Statistics and Program Implementation (MoSPI) showcases consolidating data from multiple sectors to provide insights into gender disparities in areas like health, education and employment. The report provides critical insights into persistent gender data gaps across key areas.

For instance, when viewing economic participation of women, particularly in the informal and unorganised sectors, where women’s work is often unaccounted for. The report highlights a disparity in labour force participation, with women making up only 19.2% of the workforce. However, it lacks comprehensive data on the nature and quality of women’s employment, especially in sectors where they are over represented but undercounted.

Another critical gap can be seen when assessing asset ownership, where data on women’s control over property and land remains sparse, limiting understanding of economic empowerment. Similarly, the report provides limited information on women’s participation in areas like sports, media, and skill development, leaving gaps in understanding their representation and progress in these fields.

The MoSPI report itself acknowledges these gaps and underscores the need for gender-disaggregated data in sectors like poverty, where the Poverty Head Count Ratio is available only at the household level and not broken down by gender, obscuring the real impact on women.

Existing Efforts to Bridge the Gap

In India, the government has developed gender-responsive budgeting, which integrates gender data considerations into budget processes. Through the Gender Budget Statement (GBS), the government tracks budgetary allocations aimed at women-centric schemes across ministries. This system not only highlights the allocation of resources but also drives the need for gender-disaggregated data to monitor the impact of these schemes.

Government implementing agencies and ministries are encouraged to generate gender data to assess whether the funds being allocated for gender development and empowerment are achieving the intended outcomes. For example schemes and funding to improve women’s health, access to quality education, and economic participation. Sectors like health and education, where gender-sensitive interventions are critical, now we see an increased reliance on data that measures both access and outcomes for women and men.

Furthermore, initiatives like the National Data Bank, focus on data related to marginalised groups, ensuring intersectional gender data collection. These reforms aim to improve the availability, quality and use of gender data for evidence-based policymaking and monitoring gender equality.

But gender data gaps persist. CivicDataLab’s work in strengthening the data for public good ecosystem has shown us that data collaboratives would help enhance evidence based decision making processes. Below, we share some insights on how best to improve and enhance gender disggregated data availability.

Gender Data Collaborative can address the persistent gender disparities and enhance the effectiveness of policies

Multi-stakeholder partnerships play an important role in facilitating dialogue and data collaboratives. One such example is the Women’s Work and Employment Partnership (WWEP), a collaborative initiative led by the World Bank, ILO, and FAO, with support from Data2X and the Hewlett Foundation. This partnership drives inter-agency efforts to apply updated international definitions of work and employment.

Improving gender data availability requires a concerted effort from diverse stakeholders — government bodies, civil society, academia, businesses, startups, and volunteer groups — to securely share and exchange gender-sensitive data in a standardised and interoperable way.

Collaboration maximises the potential of reusing data for gender-inclusive development, allowing planners to address the specific needs of women and marginalised genders.

Beyond data-sharing, regularly convening stakeholders for ideation, capacity building, and institutionalising gender-responsive data practices is essential to fostering a more participatory and equitable approach to gender data.

With just a few years short of of the 2030 Agenda for Sustainable Development, the world is failing to achieve gender equality, making it an increasingly distant goal. According to Sustainability Goals, only about 29% of the gender data needed to monitor the Sustainable Development Goals (SDGs) are available in India, leaving critical gaps in understanding women’s and girls’ lived realities across various sectors.

This makes a gender data collaborative in India a crucial initiative for addressing key data gaps and deepening insights into gender disparities across labour force participation, health, education, digital access, and policy impact through gender budgeting.

Such a collaborative would enhance targeted interventions by strengthening gender data monitoring across sectors, fostering data-sharing mechanisms, and driving participatory engagement. By effectively leveraging data, this initiative can play a transformative role in advancing gender equality and empowering women and marginalized communities nationwide.

This collaborative can maximize collective impact, streamline resources, and drive innovation. Contributors can identify gender use cases which can be share insights,align efforts to address shared challenges in the country and ensure efficient capacity building. Key aspects of this can focus communication and knowledge exchange, developing guidelines for collaboration, and jointly advocating for gender data’s critical role in achieving gender equality. Regular engagement and shared lessons will help partners collectively strategize, integrate new initiatives, and amplify their impact.

Using Data as a core medium to inform gender data Reforms and engage stakeholders

At CivicDataLab we have been working actively to institutionalise evidence-based decision making through data. Highlighting gender data with diverse topics like education, government budgets and health, we have attempted to capture and analyse crucial datasets and use cases as an attempt to address the gender data gap.

Listed below are a few of our work examples of using data as a core medium to highlight gender gaps to inform data driven decision making processes.

Strengthening fiscal accountability through leveraging data to monitor gender impact in education

In India, despite education being a fundamental right, significant challenges remain in ensuring quality and access to education for girls. According to Census 2011, 1 in 4 school-age children are out of school, and dropout rates are higher among girls due to factors like early marriage (20% of girls aged 15–19 are married), domestic responsibilities, and inadequate infrastructure. While the government has achieved universal enrollment at the primary level, dropout rates rise to 17% at the secondary level, especially for girls. CivicDataLab, in partnership with HAQ: Centre for Child Rights, is addressing these challenges through the Girl Education Spending Tracker in the states of Uttar Pradesh, Bihar and Assam. The project tracks government fund allocation, aims to improve budget transparency, and advocates for better financial accountability in education, especially for girls.

By analysing data from various government departments, the initiative sought to bridge gaps in education funding, improve policy implementation, and strengthen grassroots organisations’ capacity to advocate for better educational outcomes.

Tracking fund allocations and utilisation towards Gender-responsive initiatives in India through intuitive budget data explorers

CivicDataLab along with Center for Budgets and Governance Accountatibility (CBGA) developed OpenBudgetsIndia (OBI). This open data platform is advancing gender equity by integrating gender-sensitive data into its Budget Portal, enabling a more comprehensive understanding of how public funds are supporting gender-responsive initiatives.

Through the Gender Budget Statements available on the portal, stakeholders can examine funds allocated specifically for programs aimed at improving the socioeconomic status of women and other gender minorities.

By presenting data categorised by gender-focused schemes, and gender major heads of expenditure at the national and state levels OBI helps users analyse how effectively government resources are being directed towards bridging gender disparities in areas like healthcare, education, employment, and social protection.

Accessibility to gender-disaggregated budget data empowers policymakers as well as other stakeholders like researchers and civil society organisations to track funding trends, advocate for resource allocation aligned with gender needs, and hold decision-makers accountable for equitable public spending.

The data can be used for increasing citizen participation and also improves government accountability, as the dashboard reveals spending patterns and outcomes, helping stakeholders understand if public funds are addressing the needs of gender and marginalised communities.

Developing collaborative Gender Budgeting for the states in India through a participatory approach

India has repeatedly pitched gender budgeting as a powerful fiscal innovation for women’s empowerment. Its experience has widely been used as a frame of reference for other gender budgetary control initiatives with several states such as Odisha, Gujarat, Uttar Pradesh and Tripura beginning to mainstream gender into their budgetary commitments as early as 2005–07

CivicDataLab, in collaboration with UNICEF, has worked to make government budgets in Assam more reflective of and responsive to the needs of women and transgender persons. This collaboration emphasises gender mainstreaming at every stage of the policy process — planning, budgeting, and implementation — to ensure that gender perspectives translate into concrete budgetary actions.

Through participatory gender budgeting, the initiative integrates the voices of women, transgender persons, and other marginalized groups into the budget cycle, addressing their specific challenges and priorities. It begins with identifying gender-related issues through consultations and gender-disaggregated data analysis, followed by collaborative policy design that aligns with development goals.

The process includes preparing a Gender Budget Statement (GBS), categorizing allocations for women-specific or gender-inclusive schemes, and ensuring effective implementation through capacity building and community involvement. Regular monitoring, participatory audits, and feedback loops are used to assess impact and refine policies. This joint effort fosters equitable and inclusive public spending, advancing gender equality and accountability in resource allocation.

The inclusive budget cell within the Department of Finance, a dedicated unit towards making the state budget more inclusive for the vulnerable sections of the society, had introduced its first gender budget in FY 2017–18. 24 departments reported allocations for Gender budget in 2024–25. The departments reporting budgetary provisions for gender empowerment in FY 2024–25 have been segregated into five sectors: Social, Economic, Infrastructural, Science and technology and Others. Sectoral distribution of gender budget illustrates important trends with the social sector receiving 61% of the allocation, Economic sector receiving 33% and Infrastructure sector with around 7% of the distribution.

Using Data Analysis techniques to identify gaps in access to maternal and child health services

CivicDataLab used procurement data, health data and geospatial data to review public health spending in Assam, India. We developed a method to verify if the money is reaching districts where mothers and babies need it most. Their initial findings suggest it isn’t, as fewer maternal and child health tenders are published for districts with high maternal and infant mortality rates, but more complete data would allow them to draw more accurate conclusions.

CivicDataLab’s analysis revealed crucial gender disparities in the allocation of maternal and child health resources across Assam, underscoring the pressing need for a gender-sensitive approach to health funding. By examining procurement, health, and geospatial data, they identified that while Assam published around 165 tenders specifically targeting maternal and child health over five years — totalling approximately INR 360 million ($4.7 million) — these tenders constituted only 5% of the state’s health-related tenders. Yet, districts with the highest maternal and infant mortality rates, such as Cachar and Karimganj, received disproportionately fewer maternal and child health tenders. This inequitable allocation highlights the need for a more strategic focus on women’s and children’s health to close critical gaps in care for vulnerable populations.

Through a gendered data lens, CivicDataLab found stark contrasts in health outcomes that disproportionately affect women and children in marginalized regions. In certain central and upper Assam districts, infant mortality rates were nearly double those in districts where tenders were more concentrated. Areas with higher tender activity, such as Kamrup and Goalpara, showed comparatively better health outcomes. In contrast, regions like Karbi Anglong and Dima Hasao — where women and children face compounded vulnerabilities due to geographic and socio-economic isolation — suffered from both high infant mortality and low access to health tenders. CivicDataLab’s findings stress the importance of gender-responsive budgeting and timely data publication, noting that only 20% of Assam’s actual contracting data is currently available. Increasing data availability and standardizing health-related tender processes could enable more equitable resource distribution, ensuring that health funding better addresses the needs of women and children in underserved areas.

In conclusion, bridging the gender data gaps in India is essential for achieving gender equality and empowering women and marginalised communities. The persistent lack of reliable and comprehensive gender-disaggregated data hampers the ability to monitor progress and design effective interventions. A Gender Data Collaborative offers a transformative solution, bringing together diverse stakeholders — governments, civil society, academia, businesses, and international organisations — to build a robust, interoperable system for gender-sensitive data collection, sharing, and analysis. Such a collaborative would enable evidence-based policymaking, enhance accountability, and ensure that interventions are tailored to address the lived realities of women and marginalised genders. By institutionalising gender-responsive data practices, India can strengthen its efforts toward achieving the Sustainable Development Goals, creating a future where equality is informed, measurable, and actionable.

The need to strengthen Gender Data Collaborative to reduce the Gender Data Gap was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

Creating and ensuring a closed feedback loop that includes all stakeholders is key for a long lasting impact of any interventions built to strengthen the data for public good ecosystem. Feedback from stakeholders is essential for identifying issues or bottlenecks regarding data-driven interventions on a near real time basis.

Onboarding sessions help gather local knowledge about nuances of climate vulnerable geographies and populations, which help customise Climate Action interventions to the region’s unique needs as well as making such interventions more robust and accurate. Such sessions and feedback mechanisms play a crucial role in the success of data-driven initiatives. Keeping this in mind, in October we conducted interactive onboarding sessions with different stakeholders and partners including government officials in the state of Assam and Himachal Pradesh.

The onboarding sessions and capacity building exercises aimed at sharing our collaborative Intelligent Data Solution for Disaster Risk Reduction (IDS-DRR) with key stakeholders and main target users. During this enaggement inputs and feed back was sought regarding the user journey as well as the ease of use of the platform.

CivicDataLab is developing this solution in collaboration with Open Contracting Partnership with the support by The Rockefeller Foundation and The Patrick J. McGovern Foundation. IDS-DRR helps strengthen, prioritise and map public expenditure and resource mobilisation by government agencies for long term resilience. It is informed by the #SENDAI framework and leverages data and technology to enhance sustainable climate action.

In our sessions with the Assam State Disaster Management Authority (ASDMA), more than 225 state government officials from across administrative divisions participated. The demonstration cum workshop was designed to introduce stakeholders (like district officers, field officers among others) to the IDS-DRR platform.

Engaging Decision Makers for Enhanced Impact

CivicDataLab’s collaborative approach has shown us that on-boarding sessions help the authorities understand the goal, scope and expected outcomes for a data-driven intervention. And also ensures that the data led intervention aligns with local priorities. Through proper understanding and usage of the platform, authorities are able to respond more efficiently, allocate resources effectively and build resilience and sustainability.

We work with authorities to foster understanding, build capacity of officers to use these systems, new tools and technologies for effective climate action. Feedback and communication with local officers helpful in finding loopholes and also helps understand the stakeholders’ perspective informing and making the intervention more effective and deployable.

During onboarding, training is provided on technical aspects of using the platform including analysing data insights among other features offered. This ensures that all participants are equipped with the knowledge and tools to efficiently use/ deploy the data-driven approach.

Through such onboarding sessions we have learnt that district officials, who have firsthand knowledge of the region’s flood history, vulnerable populations, critical infrastructure and administrative aspects. We are able to incorporate these local insights and match local inputs with the data-drive platforms to ensure accuracy.

Additionally, we have found it helps highlight the need for community participation and citizen engagement in policy reform. On-boarding sessions help showcase and emphasise how community involvement and participation can increase resilience.

Highlights from the Assam Sessions

During the session district and state level officers were shown the specific objectives of the IDS-DRR platform and the tender dashboard to view and analyse data related to reducing flood risk, improving resource allocation, data management and enhancing disaster preparedness. We aimed at helping participants get familiarised with and understand the expected outcomes and the impact that can be achieved through this solution.

Officers were tasked with conducting risk and vulnerability assessments of their districts and identifying areas at greater risk of flooding from the analysis available on the IDS-DRR platform. This approach has helped us derive better understanding and ground truthing of our solution. We also introduced the tender dashboard, especially analysis related to specific districts and all revenue circles of those districts to showcase how officers can use this dashboard for garnering insights on resource mobilisation.

IDS-DRR platform gives access to dataset on Hazard, Exposure, Vulnerability, Government response of district which helpful in response, recovery and resource allocation. It helps officers in identification of key flood risks specific to the district, such as vulnerable population, critical public infrastructure, areas prone to flooding, and government response.

Feedback from participants

The major takeaways from the on-boarding session of IDS-DRR with districts of Assam revolve around key aspects of the platform’s goals, strategies, data management, capacity-building and coordination mechanisms.

Project officer District Disaster Management Authority (DDMA), Dibrugarh said, “The platform will definitely help and will save time but some data may be missed out, which need to be fixed for better usage of the platform.”

Field Officer DDMA Kokrajhar said, “In terms of district level, the tender dashboard is insightful. Village level information should be included that would be helpful in disaster mitigation projects. Also, helpful in checking for tender funded or not and for resource allocation. Will be helpful in decision making — go for a better plan for future — effective disaster management plan. Beneficial for the public also to check their risk in the areas and government response in their district. The dashboard is helpful for checking which areas require more funding.”

At CivicDataLab, we find such stakeholder engagement helps institutionalise data driven decision making and enhance evidence based strategies. This alignment prevents confusion and promotes unified efforts by the authorities and all other stakeholders improving impact.

CivicDataLab (CDL) works at the intersection of data, technology, design and social science to strengthen access to public information, evidence-based decision-making and citizen participation in governance. We harness the potential of open knowledge movements to strengthen the data-for-public-good ecosystem and enable citizens to engage in matters of public reform. We work closely with governments, non-profits, think-tanks, media houses and universities to enhance their data and technology capacity to better data-driven decision-making at scale.

Our current areas of expertise include digital public goods & infrastructure (DPGs & DPI), climate action, public finance, urban development, open contracting and law & justice. We have co-created digital public goods like open data platforms, data exchanges, data science models and citizen-led apps for improving participatory data-driven governance in India and other countries.

Impact of Data Driven Interventions Must Include Stakeholder Engagement was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

We at CivicDataLab are honoured to be selected by Ministry of Electronics and Information Technology and Digital India Programme to co-create ParakhAI — an open-source framework for Participatory Algorithmic Auditing of AI solutions, especially high-risk algorithmic decision-making systems (ADS). Enabling data driven decision-making processes is key to creating sustainable interventions for inclusive development.

High-risk Algorithmic Decision-making Systems

There is an increasing need for citizens to have more agency over how their data is used in Artificial Intelligence(AI) models, especially the ones deployed to impact lives and livelihoods. We are witnessing citizen-centric AI models now being developed through both public policy interventions and by private entities to deliver essential citizen services like public health, education, banking, e-commerce and others.

These models are also known as High-risk ADS (Algorithmic Decision-making Systems) that directly or indirectly impact benefits, punishments, or opportunities individuals or groups can receive. The risk here being that individuals or groups are harmed by the output of the decision-making system, resulting in incorrect or unfair outcomes.

Most high-risk ADS may undermine the fundamental rights of citizens related to equality, privacy, freedom of speech and expression, and may even lack protection from any potential discrimination on grounds of religion, race, caste, gender or place of birth.

Making Algorithmic Auditing more Participatory & Inclusive

Often ADS may lack participatory auditing frameworks that can help citizens report issues related to discrimination, exclusion, unfair treatment and more potentially due to different types of biases arising from the training data, technical constraints as well as societal or individual biases.

Our proposed framework will enforce the adoption of continuous lifecycle with Public Consultation, Development, Deployment and Assessment as key steps in the lifecycle. This ensures adequate avenues for impact groups to timely and effectively participate, collaborate and co-create the ADS system.

The citizen engagement models for high-risk ADS auditing need to be tailored keeping in mind the diverse nature of Indian citizens and groups, especially looking at the issues related to volume of usage, language, regional diversity, digital literacy, access and usage of internet & smartphones and more.

Citizen participation in ADS needs to be formulated from both formal, which is instituted into governance and legal regulation as well as the informal modelling that is without legally-binding engagement requirements.

ParakhAI will provide framework and Chatbot for Participatory Algorithmic Auditing

Thus, with this research project, we aim to develop a comprehensive open-source framework for Participatory Algorithmic Auditing — ParakhAI in India to help individual citizens and citizen groups have more agency to audit High-risk ADS.

The proposed framework will enable active citizen involvement and responsible design while developing or deploying ADS. It will help increase awareness and participation of citizen groups, training for High-Risk ADS deployed in India to audit and report biases and risks. And will help both public and private agencies developing ADS to ensure administrative decisions are made in a more fair, efficient, transparent and accountable manner.

In coming months, we will host a series of participatory consultations and focus group discussions with diverse stakeholders — AI practitioners, policymakers, researchers, academia, businesses and other relevant stakeholder to understand various risks with high-risk algorithms. We will also explore pathways to set up a participatory auditing framework for High-risk Algorithmic Decision-making Systems.

The framework will be followed with the development of a Participatory Algorithmic Auditing Chatbot (working in English, Hindi & other select local languages) to increase awareness and participation of Citizen Groups training for High-Risk ADS deployed in India to audit and report various biases and risks.

References

• Beyond the “Black Box”: Enabling Meaningful Transparency of Algorithmic Decision-Making Systems through Public Registers — By Dr. Maya Murad https://dspace.mit.edu/handle/1721.1/139092
• Power to the People? Opportunities and Challenges for Participatory AI. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO ‘22). Association for Computing Machinery, New York, NY, USA, Article 6, 1–8. https://doi.org/10.1145/3551624.3555290 by Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed — 2022
• Localising AI for crisis response: Putting power back in the hands of frontline humanitarians and local communities. https://www.nesta.org.uk/report/localising-ai/

ParakhAI — An open-source framework for Participatory Algorithmic Auditing was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

Indian cities must leverage Open Data and citizen engagement to strengthen participatory urban development

It is estimated that close to 1/3rd of India’s population contributes 2/3rd of the country’s GDP and an increasing number of people are migrating to cities and urban towns for better economic opportunities. This rapid urbanisation in India is likely to further intensify, with over half of the country’s population expected to live in cities by 2050. Our cities are undergoing a rapid digital transformation, making more data available for understanding urban growth, infrastructure and public service delivery.

The Government of India has initiated various urban reforms for our cities like — Atal Mission for Rejuvenation and Urban Transformation 2.0 (AMRUT 2.0), Smart Cities Mission (SCM), Swachh Bharat Mission 2.0 (SBM-Urban 2.0) and development of Urban & Industrial Corridors. To help empower cities with data, the Ministry of Housing & Urban Affairs(MoHUA) has also launched various data initiatives including — DataSmart Cities Strategy, Data Maturity Assessment Framework, Smart Cities Open Data Portal, City Innovation Exchange, India Urban Data Exchange and the India Urban Observatory.

Unfortunately, most of the urban data remains in silos and capacities for our cities to harness urban data to improve decision-making, strengthen citizen participation continues to be limited. As per the last Data Maturity Assessment Framework (DMAF) assessment conducted in November 2020 by MoHUA, among 100 smart cities only 45 cities have drafted/ approved their City Data Policies with just 32 cities having a dedicated data budget in 2020–21 for data-related activities. Moreover, in-terms of fostering data collaborations, only 12 cities formed data alliances to achieve tangible outcomes. We hope smart cities continue this practice by conducting a yearly self-assessment to progress in their journey to harness data for improving their urban planning.

Seeding Urban Data Collaborative to advance City-level Data Engagements

There is a need to bring together a diverse set of stakeholders including governments, civil societies, academia, businesses and startups, volunteer groups and more to share and exchange urban data in a secure, standardised and interoperable manner, deriving more value from re-using data for participatory urban development. Along with improving data sharing among these stakeholders, it is necessary to regularly convene, ideate and conduct capacity building sessions and institutionalise data practices.

Urban Data Collaborative can bring together such diverse stakeholders who could address some of these perennial challenges in the ecosystem while spurring innovation. The rapid urbanisation in India is likely to further intensify, with over half of the country’s population expected to live in cities by 2050. To plan for such growth, our cities need to invest more in improving urban public service delivery, climate-resilient urban infrastructure and better civic-engagement.

Enhancing citizen participation leveraging developments in Data & Artificial Intelligence

Urban Data Collaborative can also help in piloting recent developments in AI-driven data platforms and chatbots, we can move beyond one-way reporting digital solutions to dynamic, to more nuanced two-way information sharing that enables effective citizen participation in governance. With applications like Swachhata, citizens can report issues like potholes or water quality issues to city officials, their inputs can result in near-real time changes in mobilising resources to wards and societies that need more attention. City-level application can also be used to actively seek their inputs on city-level budget information, recent policy developments, city climate action plan and more. For example, Taipei City Government has been using https://join.gov.tw/ for working with citizens to pass participatory budget proposal on a variety of civic issues like — park renovation, environmental restoration, traffic safety, care of the elderly and children.

Globally, we are witnessing a surge in research and pilots to determine how cities leverage large language models for participatory urban planning. But the true potential of AI developments for improving our cities, can only be realised when our cities double-up on their commitment to publish high-value open data and actively collaborate with more stakeholders. At the same time, we have to ensure apt safeguards for data and AI pilots by deploying provisions for inclusive access, privacy protection and algorithmic auditing.

Harnessing Data for Urban Development

At CivicDataLab we have been working actively to institutionalise evidence based decision making. Here, we share with you a few of our projects, explaining how urban data can be leveraged to enhance citizen participation in cities:

Harnessing Citizen Grievances & Hyper-local Data to improve Participatory Urban Development

Smart City Ahmedabad Development Limited (SCADL) has issued a work contract to CivicDataLab team to help build data capacity of city officials of Ahmedabad on harnessing emerging technologies like data analytics, machine learning and AI to improve resource planning and timely address citizen grievances in the city.

Our team has developed a data solution, which does time-series analysis of citizen grievances collected from various sources to generate weekly and monthly insights for all zones, urban wards and priority neighbourhoods of Ahmedabad.

These data insights help city officials plan better for various urban issues like drainage maintenance during monsoons, road repairs, water resource management and more. The data solution has considerably reduced the time taken to identify, validate and resolve citizen grievances in addition to improving timely urban planning in the last few months.

CogniCIty: Enhancing Citizen-led Disaster Reporting in Asia & Other Countries

CivicDataLab & PetaBencana have worked to customise & deploy CogniCity — an open-source citizen-led disaster reporting platform for hazards like floods, earthquakes, volcanoes, air quality issues & more.This platform creates a unique two-way information sharing mechanism via social media platforms at the time of disasters, currently deployed in Indonesia, Philippines and Panama, enabling more than 23,000 citizens to report disasters.

These reports are shown as a web-based, publicly accessible, real-time map collating both citizen inputs on neighbourhood conditions and government advisory information in the affected regions. We are now working to expand CogniCity deployments in more Asian countries and other regions in the global south.

OpenCity: Crowdsourcing City-level Data to enhance Civic Participation in our Cities

OpenCity is a city level data platform hosted and developed by Oorvani Foundation and Data Meet. CivicDataLab has developed recent version of the platform OpenCity focussing on crowdsourcing city-level data and its uptake among data communities.

The platform is for citizens and civil society to help bring visibility and transparency into local governance. Currently, there are 500+ datasets and 1300+ documents hosted on the platform and continuously growing covering cities like Bengaluru, Chennai etc. On average, there are over 60K+ users using the platform on a monthly basis.

We look forward to collaborating with more stakeholders to strengthen Urban Data Ecosystem in India and beyond.

Need for Co-creating Urban Data Collaboratives was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

To reduce the mighty budget to just a buyer’s guide is a grave fallacy, even for common citizens who do not study public policy.

Allocation, expenditure and revenue by governments forms the back-bone of public policy — decisions that impact citizens as tax-payers as well as individuals both directly and indirectly. However, we rarely view the union or state budget documents as anything more than a retail or tax guide informing our decisions.

Government Budgets can be defined as a moral document since they deal with promises and decisions that can have profound moral implications. Budgets are also called a promise document as they represent the government’s promises to the people. It is a financial instrument that represents the political and financial priorities of the government. One may disagree and view the budget only as a collection of numbers.

Policies, schemes, ministerial allocations, expenditure prioritisation and even revenue collection impact our lives beyond purchasing power. Understanding and participating in or with the budgetary processes often appears difficult and complex.

But to formulate and express popular will of the people, citizens need adequate means of information about the functioning and priorities of the government itself. Budgets represent perhaps the most important and accessible medium of such information. Citizens can utilise the datasets and initiatives like our collaborative Open Budgets India (OBI), co-created with Centre for Budget and Governance Accountability (CBGA). This platform helps inform public representatives, bureaucrats and citizens enabling participatory policy making processes.

Do you know how has Tree Cover Changed in your city?

Complexities of Budget Documents

Government Budget documents are large, complex and often without the necessary context and accessibility required to understand such a large set of numbers. The Finance Minister’s speech, the finance bill, the outcome budget, the ministry-wise demand for grants are all critical for understanding the process and product. The collection of all these documents and features are colloquially referred to as the Budget.

The union Budget, along with its myriad of supplemental documents, talks about what is going to be taxed more and less, hence determining market costs of a variety of goods and services. That’s why it is not surprising that for many, the introduction and end to the budget is the review of what all they have to pay more for and what is cheaper to buy now. By no means are they wrong.

However, everyday issues of citizens, ranging from lack of funds in hospitals, urban flooding in cities in monsoon, to the number of child care centres in your city — all aspects can be traced to government budgets. The Union Budget of India, is complemented by some other key procedures and documents including the annual consultations — where the government seeks inputs from the general public, institutes, agencies and other stakeholders to inform the decisions outlined in the document.

How much is your state spending from Nirbhaya Fund?

Impact of Government Spending on Citizens

To understand the impact of government spending on daily lives of citizens, budgets serve as a key. Here, we highlight some of the key points of the 2024 Union Budget to help understand the technical terms, topics and their potential impact on citizens:

• In 2024–25, the fiscal deficit estimate was lowered to 4.9 per cent of GDP from 5.1 per cent in the 2024–25 Interim Budget. The fiscal deficit is simply the gap between how much the government earned and how much it spent (expenditure). One can think of it as the loan government takes to manage the gap. But just like individuals have to pay interest on loans, so do governments and since higher fiscal deficit means governments spend more on interest, it is left with less money to spend on other important things and therefore lower fiscal deficit in the current budget is a positive development for the nation’s economic environment.

• The share of Revenue Receipts to Total Receipts remains lower than the peak of FY 16–17. The government receives (and hence called receipts) money through mainly two sources, Revenue receipts and Capital receipts. Together these two are known as total receipts. Revenue receipts include both tax (income, corporate, etc) and non-tax revenues. In 2016, the share of revenue receipts was 70 per cent of the total receipts, this started to decline since then. It has started to increase post pandemic, but remains lower (65 per cent) than FY 2018–19 levels of 67 per cent. This leads to higher dependency on capital receipts (that may cause liabilities or reduction in public assets). So, the phenomenon of governments being unable to collect greater revenue through taxes means they may have to sometimes divest public sector enterprises or assets to raise the required funds.
• Government revenue from income tax is now 19 per cent, higher than the 17 per cent contribution from corporations. This indicates a higher share of tax collected from individuals as compared to companies / organisations. Ideally, since most corporations generate more revenue than individuals, the taxation share should be higher for corporations as compared to individuals. In an ideal system common individual citizens might pay lower taxes than corporations.

What the Budget Does

The budget describes what commitments will receive focus and which ones remain unfulfilled. The budget numbers also indicate how the public expenditure (government spending) varies year by year across key sectors like health, education, sanitation, women safety among others.

The budget determines how many new schools a district will get or how many new houses might be built by the government. The budget decides which ministry (or department) receives how much funds (allocation) for specific schemes (Jal Jeevan, Swach Bharat, MGNREGA, etc), and how has its utilisation been in the last financial year.

It determines the efficacy of individual ministries and departments and their schemes; a reduced budget often means reduced scope of a scheme. Budgets impact the qualitative features of the economy — best represented by the stock market swings pre and post the budget announcements.

These are just a few of the points that become clear, post going through the budget, with necessary context. The budget, government spending, the associated jargon and its implications are often hard to understand. But they nevertheless remain important to understand the government’s focus areas and its impact on citizens.

Participating for Reform

It is crucial that citizens regularly engage and participate in the budgetary process, as the budget determines so much of every individual’s lives directly. Organisations like PRS Legislative Research, Civis, Maadhyam, and CivicDataLab are working to increase citizen participation through various initiatives and interventions.

CivicDataLab has worked with the Government of Assam to pilot a chatbot that enables citizens to provide suggestions for their state budget. We have also tried to improve citizen participation through dissemination initiatives such as #DataDekho. Citizen engagement is important for understanding key sectors as well, and Justice Hub has been one of the key collaborative initiatives looking at the law and justice budget, especially since the everyday intersection of the legal system with citizen lives.

To address the challenges of lack of resources to understand budget data, CivicDataLab, in collaboration with their technical partner, the Centre for Budget and Governance Accountability (CBGA) has developed the Union Budget explorer, as an important component of the open data platform, Open Budgets India (OBI). The explorer provides both an overview of the government’s expenditures and receipts from various sources, as well as expenditure of ministries and departments, across years. It also provides the outlays on major schemes and year-wise union receipts. These openly accessible (and downloadable) datasets can help in understanding budgets better, providing much needed context to the announcements by government, and bring them within the reach of common citizens.

Who are Government Budgets for… was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

Evidence based interventions for Climate Action and disaster risk reduction (DRR) must engage communities to ensure resilience.

In the fight against climate change and disaster risks, communities are crucial players. The role of communities increases even more so in vulnerable regions like the Himalayan state of Himachal Pradesh in India. Here combining local knowledge with near real-time data can transform disaster risk reduction (DRR) and climate action strategies strengthening resilience. Our bandhus recently completed a series of intensive consultations with diverse stakeholders working to improve climate mitigation in the hill state. These convenings were a part of our ongoing efforts to expand CivicDataLab’s Intelligent Data Solution for Disaster Risk Reduction (IDS-DRR) and create a robust climate data ecosystem for the state.

This blog explores the benefits of a multidisciplinary approach to data-driven interventions, highlighting how community engagement and diverse expertise can drive effective policy reform. It is informed by the discussions and workshops conducted with support from the Himachal Pradesh State Disaster Management Authority (HPSDMA) and not-for-profit Doers, last month.

The Power of Local Expertise

Communities living in disaster-prone areas are often the first responders. They have deep knowledge of their environment as they understand local risks, historical patterns and nature-based solutions — insights that complement scientific data. We at CivicDataLab believe that evidence based decision making needs to be enhanced using local knowledge pools and civic engagement. This results in a more holistic and impactful approach for ensuring long-term resilience.

The role of communities in data-driven disaster risk reduction (DRR) goes beyond just supporting rescue and relief efforts. It includes local knowledge and expertise that need to feed into evidence based policies and decision making to ensure inclusive and impactful interventions for building climate resilience.

For instance, Himachal Pradesh faces a multitude of hazards, including earthquakes, landslides and floods. Local knowledge about these risks is invaluable for improving risk assessments and enhancing disaster preparedness.

In 2023, extreme weather events like flash floods, cloudburst and landslides, cost a cumulative loss of Rs. 9905.77 crores to Himachal Pradesh, according to State Emergency Operation Centre, (SEOC). HPSDMA stated that 428 people have lost their lives due to various disasters caused by extreme weather events, which included 83 flash floods, 5502 landslides, 27 cloudbursts resulting in loss of both lives, livelihoods and infrastructure.

Bridging the Gaps

Hyper-local information through communities can help authorities react faster and more effectively, making disaster management more responsive. Given that the first impact of any extreme weather event is people, the communities become a crucial source of near-real-time data, which fills crucial gaps in strategic evidence based decision making processes. In Himachal Pradesh, where extreme weather events are frequent, this data is essential for timely, accurate and sustainable interventions and response.

Community participation also plays a crucial role in building local resilience. The communities can inform and assist in ensuring resource allocation is effective on the ground in the impacted areas. Given the highly localised nature of geography, community engagement helps garner a deeper understanding, which in turn strengthens climate mitigation and adaptation.

Disasters affect vulnerable groups — such as women, children, the elderly, and those with disabilities — differently. Engaging these groups in disaster planning ensures their specific needs are addressed, making DRR strategies more comprehensive. This inclusivity enhances overall community resilience and reduces disaster impact.

Role of Patwaris in Data Collection

In Himachal Pradesh, Patwaris (local officials) play a critical role in disaster management, who are also seen as local community leaders and sources of granular information. Traditionally, as well as today, in large parts of the himalayan region Patwaris gather damage reports from residents, conduct on-site inspections and compile data for higher government authorities.

Patwaris’ reports are crucial for the District Disaster Management Authority (DDMA) to allocate resources and coordinate responses. This data integration ensures timely and accurate disaster management, demonstrating the importance of local data in policy-making.

Integrating Diverse Expertise

A multidisciplinary approach combines local knowledge, near real-time data, scientific research, and policy analysis to create robust disaster management strategies. This approach brings together experts from various fields — such as meteorology, engineering, social sciences and public health — to address complex challenges posed by claimate change.

For instance, integrating meteorological data with community reports can improve flood forecasting models. Collaboration between engineers and local experts can enhance infrastructure resilience. Social scientists can help design inclusive policies that address the needs of vulnerable populations.

Enabling Policy Reform with Community Engagement

A multidisciplinary approach also drives policy reform. By integrating diverse perspectives, data-driven interventions can lead to more effective and inclusive policies. Community engagement ensures that policies reflect local realities and address specific needs.

In Himachal Pradesh, the State Action Plan on Climate Change (SAPCC) and the 2030 Sustainable Development Goals (SDGs) support this approach. The SAPCC aims to incorporate climate data into policy-making, while the SDGs emphasise resilience building across various sectors. By aligning these frameworks with community input and multidisciplinary expertise, policies can be more responsive and effective.

Strengthening Participatory Climate Action

Communities are vital in shaping climate action and disaster management. A multidisciplinary approach that combines local knowledge, near real-time data, scientific expertise and community engagement can revolutionise disaster risk reduction. By integrating diverse perspectives and fostering collaboration, we can create policies that are not only effective but also inclusive and adaptive.

Let’s harness the power of community insights and multidisciplinary expertise to build stronger, more resilient systems for the future. Together, we can turn data into actionable strategies and drive meaningful policy reform.

Critical Role of Local Communities for DRR was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

Intelligent Data Solution for Disaster Risk Reduction (IDS-DRR) is aimed at helping implementing agencies take informed and impactful decisions to prioritise resource allocation for the most vulnerable communities and geographies impacted by climate change. This framework is built to enable authorities’ efforts in DRR and institutionalise inclusive, sustainable and transparent policy reform. IDS-DRR is informed by the SENDAI framework and leverages data and technology to enhance resilience.

This effort has been made possible through expanding partnerships and leveraging open data collaboratives. We would like to thank all our supporters — Open Contracting Partnership, The Rockefeller Foundation, Patrick J McGovern Foundation — who have helped strengthen the data for public good ecosystem. Here, we would also like to make special mention of the disaster management authorities from the states of Assam, Himachal Pradesh and Odisha who are partnering in this endeavour.

The following blog is an authorised repost of the original post published here.

The Patrick J. McGovern Foundation has awarded a US$400,000 grant to the Open Contracting Partnership to scale up the Intelligent Data Solution for Disaster Risk Reduction (IDS-DRR). Developed in partnership with the Indian civic tech organization CivicDataLab, this cutting-edge data model identifies the most critical areas for investment to increase climate resilience and mitigate disaster risk.

“As climate change continues to affect the frequency and scale of natural disasters, effective planning for floods, heat waves, and other dangerous conditions is critical to protect vulnerable populations. Using advanced data models to target the deployment of government resources, Open Contracting Partnership and CivicDataLab are transforming how post-disaster aid reaches communities in need. This innovative approach not only streamlines access to critical resources, but ensures disaster relief efforts are targeted, precise, and equitable — contributing to a future where communities are better prepared and protected,” says Nick Cain, VP of Strategy & Innovation at the Patrick J. McGovern Foundation.

Piloted in Assam, India, the model successfully assesses each district’s flood proneness, preparedness and losses. The grant will enable expansion to other Indian states — Himachal Pradesh, Odisha, Bihar and Uttar Pradesh, with the potential for global adaptation in the future.

“Open contracting strategies for better flood-related procurement are empowering Assam authorities to better protect vulnerable communities from extreme weather. This unique approach to improving public spending and climate finance has the potential to significantly impact people’s lives,” says Bernadine Fernz, Head of Sustainability, Infrastructure and Asia at the Open Contracting Partnership.

Thanks to the Intelligent Data Solution for Disaster Risk Reduction, Assam authorities are now able to make better, evidence-based decisions around how flood-related funds are spent. For example, in March 2023, 95% of the budget for the latest round of flood-related spending went to districts identified by the data model as highly vulnerable to flooding. Those funds will mostly be used for procurement related to repairing and restoring roads, bridges and embankments, benefiting approximately 6.5 million people.

“Previous disaster risk reduction plans or decision-making approaches could not take into account the multidimensional nature of the problem, as the data informing hazard, exposure, vulnerability, and coping capacity were scattered and siloed in innumerable reports across various departments. Our data model addresses this challenge by providing decision-makers with a comprehensive understanding to make informed decisions,” says Gaurav Godhwani, CEO at CivicDataLab.

Using the data model promises faster and better future disaster planning. Initial trials show measuring disaster preparedness using the data-driven approach would significantly reduce the number of district representatives needed to assess preparedness and streamline the process across multiple departments.

Learn more about the project and the model.

Media contact

Georg Neumann, gneumann@open-contracting.org, +1–202–7144460
Moyna Manku, moyna@civicdatalab.in, +919971054261

About

CivicDataLab (CDL) works at the intersection of data, technology, design and social science to strengthen access to public information, evidence-based decision-making and citizen participation in governance. CDL harnesses the potential of open knowledge movements to strengthen the data-for-public-good ecosystem and enable citizens to engage in matters of public reform. We work closely with governments, non-profits, think tanks, the media and universities to enhance their data and technology capacity to better data-driven decision-making. In the last six years, we have collected, cleaned and published nearly 40,000+ public interest datasets and are catering to an active user base of more than 1.5 million citizens.

Our current areas of expertise include digital public goods & infrastructure, climate change, public finance, urban development, open contracting and law & justice. We co-create digital public goods like open data platforms, data exchanges, data science models and citizen-led apps for improving participatory data-driven governance in India and other countries.

About the Open Contracting Partnership

The Open Contracting Partnership is a silo-busting collaboration across governments, businesses, civil society, and technologists to open up and transform government contracting worldwide. We bring open data and open government together to make public contracting fair and effective. Spun out of the World Bank in 2015, we are an independent not-for-profit working in over 50 countries around the world. We help make reforms stick and innovations jump scale, and foster a culture of openness about the policies, teams, tools, data, and results needed to deliver impact.

About The Rockefeller Foundation

The Rockefeller Foundation is a pioneering philanthropy built on collaborative partnerships at the frontiers of science, technology, and innovation that enable individuals, families, and communities to flourish. We make big bets to promote the well-being of humanity. Today, we are focused on advancing human opportunity and reversing the climate crisis by transforming systems in food, health, energy, and finance.

About the Patrick J. McGovern Foundation

The Patrick J. McGovern Foundation (PJMF) is a philanthropic organization dedicated to advancing artificial intelligence and data science solutions to create a thriving, equitable, and sustainable future for all. PJMF works in partnership with public, private, and social institutions to drive progress on our most pressing challenges, including digital health, climate change, broad digital access, and data maturity in the social sector.

Expanding Data Driven Disaster Risk Reduction Collaborations was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

Our work on child-protection over the past six years is aimed at building a better data ecosystem for citizens, civil-society organisations, decision makers and other stakeholders working on child-rights. Such an ecosystem is critical for understanding enforcement of child protection laws. Access to granular information from court cases helps organisations working on issues related to child rights for advocacy with the government and also the judiciary to strengthen the enforcement of laws relating to children. In this blog we write about our experiences of curating and analysing the data obtained through the eCourts platform.

Earlier this month, in partnership with Enfold (Enfold Proactive Health Trust), we published two reports examining the implementation of child protection laws in India.

The first report titled Decoding data on Implementation of the Child and Adolescent Labour (Prohibition and Regulation) Act, 1986 is based on an analysis of 10,800 ongoing and disposed cases available on eCourts from 6 States — Assam, Bihar, Jharkhand, Maharashtra, Tamil Nadu and Uttar Pradesh. The report also includes the analysis of 142 judgements from Maharashtra, Tamil Nadu and Assam, between January 2015 — January 2023.

The second report titled Trends in Child Marriage is based on the analysis of 174 judgements under the Prohibition of Child Marriage Act, 2006 from Assam, Maharashtra and Tamil Nadu between January 2015 and January 2023.

The datasets used for research and analysis can be accessed through the Justice Hub.

The Beginning — From NCRB to eCourts

The journey of creating a comprehensive data ecosystem for child protection dates back to 2019 when we started our collaboration with HAQ — Centre for Child Rights (HAQ) to analyse the court proceedings in cases registered under the Protection of Children from Sexual Offences (POCSO) Act, 2012. At the time, National Crime Records Bureau (NCRB) reports were used to analyse criminal statistics but we soon realised it was hard to curate information to understand how cases are dealt by the various courts.

The annual reports published by the NCRB only provide a few state level aggregated metrics for certain criminal laws. The NCRB follows a primary offence rule in reporting these metrics. This means that where more than one offence is committed simultaneously, only the most serious offence is recorded. This approach might lead to under-counting of cases and these metrics cannot be relied on for accurate analysis.

The Government of India began a project to digitise the courts in 2007. One of the goals of the eCourts project was to improve access to court or case related information for citizens. The POCSO project gave us an opportunity to dig deeper into eCourts to understand what information is available and what processes can be analysed. We published the report in April, 2021. In 2022 (also the year when the POCSO Act completed a decade), the JALDI team at Vidhi extended this research by analysing the judicial processes followed for POCSO cases in 486 courts. Their report can be accessed here.

While the initiatives to understand the court proceedings in POCSO cases did contribute a few important datasets and research frameworks to understand the implementation of the POCSO Act, we still found a dearth of data to study three other important child-protection laws of the country. These are:

1. Child and Adolescent Labour (Prohibition and Regulation) Act, 1986 (CALPRA)
2. The Prohibition of Child Marriage Act, 2006 (PCMA)
3. The Juvenile Justice (Care and Protection of Children) Act, 2015 (JJ)

The project

The work on this project started after we received a grant from the Patrick J Mcgovern Foundation (PJMF) under the 2023 Data and Society Accelerator Program. As part of this grant, our aim was to collect and study two categories of data points. The metadata for each case and the judgements wherever available. Each case on eCourts comprises of its metadata i.e. everything available on the case page such as: Date of registration, Type of a case, Designation of Judge, Legislations, Hearing Dates and the purpose of hearing, Nature of disposal of a case for disposed cases and details regarding the Transfer of cases across courts. A few disposed cases also have orders and judgement linked to the case page.

Refer to these blogs — Harnessing Legislative Data to Support Child Protection Efforts in India and Creating a dataset of cases to analyse the implementation of child protection laws in India to know more about the project. The project roadmap can be referred to at this link.

The initial study and scoping of information showed that we needed an extremely focussed approach and identify specific data points in order to be able to understand the enforcement of child protection. In the sections below, we share our observations from dealing with these two categories of data points available on the eCourts platform.

The case metadata

Our plan was to study cases under all three laws cited above. But given the time constraints and the vast number of variables within the data-sets we chose to focus on two legislations. This has helped us build a framework, which can be further expanded/ customised to study other laws governing child protection.

Insights from the process to study the metadata are summarised below:

• Start small — We wanted to curate data for at-least 6 states to ensure the final dataset is geographically diverse. The POCSO study was limited to 3 states but each state contributed to a larger number of cases. Working with fewer states/districts is a better approach as the data quality parameters differ for each state. Studying a sample of cases from each state before finalising the states might help in evaluating the time needed to prepare the data for analysis.
  – The same principle applies for selecting the laws. The number of cases registered under each law is directly proportional to the time it takes to prepare the dataset. More cases will lead to a more number of acts and sections (and their combinations) to ṣtandardise.
• Figure out the right set of variables — As mentioned above, information for each case is categorised under several sections and each variable has to be standardised before it can be used for any analysis. Framing of questions (hypothesis) at the beginning will help in restricting the scope of the data cleaning required.
• Review the information curated by other projects — There has been a rise in the number of studies conducted using data from eCourts. Some of these studies provide access to datasets under open licences. Referring to these standardised datasets while processing the datasets might save some time. Some of these datasets are:
   — The POCSO dataset published by HAQ and CivicDataLab
   — The judicial dataset published by DevDataLab
• Build a network of people who understand the legal processes — The judicial practices followed in each state might be different. The same holds true for certain data practices followed by courts while entering data in the eCourts portal. The people from these states might be more familiar with such practices and can help in deciphering certain variables or case patterns.

This study from the National Institute of Public Finance and Policy (NIPFP) highlights a few issues with the eCourts data.

The judgements

The grant provided us the time and resources to experiment and test proof-of-concept data approaches. The following quote from the PJMF website aptly summarises the purpose of this grant:

The program provides resources to seed potential rather than to scale proven solutions, foregoing the need for nonprofits to set up and invest in a test data infrastructure or strategy before knowing if it’s a good fit for their work.

The legal judgements are a crucial source of information to understand the processes followed by courts. But at the same time, analysis of several hundred judgements is a resource intensive process especially for non-profits. In the last few years, we have seen a rise in the number of Natural Language Processing (NLP) methods that are well suited to process legal texts. In this project, we wanted to test a few algorithms to extract relevant information from the district court judgements. For this task, we limited ourselves to the analysis of English language judgements. The process to study judgements is described below:

1. We began by testing the OpenNyAI library on a set of POCSO judgements. This library performs well on extracting certain features which are described here. Refer to this blog to know more about the process and results.
2. Over a few iterations in collaboration with the Enfold team, we compiled a list of variables needed for analysis of CALPRA and PCMA cases. We had to study a few judgements to understand the semantics of these variables and how and where they are mentioned in these judṣgements. We analysed a sample set of judgements from all relevant states to figure out relevant patterns and differences between each state in recording these variables.
3. After testing a few NLP methodologies, we went ahead with a combination of RegEx (regular expressions) and the OpenNyAI library to extract information for these variables. Going forward, we would like to test a few Large Language Models (LLM) and context-based ṣQuestion-Answering models as an alternative to RegEx.
4. The information extracted from judgements was verified at-least a couple of times to ensure accuracy. We also eliminated certain variables after this process. Overall, the process worked well for extracting dates, charges, and for other keyword reference checks like reference to a child protection officer. Overall accuracy for certain variables improved over a few iterations.

Through this process, we managed to curate relevant information from over 300 district court judgements. Now we have a framework which can be used to extract information from judgements, for at-least a few indicators. Going forward, we would want to test this framework for other use-cases. So, do write to us if you’re looking to analyse judgements. We’d love to collaborate with you.

Next Steps

Our first consultation took place at the Karnataka Judicial Academy, where the Enfold team shared insights from both reports. The panelists addressed not only the legal challenges in implementing these acts but also the cultural and political factors contributing to the persistence of child labor and child marriage. Participants agreed on the need to establish a data ecosystem to enhance the current discourse and shed light on the shortcomings of existing policies.

Moving forward, our aim is to curate more relevant datasets in these areas and connect with other organisations and individuals dedicated to these issues. We hope to collaborate towards creating a robust data and research ecosystem. Please share your ideas and suggestions with us and we look forward to your feedback on the reports and datasets.

Lost in the crowd: Data for Child Protection was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

Why is it difficult to engage with government budgets at the sub-national levels?

Government budgets in India have historically been characterized as being complicated with heavily coded documents. This has limited any meaningful engagement with and discourse on budgets by a wider base of citizens. Furthermore, accessibility to comprehensive and simple to understand budget data becomes increasingly a challenge as we move down from the central level to the States and then further below to the local levels of government. To add to this, improvements made in the availability and quality of fiscal data in the country over the last decade have remained uneven across different states. As a result, while a lot is discussed and dissected on the Union Budget, citizen’s engagement is far less when it comes to analysing state budgets and making comparisons across them.

States are critical focal points for implementing a majority of the flagship government schemes and programmes, across sectors in the country and the onus of spending on development outcomes increasingly lies with them. However, there are very few platforms which allow users to comprehensively analyse insights from different state budgets under one roof and scrutinize them sector-wise in an easy to comprehend manner. To address this gap, CivicDataLab, in collaboration with their technical partner, the Centre for Budget and Governance Accountability (CBGA) has developed the Sector Dashboard on State Budgets, as an important component of the open data platform, Open Budgets India (OBI).

What does the Sector Dashboard provide?

The dashboard makes available sector-wise curated data for 11 social and economic sectors like education, health, rural development, agriculture etc. for all states as well as a set of Aggregate Budget and Receipts Indicators. Each sector has a group of 5 common indicators such as ‘Per-capita Expenditure, ‘Total Sector Expenditure’, ‘Total Sector Expenditure as a % of State Budget’, ‘Revenue Expenditure as a % of Total Sector Expenditure’ and ‘Capital Expenditure as a % of Total Sector Expenditure’ which can be filtered as per the user’s requirements. Data is available for seven financial years from 2017–18 to 2023–24. It enables users to make interstate comparison of budget data by generating simple and interactive visualisations, by choosing from a set of different types of presentations like the map or choropleth view, the table form and the bar diagram option, which can be downloaded for reuse. Detailed descriptions are included to help users understand the specificities of calculation for the indicators presented. In addition to the dashboard, the complete database for all the sectors is openly available for the users on OBI.

What is the significance of developing the Sector Dashboard?

In addition to being ponderously coded and jargonized, budget documents of different states are often arranged and presented differently. Hence, it takes a mammoth effort to create a standardised database of fiscal indicators for any meaningful comparison across states. It entails collating data from multiple sources for multiple states, and thereafter making sense of the varied formats of the available budget data. Further, sector-wise curated budget information is not readily available. Spending on a sector is typically spread across multiple departments and requires manual effort and sectoral understanding scanning through the detailed budget books of all such departments and identify the relevant budget heads. For example, spending on health, as a sector, is incurred by not only the Health Department of a state, but by several other departments like Medical Education, Training and Research, Urban Development & Public Works, Home, Social Justice and Empowerment, Tribal Development, Labour and Skill Development, etc. to name a few. The Sector Dashboard on OBI addresses both these issues by making available sector-wise data on budget allocations and spending, as well as making such data comparable across states and financial years.

How can the dashboard be used?

The dashboard has a wealth of information, carefully assembled from state budgets, which can be used by researchers, policy analysts, journalists, and budget enthusiasts alike. Users can explore the functionalities of the dashboard by leveraging the interactive interface where they can switch between the indicators, budget attributes (e.g., Budget Estimates, Revised Estimates and Actuals) and financial years.

Comparing, say, the per-capita health expenditure for the north-eastern states become easy with the dashboard where it comes out clearly that in 2020–21, Sikkim has the highest per-capita spending on health among all north-eastern states, followed by Arunachal Pradesh (the graph below showing the comparison is generated from the dashboard). One can also use the data tables from the dashboard to generate further insights and weave their own stories. The detailed concordance tables provided alongside the data, highlight the definitions used in creating the sectoral boundaries and at the same time enables researchers to modify and add to those definitions, creating opportunities for collaborative learning.

The sector-wise arranged data on the Dashboard helps in directing the spotlight on the government’s budgetary commitments for key development sectors. It can also serve as a useful tool for the policy makers to understand their relative fiscal position, among other states. The ongoing budget discussions of FY 2024–25 can effectively be informed by unpacking nuanced insights from the dashboard and in turn make budgets more accessible and engaging for many.

Unpacking State Budget Trends through the Sector Dashboard on Open Budgets India was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.

Climate change and its adverse impact on lives and livelihoods is undeniable. Every year millions of dollars are spent on planning and strategy for addressing it. The resultant climate action needs to be swift, resilient and transparent. Mapping, analysing, collecting and collating data for action oriented insights to climate change and reducing the loss of lives and livelihoods due to the increasing number of extreme weather events, is one such crucial approach for doing this. And the need to build innovative and sustainable interventions including viewing disaster risk reduction (DRR) beyond rescue and relief is more urgent now than ever before.

By harnessing data to understand the drivers of climate change and assess its impacts, we can develop proactive measures to mitigate risks and build resilience. From incorporating climate considerations into public procurement processes to advocating for climate-smart policies, data-driven approaches empower decision-makers to take meaningful action in the fight against climate change.

Based on these principles, we at CivicDataLab, along with Open Contracting Partnership with the support of The Rockefeller Foundation have developed a data driven solution to help decision makers take informed and impactful decisions to prioritise resource allocation for the most vulnerable communities and geographies impacted by climate change. This framework is built to enable authorities’ efforts in DRR and institutionalise inclusive, sustainable and transparent policy reform.

Our journey with the intelligent data solution for disaster risk reduction (IDS-DRR) started close to three years ago with support from the Patrick J McGovern Foundation, where we scoped and tested data models to inform strategic decision making for DRR. At present, the alpha version of this solution is being tested by the Assam Disaster Management Authority (ASDMA) and other stakeholders. It will be publicly launched in the near future and we have begun work to scale the project to other vulnerable geographies like Odisha and Himachal Pradesh.

Making Data Effective and Actionable for DRR in Assam

ASDMA is among the forerunners in initiating the process of DRR in alignment with the recommendations of the Sendai Framework for Disaster Risk Reduction 2015–2030 (SFDRR). The authority has already set up in multiple ways like the DRR Roadmap for Assam, capturing daily flood damage data through Flood Response Information Management System (FRIMS), maintaining a web GIS portal among other initiatives to address climate change in a sustainable manner.

Building upon these policy advancements, CivicDataLab is co-creating the IDS-DRR, which combines satellite & weather data, demographic data, access to Infrastructure data, losses and damages data and government response as seen through public procurement data. The framework is built to enhance effective flood mitigation strategies for the authorities. Our solution leverages advanced data science techniques like machine learning (ML) to derive insights for decision-makers and make it into a user-friendly, interactive open-access tool.

We have found that a significant obstacle in leveraging data for effective climate action and DRR is the fragmented and siloed nature of data. Relevant data is often scattered across different agencies, stored in disparate systems and presented in various formats. This fragmentation makes it challenging for decision-makers to build the required capacity, access, analyse and utilise data in a timely and coherent manner to take informed timely actions and create a robust data-driven policy framework.

However, our solution has attempted to address these challenges by following three key steps — creating a single point of access for all relevant high-value datasets (HVDs), enabling actionable data insights using a ML model and enhancing institutional data capacities of the implementing agencies like Disaster Management Authorities. IDS-DRR uses HVDs from 10 different sources that have been standardised and geocoded making them interoperable and enhancing strategic decisions for building long term resilience against floods in most vulnerable regions.

We piloted this project with the state of Assam authorities in 2021. We shared our findings and methodology with the state authorities in 2022, which they were able to use for their decision making showing us early impacts in resource allocation where 95% of the fund approvals in March FY 2023 were allotted to the affected districts identified by our data model.

IDS-DRR is an open-source platform that helps state-level and district-level Disaster Management Authorities to take timely data-driven decisions, prioritise expenditure of public funds and conduct public procurement in a manner that strengthens long-term DRR. From analysing risk profiles to tracking losses and damages, our intelligent data solution provides invaluable insights that inform decision-making processes at every level. In this platform, we bring together diverse HVDs from satellite, environmental, social, economic, demographic, infrastructure, loss & damages to data regarding government response to derive timely insights.

In a recent meeting (April 2024) held by the Assam State government to discuss flood preparedness in the state, CivicDataLab’s data-driven framework received a special mention. Covered in various media publications, our IDS-DRR is the result of working at the in collaboration with all stakeholders at the intersection of data, technology, people centric design and research. We believe that the success of intelligent data solutions for DRR is closely tied to collaboration and partnerships across sectors and stakeholders. From government agencies to international organisations, academia and the private sector, collaboration is essential for harnessing the full potential of data in DRR efforts.

NOTES:

With support from The Rockefeller Foundation, we at CivicDataLab, have been expanding our climate related work through co-creation of knowledge and dialogue between a diverse group of stakeholders. We have conducted virtual as well as in-person convenings to further strengthen the open data for public good ecosystem for leveraging evidence based decision making for sustainable climate action.

This article is part of CivicDataLab’s initiative Open Contracting India, and the project Intelligent Data Ecosystem Assam — Disaster Risk Reduction. The team members who contributed are Kabeer Arora, Gaurav Godhwani, Jeeno Soa George, Phani Datta Surampudi, Sai Krishna Dammalapati & shreya agrawal.

About us:

The Rockefeller Foundation is a pioneering philanthropy built on collaborative partnerships at the frontiers of science, technology, and innovation that enable individuals, families, and communities to flourish. We make big bets to promote the well-being of humanity. Today, we are focused on advancing human opportunity and reversing the climate crisis by transforming systems in food, health, energy, and finance.

For more information, sign up for their newsletter at rockefellerfoundation.org and follow them on X @RockefellerFdn.

Open Contracting Partnership (OCP) is a not-for-profit organization working with governments, businesses and civil society to transform government procurement so that it delivers better outcomes for people, planet and prosperity. We use the power of open data alongside inclusive and participatory approaches to open up public contracts, break down silos and inform decision-making. Over 50 countries all around the world are already implementing open contracting solutions. Our work has been endorsed by the G7, G20, and the UN amongst others. The proposed project blends our expertise on procurement, sustainability and open data. It will be overseen by Lindsey Marchessault, Director of Data and Engagement, and Bernadine Fernz, Head of Sustainability, Infrastructure and Asia. OCP team member profiles are available at: https://www.open-contracting.org/about/team

CivicDataLab works at the intersection of data, technology, design and social science to strengthen access to public information, evidence-based decision making and citizen participation in governance. Our work harnesses the potential of open knowledge movements to strengthen the data-for-public-good ecosystem and enable citizens to engage in matters of public reform. Our current areas of expertise include digital public goods & infrastructure (DPGs & DPI), climate change, public finance, urban development, open contracting and law & justice. We have co-created digital public goods like open data platforms, data exchanges, data science models and citizen-led apps for improving participatory data-driven governance in India and other countries.

Some of our publicly available open data initiatives include Open Budgets India, Justice Hub, Open Contracting India, Open City, CogniCity among others. We have co-created digital public goods with National Informatics Center (NIC), Ministry of Electronic & Information Technology (MeitY) and the Government of Assam. Additionally we actively build capacity for a diverse group of partners working to enhance social impact, situated in India, Indonesia, Philippines, Thailand, Panama and Scotland.

Using Data for Disaster Risk Reduction(DRR) and enhanced Climate Adaptation was originally published in CivicDataLab on Medium, where people are continuing the conversation by highlighting and responding to this story.