Announcing the CaDC Wholesale Data Action Team
Streamlining Data Management in the Water Industry
Rapidly changing water supply conditions are placing increased demands for adaptive resource management on the water industry. It’s become increasingly apparent that using data effectively is key to making effective decisions such as identifying the highest-leverage investments in system reliability. Yet just getting access to data can be a challenge for many water agencies, and this is especially true for wholesale water suppliers who are often one level (or more) removed from the people and businesses that actually put water to beneficial end uses.
Add to this the fragmented and interdependent nature of California’s water system, which features multiple and overlapping layers of agencies responsible for water supply, resource conservation, regulation and oversight. Collaboration is essential to get anything done in a system like this, but ensuring that the right decision maker has the information they need can feel like a nightmare given the fragmentation. It is likely that a single data point changes hands five or more times before it reaches the final user, with transformations, assumptions, and aggregations potentially added at every step along the way.
For example, consider the meter on a local groundwater well, and how the data from this one water source may ultimately make its way “upstream” to influence water policy at the highest levels:
- An operator may read that meter and write it on a sheet of paper.
- Back at the office it is entered into a local database.
- From there, an analyst in water resources copies that information into a spreadsheet requested by their water wholesaler and sends it off in an email.
- The wholesaler in turn compiles spreadsheets from all of their retailers and forwards it on to their regional wholesaler.
- The regional wholesaler compiles spreadsheets from all of their members and adds the data into a database they maintain.
- This database is then accessed and used for a variety of purposes, from integrated resource planning to state reporting.
- From there, other agencies, the State, and the general public may then take this data and use it or republish it for its own purposes.
All of this represents a lot of time and effort, and each step introduces potential errors, especially when performed manually. Might there be ways to improve the information flows, especially to and from wholesale water agencies? How can the water community collaborate to identify and streamline these processes?
Addressing complex data workflow challenges in the water industry calls for coordinated solutions. It was this realization that inspired the California Data Collaborative (CaDC) to form the Wholesale Data Action Team. The team represents the latest initiative by CaDC to rally around emerging topics and tackle them through the power of collective wisdom and collaboration.
Challenges and Opportunities
After meeting twice during the Spring of 2023, we are pleased to announce that the team has identified five significant opportunities for improvement. Let’s dive into these opportunities and explore how we can work together to transform these challenges into opportunities for success.
Opportunity 1: Modernizing Invoice and Water Purchase Data Management
Currently, many water purchase invoices between suppliers are provided in PDF format. In order to be easily aggregated, analyzed, and published, this data needs to be pulled from PDF format into a more machine-readable format such as a spreadsheet or database. The team discussed the following solutions:
- Collaborate with wholesalers to help them share accurate, machine-readable data, in addition to PDF files.
- Develop scripts to convert PDF invoices into a machine-readable format.
- Create an invoice database with reporting tools for easy analysis.
By implementing these solutions, we can create a streamlined, easily accessible system that will increase operational efficiency and data accuracy.
Opportunity 2: Enhancing Local Water Production Data Collection and Reporting
Wholesale water agencies often encounter difficulties in collecting local water production data, as outlined in the example above. The team suggests developing a unified reporting platform to ease data storage and reporting processes. This platform could provide retail agencies with robust systems to monitor their production data, potentially eliminating inefficiencies associated with manual data merging and emailing and copying spreadsheets back and forth.
Opportunity 3: Streamlining Access to State Reporting Data
Even though agencies report a lot of data to the State and the State republishes much of this data, accessing specific insights quickly when required can be a challenge. The Wholesale Data Action Team suggests developing a platform that pulls from various open state datasets, connects the data, and publishes the data in accessible dashboards and visualizations. This approach would simplify the process of data analysis and promote better data-driven decision-making.
Opportunity 4: Improving Collection and Reporting of Water Use Efficiency Rebate Data
Data collection related to rebates from local agencies often occurs in an ad hoc manner, with each agency creating their own processes for collecting and storing data. When the time comes to take a step back and report on outcomes, evaluate programs, and plan for the future, this scattered approach complicates analysis. The team suggested options such as creating a standard format for rebate data tracking, or developing a software system to centralize rebate data.
Implementing this system could help eliminate inconsistencies and inefficiencies in data collection and analysis. It also offers additional benefits in the form of heightened levels of trust enabled by increased levels of transparency and visibility into rebate funding and results, as demonstrated by staff from the Inland Empire Utilities Agency which published a rebate dashboard for their member agencies.
Opportunity 5: Optimizing Monitoring and Reporting of Water Use Objectives
Wholesalers often face difficulties in monitoring their retailers’ water use and efficiency and leveraging this data for demand forecasting and resource planning. Resource planning will become even more complicated after the implementation of the upcoming water use objective (WUO) regulation which will set caps on water use for each retail urban water supplier and require reporting towards compliance. The team suggested a couple of possible solutions to help wholesalers make use of the WUO data:
- The simplest approach would be to wait until the WUO data has been collected and published by the state, and then to leverage the published water use objectives and visualize them as needed for planning.
- Another approach would be to build on the functionality that CaDC has built (and is currently expanding) to ingest retail billing data, align this data with the WUO reporting, and offer analysis and summary information. This would allow for capabilities like mid-year forecasts of WUO compliance, and data sharing of additional WUE data such as CII classifications.
Next Steps
These identified opportunities represent significant steps towards a more effective, data-driven future for the water industry. The Wholesale Data Action Team is committed to addressing challenges like these and driving forward initiatives that will improve data workflows and foster sustainable decision-making. We are especially excited to develop local solutions that can plug into other ongoing work to streamline urban water data reporting.
As part of this commitment, the team consensus was that Opportunity 2: Enhancing Local Water Production Data Collection and Reporting was the highest priority of the 5 current opportunities. To take the next steps toward action, CaDC staff have begun a series of individual and group interviews to learn more about the current processes used by suppliers around their local production data, and to brainstorm ways to streamline these workflows. Stay tuned for more updates from the CaDC as we continue to break new ground in this exciting journey.
Join Us
CaDC is a non-profit staffed by data experts and governed by its member water agencies. To learn more about our membership benefits, visit us at www.thecadc.org or schedule an introduction call.
