But really, how do you build demand for data science in Indonesia’s public sector?

We recently teamed up with our main government counterpart, the Indonesian Ministry of National Development Planning (Bappenas), to host a seminar showcasing our collaborative research on using new data sources to monitor the dynamics of Indonesia’s economy. When we first started writing this blog, it was meant to recap some of the highlights from this seminar. Caught by the contemplative mood of the Lab towards the year’s end, however, we decided to reflect a bit on how we actually got there.

Articulating demand

Contextual analyses of knowledge-to-policy processes in Indonesia (see, for instance, KSI’s solid collection of diagnostics here) emphasise the importance — and the formidable challenges — of building demand for the use of analytics and evidence in formulating public policy. Over the past couple of years, we’ve been working closely with our friends at Bappenas’ Data and Information Centre, trialling different ways of whetting policymakers’ appetites for experimenting with new data sources and advanced data analytics. These include hosting open lab sessions and data innovation clinics, as well as many one-on-one meetings with heads of different divisions. It got to the point where Pak Suharmen, the Head of the Data and Information Centre, knew our slide deck by heart and could jump in at any given point to explain most of our projects.

Perhaps it was a combination of increased awareness, technical imperatives to fill information gaps, and internal advocacy within Bappenas, but in our last Steering Committee meeting, we started to see a distinct shift in the nature of the research requests coming from Bappenas’ high-level officials. For example, instead of asking for a specific piece of research, Bappenas’ Deputy Minister for Economic Affairs put forward a list of priority policy issues on which his division needed input, and asked how digital data sources could best complement their existing data and analysis.

We reviewed each of the proposed policy issues and provided technical suggestions on which types of research would be feasible — and which ones would be less so. Our team at the Lab then sat together with the Deputy Minister and his team to go through each of these and decide together specific topics that we would work on. This subsequently kicked off another round of one-on-one meetings with each of the directorates under the Deputy Minister of Economic Affairs. Pak Suharmen, who attended almost all of these meetings, was instrumental in getting each of the Directors to assign their own analysts to work with our researchers: “You can’t just request a research project and leave it — it doesn’t work that way! You need to get your people working in tandem with the Lab; it doesn’t work without your domain expertise.” Thank you, Pak Harmen.

For our own team, the meetings with each directorate helped clarify the drivers behind the policy issues, what kind of analysis was already available, and how Big Data analytics would actually be useful. In some cases, we found that pursuing analysis of digital footprints would be much less useful than, say, ethnographic research to gain behavioural insights on a specific segment of the population.

Collaborative Research

Considering the Lab’s forte in data processing and data analytics and BAPPENAS’ domain expertise in economic planning, the directorates took the lead in designing and completing the substantive parts of the projects. Each of the directorates assigned two staff members to work closely with Pulse Lab Jakarta on each project. This approach turned out to be highly rewarding as it enabled knowledge exchange, information flow, skill-set complementarity and more straightforward discussions on the operational feasibility of each project.

Some of the projects include:

• Building on our earlier nowcasting model to monitor food price dynamics, especially of priority commodities such as rice and beef, to provide early indicators if commodity price movement approaches maximum retail price and price variation coefficient.
• Trying out new data sources, such as e-commerce data, to use as proxies to map the state of the domestic economy.
• Testing assumptions of correlation between inflation rates and public sentiment of the increase in prices of certain commodities.
• Perception analysis of the “Wonderful Indonesia” brand and Indonesia’s flagship tourist destinations.

The goal was to have research outputs that we could showcase towards the end of the year, so in late November, our joint research teams sat down to take stock of our progress. There was significant variation in terms of having “finished products” for each of the research topics. In some cases, we have had to modify and sharpen research questions again, and rerun the analysis. The project on inflation, for example, was still rough and in need of further iteration because we have yet to establish a formula that shows a strong correlation between inflation and people’s perception of commodity prices (visualisation can be found here).

In other cases, such as exploring the use of e-commerce data, we’ve had to narrow the scope of analysis to something much more specific. We decided to focus on house price data across provinces and see whether we can gain insights from the comparison. While we did manage to clean the data and show the price dynamics pattern (visualisation here), the actual analytic part to correlate this to events, shocks or stresses, or other economic indicators is yet to be done. This is something we are keen on exploring further with the Bappenas team next year.

Our Bappenas colleagues led the way in exploring the connections between digital footprints and other contextual datasets. For example, we worked with analysts from Bappenas’ Directorate for Industry, Tourism and Creative Economy to run a social media analysis of the “Wonderful Indonesia” campaign, comparing this to other campaigns of a similar nature. Bappenas’ analysts then combined this with their own data on strategies for promoting Indonesia’s tourism brand, including where and when promotional events were held. Using time-selected and geographically-selected analysis, the joint team uncovered new insights on the potential effectiveness of particular strategies.

We think it is still too early to measure the value of the results, or to confidently state that the results will be utilised in policy formulation process by Bappenas as originally intended. Looking back, we feel that the most valuable outcome to emerge during these last few months was our counterparts’ genuine commitment to learn new approaches and to lead the analytical parts of these projects. Yes, the Lab played a large role in terms of methods to process the data, but our Bappenas colleagues led the way in making the outputs grounded, and gave them meanings, functions and context within each of their own development policy agenda.

Amplifying lessons learned

The intent was always to showcase the research results to other divisions in Bappenas as an example of how Big Data analytics could be used to inform development planning and policy formulation. The idea of doing this in a larger forum gained momentum when the Bappenas Minister expressed his enthusiasm for exploring the use of new data sources to complement conventional data sources. As we mentioned in the beginning of this blog, we worked with our counterparts in Bappenas to host a seminar where analysts from four different Directorates presented the results of their research to a broader Bappenas audience.

Pulse Lab Jakarta’s role in this forum was to moderate the discussion and answer technical questions on data analytics. The bulk of the discussion, however, was between Bappenas sector experts, technical advisors, and policy analysts on how best to utilise new data sources to complement existing analysis on upcoming national planning priorities. In addition to discussing the benefits of integrating different data sources, it was also good to see a lively debate around the flaws and limitations of some of the approaches. Several members of the audience also raised the point that, although concepts like perception analyses can be useful if used appropriately, optimal use of Big Data analytics means looking beyond social media data and thinking critically about which data sources would yield the best insights.

At the end of the discussion, the analysts shared some of their key takeaways from this entire process:

• Although policymakers need not master the more technical aspects of advanced data analytics, they need to have an inherent understanding of the principles — and more importantly, its limitations. This will help identify which knowledge gaps can be filled with exploratory data science, and which ones would be best met with other ways of obtaining evidence.
• Exploring whether new data sources can be used as indicators for specific economic conditions is an intensively iterative process. This requires not only substantive involvement of policymakers in the research process, but the flexibility to modify a line of inquiry when needed.
• The results from Big Data analytics is one of many inputs that may be used to inform policy processes. To use it optimally, it should be combined with other pieces of evidence that provide contextual depth to the policy issue at hand. In using results of data analytics, policymakers should also be cognisant of the limitations and inherent biases of the data sources that are being analysed.

One final reflection….

Moving forward, we feel that our role in building demand for data science within government is twofold. The first one is in helping to shape and to articulate the demand for the type of analytics that might yield the kind of insights that are needed. This includes suggesting possible methods for analysis, but it also includes being very honest about the feasibility of such an approach and, for exploratory research, the possibility of failure to gain any meaningful insights.

The second role that we need to play is in expanding policymakers’ horizon of options on what is possible to do using data science. Doing this requires us to constantly push the boundaries of our own research and experimentation, and this is something that we very much look forward to doing next year.

Bring on 2018!

Oh, and we did do a recap of the highlights from our seminar with Bappenas — check out the video below.

Pulse Lab Jakarta is grateful for the generous support from the Government of Australia.