Predicting Crop Yield and Profit with Machine Learning
Using data and design thinking to improve the financial outlook of marginal farmers
Over the last few months, I and a team of students at Carnegie Mellon University partnered with KONAM Foundation to research, design, and develop a tool that marginal farmers in India can use to predict crop yield and profit in order to better plan what crops to grow. Together, we were able to develop a data model and user interface that would help improve outcomes for farmers in Rayagada and Nayagarh in Odisha and we’re looking to develop it further to prepare for a pilot in the near future.
Background
In recent years, the stability of rural communities in the state of Odisha, India has been shaken by economic and social forces related to higher suicide rates amongst small and marginal farmers. KONAM Foundation aims to offer assistance and tools to help these farmers and communities address these issues. Generally, this group faces challenges accessing and trusting educational outreach and training to better understand how to increase crop yields and improve financial standing. Because of the serious nature of the issues at stake and general hesitance to trust help from outside the community, any service or product meant to help must be carefully designed and tested in order to ensure positive outcomes and successful adoption.
Focus
While there are many ways to contribute to improvements in the lives of our target audience, our task was to leverage data to predict a valuable result so that farmers and aid workers would be able to make informed planning decisions. Ultimately, the focus of the work during this project was to both conduct audience research that would direct the design of the product and design a data model that would produce the desired results.
Product Goal
The tool that we developed during the course of this project is meant to deliver an actionable prediction, based on individuals crop and financial information, that allows them to achieve sustainable financial independence. For our purposes, financial independence means in the short-term, paying off existing loans; in the long-term, financial self-sustainability and removing dependency on loans. The version of the product that we developed focuses on the profit prediction, but eventually, as more data is available and included as features in the data model, we envision the output of the model to be a plan or detailed recommendation set for farmers to optimize their crop selection based on individual factors such as location, farm size, and finances.
Initial Scope
In support of this larger goal, our tasks over this initial project phase were to:
- Conduct user research to assess needs, constraints, and product market fit
- Develop longer term product vision roadmap and adoption strategy
- Define product design guidelines
- Define data features and clean data sets
- Construct a data model
- Build end to end web app
Research and Design
Before any progress could be made in building a tool, we needed to understand our users and the context in which they’d be accessing our product. This would allow us both the benefit of creating a compelling and relevant tool for our audience as well as ensuring market fit with a region that most of the team was largely unfamiliar with. Key questions we sought to answer include:
What are our users currently using to plan their crops and manage their finances?
What problems do they experience with their current process?
What prevents them from adopting our service or product, if anything?
User survey
We assembled a survey that would give us a better understanding of our users’ lives and experiences. We asked questions that addressed crop planning, farm work distribution, finances, land ownership, and device ownership and usage. Over the course of roughly three weeks, we saw responses from 42 small farmers living and operating farms in Odisha.
Findings
The survey results revealed that the majority of our farmer users would be using feature phones, as opposed to smartphones. Additionally, Internet access is very limited, so even those with smartphones do not have reliable access to the web and instead stick to lower bandwidth apps like WhatsApp. Our participant group tended to plan their crops based off of past success or tradition, but if they did change their crops, it was because of influence from their neighbors. Generally, our participant group was well off with little revolving debt and access to farming equipment, irrigation, and more educated relatives to help introduce new practices and bring products to the nearest market. This is helpful to know, but not representative of our more rural target audience.
Implications
While the initial concept of a web app for farmers to use directly is very attractive, in order to reach our target user base, our product will need to serve multiple use cases including offline mode and use by NGO staff as they visit or remotely meet with farmers to offer them guidance on planning their crops. A web app would work for literate NGO staff with access to smartphones, but not for farmer users until smartphones prevalent and internet access is more stable.
Prototype and testing
In parallel to the user survey, we developed an initial prototype of a web app that collects individual information such as location, farm size, intended crops, and loan and budget amounts. From there, that data is used to create a crop yield and profit prediction that is meant to be a part of a farmers’ crop planning process.
We first created an English prototype to align on the first design and then translated the prototype into Odiya for the field test. For our first round, we tested the prototype with six farmers and five NGO staff members. Overall the concept was very attractive, but financial literacy and cost benefit analysis associated with it are areas that the farmer audience doesn’t yet generally grasp, which indicates that providing this tool as just a part of an NGO staff member’s advice and guidance would be a successful route to product adoption. Additionally, NGO staff tend to have access to smartphones, which would allow them to use the web application.
Iterations
Subsequent versions of the prototype were made to include additional regions within Odisha and relevant crop options as our field test traveled to different sites with one of our key stakeholders. As more feedback came in around a farmers’ understanding of the written questions, we also updated the question screen layout to include room for an audio icon, and eventually, to include audio versions of each question to encourage independent usage of the web app amongst farmers, even if they’re using an NGO staff member’s smartphone.
Beyond the design of the application, we also were able to design and build a functional data model that generated crop yield and profit prediction based on individual farmer information and government collected data sets for climate, cost of production, and market pricing.
We were also able to construct a view for NGO staff to use with multiple farmer clients. The use case for this would allow this product to serve staff as they consult and help multiple farmers in a community or pilot program.
What’s Next
We were able to apply a human centric design approach to defining a solution to the problems faced by our target audience, basing our models and our recommendations on the experiences we uncovered and the data we acquired during the course of our work. All of this initial scope supports a foundation for further development of this product and pilot program with farmers. The next areas that need to be developed more include broader data acquisition, model refinements, and pilot design and planning.
Team
Sarah Papp, Shreya Prakash, Svayam Mishra, Xinwen Liu, Zhuona Ma
Mentors
Afsaneh Doryab, Systems Scientist, HCII, Carnegie Mellon University;
Anind Dey, Director, HCII, Carnegie Mellon University;
Sandeep Konam, Executive Director, KONAM Foundation;
Kanna Siripurapu, Project Coordinator, KONAM Foundation
