Farmer-Bot : An Interactive Bot for Farmers

13 min readJul 24, 2020

Abstract

Agriculture employed 50% of the Indian work force and contributed 17–18% to country’s GDP. In 2016. Agriculture and allied sectors like animal husbandry, forestry and fisheries accounted for 15.4% of the GDP (gross domestic product) with about 31% of the workforce in 2014 This work aims to implement, a Chatbot that promotes remote interaction of the users/farmers to the agriculture environment using natural language processing. We want to build a chat bot which can answer basic queries of a farmers and can also provide a possible information and solution related to agriculture.

1. Introduction & Problem Definition:

We want to build a chat bot which can answer basic queries of a farmers and can also provide a possible information related to agriculture.

1.1 Features:

The farmers and agri-experts provided the researchers with similar questions as the ones they found in the KCC dataset. Based on both sources, they identified four major areas requiring information support:

 Plant Protection: In the KCC dataset, 60.6% of the farming calls were related to remedies for protecting

 Pests and diseases: Agri-experts stated that a majority of farmers seek suggestions on which medicine to spray for a particular crop disease. None of the farmers the researchers interviewed were aware of any disease name. Usually, farmers describe crop diseases by their visible symptoms to the agri-expert; with a few back-and-forth questions, the agri-expert hypothesizes the issue and recommends medicine with dosage information.

 Weather: In the KCC dataset, 39.4% of the overall calls were about weather-related questions; 13.5% o farming questions were about weather. Farmers eagerly sought weather information, as rains can wash away expensive sprayed pesticides and weather conditions determine the best time to harvest crops.

 Best Practices: Information related to best practices can help increase yield in terms of the quantity or quality of potatoes. Common questions were: “Till what height should I put water?” “After how many days, should I harvest?” These best practices questions comprise of 6.6% of the potato farming calls in the KCC dataset. Agri-experts also stated that farmers consistently asked them tips to increase yield and, consequently, income.

 Unbiased Recommendations on Products: Farmers wanted recommendations from agri-experts on products they should purchase. Questions such as “Which fertilizer to put and how many times?” and “Which seeds are the best for red potatoes?” were commonly asked. They prefer to ask these questions to agri-experts instead of local shopkeepers, believing that agri-experts would provide unbiased and trustworthy response; they feared that shopkeepers may be motivated by the profit margin of products.

2. Proposed solution

2.1 The Chatbot will have to learn to identify the following

Chatbot basic features (answering queries related to following)

• Weather

• Nutrient Management,

• Government Schemes,

• Market Information

• Providing information about the seeds,

• planting materials,

• field information and

• cultural practices

2.2 Solution Workflow:

We propose a solution of chatbot which is interactive for the farmers and easy to answer their queries in a effective manner

The Workflow starts with the hello message reply by the bot and queries wil be asked by farmers in which the possible solution will be listed out based on the trained dataset which we give for training

2.3 Literature survey

The survey on artificial intelligence using chatbot works on various purposes and makes human life easier in daily requirements. By Chatbot research papers we concluded that they are very useful in various forms like communication,agriculture,medical,ecommerce,banking etc.

[1]FarmChat: A Conversational Agent to Answer Farmer Queries The researchers developed a knowledge base for potato farming using the KCC dataset and information collected from formative interviews with smallholder farmers and agri-experts. For each of the identified topics, they asked the two agri-experts (who participated in the Formative Study) to provide examples of typical farmer questions, the follow-up questions that they would ask in order to understand the problem, and the final advice they would provide.

All such conversations was added to the IBM Watson Conversation dialogue flow, and the informational advice was included in the FarmChat knowledge base. In the current version, the knowledge base is a SQL database consisting of four tables, one for each of the topics they identified above.

[2]AgronomoBot-a smart answering Chatbot applied to agricultural sensor networks:For agricultural purposes, it is important that the data about field conditions, such as air and soil temperature, air relative humidity, soil moisture, rainfall, wind speed and other relevant variables, be rapid and easily available for use by farm management systems, by specialists, or the farmer itself in decision-making processes.

AgronomoBot was developed focused on the search and display of data acquired from a Wireless Sensor Network deployed on a vineyard. It is based on Telegram Bot API and is able to access information collected by echo field sensors, bringing it back to a user through interaction over the Telegram application. The IBM Watson cognition services platform was also used for improving the user experience by enabling the use of natural language during the conversation experience, providing intention detection.

Further developments are planned for AgronomoBot, such as the expansion to other messaging platforms, the implementation of speech communication capacity, image classification and continuous data analysis. It is hoped that with analytical capacity over the mass of available data, it becomes possible to work towards the prevention of harmful situations to agricultural productions, early detection of diseases in crops, energy and water waste reduction, and advanced management capabilities for the farmer.It was possible to achieve the objectives, presenting a satisfactory solution for the search and display of data on a WSN applied to wine production, based on the use of natural language that combines the functionalities of the electronic message service Telegram and the power of the cognitive services platform Watson from IBM

Natural Language Processing

Natural Language Processing (NLP) is a theory-motivated range of computational techniques for he automatic analysis and representation of human language (CAMBRIA; WHITE, 2014).[3]According to Lehnert and Ringle (2014), research on NLP should not be mistaken by speech recognition but is concerned with the symbolic manipulations of meaning and interface that are needed once words are recognized. In fact, a speech recognition algorithm needs to be paired to a language processing program in order to implement actual verbal dialogues with computers.[4]The authors Cambria and White (2014) state that NLP research is in a paradigm shift, they are no longer based on techniques of recognition and understanding of loose words. But now begin to explore semantic techniques more consistently, which the authors call a jump from syntactics curve to the semantics curve, and ultimately will arrive at the pragmatics curve, wherecomputational programs will be able to investigate and build entire narratives.

Chatbots New technologies have favored the creation of intelligent and autonomous systems, and amongthem is the emergence of Chatbots. The term “chatbot” or “chatterbot” indicates a robot that cantalk and can be defined as a software that allows the simplification of interactions between humans and machines.

These interactions can occur through speech or writing in the natural language, through motion sensors, interaction with devices, and in other ways. [5]According to Hill et al. (2015), humans can easily adapt their language to human-chatbot communication, although there are notable differences in the content and quality of these conversations. Results presented by the authors show that people communicate with chatbots for longer periods, but with shorter messages when compared to human conversation. Also emphasizing that human-chatbot communication has impoverished the vocabulary used by humans and increased in the occurrence of profanity.

Chatbots can be categorized in two ways: (i) rules-based, operating by means of specific commands (or keywords), which generally obey well-defined navigation flows and produce targeted conversations; (ii) AI-based, making use of more advanced technologies such as machine learning, NLP, among other artifices to increase its capacity for dialogue and interaction.

The engine is the most important feature of a chatbot. It is responsible for the transformation of natural language into machine-understandable actions. [6]Chatbots engines are usually developed using several Natural Language Processing and Machine Learning models to provide acceptable levels of accuracy (KAR; HALDAR, 2016).

3 . Data sets

Farmers’ information inquiries with the KisanCall Center (KCC)

The Government of India has made all logs of calls to the KCC from January 2015 to September 2017 publicly available. In total, this corpus contains data for 8,012,856 calls. Each call log has 11 fields, including the date and time of the call, location, crop (one of the 306 crop types), query, and the answer provided by the KCC agri-expert

For implementing FarmerBit, our system was restricted to state of karnataka since we have relevant information mapped and dataset contain maximum information than others . There were 35,852 calls related to karnataka in the KCC dataset.

3.1 Sample data:

The data set is downloaded from kissan call center for the available data of calls from karnataka

The Dataset contains following columns

• Sector

• Category

• Crop

• Query Type

• Query Text

• Query Ans

3.2 Data Preparation

The preparation of data has majorly done , we have to do a clean up of data for the better performance of bot , EDA will be used to clean up tha data

4. Exploratory Data Analytics

4.1 Tokenization:

Given a character sequence and a defined document unit having sentences, tokenization breaks it up into pieces, called tokens , perhaps at the same time throwing away certain characters, such as punctuation.

A token is an instance of a sequence of characters in some particular document that are grouped together as a useful semantic unit for processing

4.2 NER: Named Entity Recognition:

While building any conversational bots/ Dialog system one can employ the following approaches to do so

Generative Based
Retrieval Based
Heuristic Based

we use Retrirval Based Approach

All of these approaches rely on NER some or the other way

NER is a subtask of information extraction that seeks to locate and classify named entities in text into predefined categories such as the name of a person, location, organization, contact detail, expressions of time, quantity, monetary value, percentage, etc.

Most of Entities being retrieved falls into following broader category

Numeral : Detecting numbers
Temporal: Detecting with time
Pattern: Understanding patterns and regular expression like email, mobile number etc
Textual: Detecting entities by looking at the dictionary

While NER looks for entities in pre-defined categories (for example, place names or addresses). They might also use a library called a Normalizer, which catches common spelling errors, expands contractions and abbreviations etc..

5. Implementation

5.1. Algorithm and implementation challenges

5.1.1. Extracting Information from the common dataset from the Indian government site was quite challenging, as to prepare the required information to train the data in the tool readable format was bit complex

5.1.2. Repeated questions made the data preparation much harder to find the unique data

5.1.3. We need Domain specific information for a considerable amount to train the NLU model

5.1.4. Implementation methodologies:

The followings ways are implemented with the major representation

A).Bot is to answer the queries based on the input given

B).Possible answers for the Query

C).Methodologies Followed:

5.2. Evaluation metrics

We validated the model by making it to respond the correct answers to the correct questions , in our case we have got almost ~80% accurate answers.

5.3. Final Model Output

With the fully trained model we would able to address the farmer queries related to the provided list of crops. With the provided training data of 1k+ intent data we have confident of ~80% accuracy of our model. With increment with the training data, we have the confident to increase the model accuracy to more than 90%.

5.4 Tools and hardware used to create the model

We used to RASA NLU(A natural language understanding solution) and RASA Core (A dialog management solution) tools to build this chatbot

5.5 Methodology

We used the high-level programming language Python for creating the model in reading the data, feeding it to RASA model and getting the output once it processed.
Intent — Intent is nothing but what the user is aiming for. For example — if the user says “Reserve a table at Cliff House tonight” the intent can be classified as to book the table.
Entity — Entity is to extract the useful information from the user input. From the example above “Reserve a table at Cliff House tonight” the entities extracted would be place and time. Place — Cliff House and Time — tonight.

Stories — Stories define the sample interaction between the user and chatbot in terms of intent and action taken by the bot. Like in the example above bot got the intent of booking the table and entities like place and time but still, there is an entity missing — no of people and that would make the next action from the bot.
Actions — Actions are basically the operations performed by the bot either asking for some more details to get all the entities or integrating with some APIs or querying the database to get/save some information.

This model is based on the LSTM model which is the improvised version of
Recurrent Neural Networks , which is of more convenient way to make the
conversation flow

6. Plain Results -Learning from model building

We found some key learning while building this chatbot model,

 While using such tools for building chatbot there were many version dependencies

between the model and framework  Bot response accuracy were majorly based on the intent definitio

7. Learning from model building

Precise and Localized Answers

Specificity and localization were identified as keys to the information needs of farmers. With the help of agri-experts, the researchers carefully tailored the system responses to local conditions. Participants appreciated such information contents:

Trust

Trust is another key design requirement. In general, participants trusted the responses

Actions — Actions are basically the operations performed by the bot either asking for some more details to get all the entities or integrating with some APIs or querying the database to get/save some information.

This model is based on the LSTM model which is the improvised version of Recurrent Neural Networks , which is of more convenient way to make the conversation flow

provided Six participants even asked the facilitator to write down the recommended medicines, seeds variety, and/or fertilizer with their quantities for them to refer later.

8. Business value(Discussions)

 With the information stored from our end, it can be easily make the farmers to get aware of & act upon the suggestions and get benefited

 This model is based on the LSTM model which is the improvised version of Recurrent Neural Networks , which is of more convenient way to make the conversation flow

 It’s not only help the farmers in turn it improve the yielding & production of agricultural by implying the crop prevention , high yielding methodologies provided by Gov site/experts via the recommendation provided in the chatbot

9. Next steps

We intend to continue to develop this model to increase the relevance for the farmer queries . Our next steps are as follows:

• Increase the datasets that are available for this model to learn from training

• Increase the number of datas and query types that the model trains with so it can apply to a wide range oqueries from the farmer

• Implement an UI for easier interaction

Acknowledgements

This work was towards partial fulfilment of Post Graduate Program in Artificial Intelligence and Machine Learning at Great Learning. We thank Great Learning for the opportunity and support.

We also owe an immense amount of gratitude to all the Deep learning researchers and bloggers who have generously published their work for us to learn from. It would have been impossible for us to move forward with the pace we did without having any access to the wealth of information that they have shared with the community. While they have been mentioned as part of our references, we cannot emphasize enough the importance of their work as the central support for our work as well as furthering the understanding of deep learning frameworks for the community in general.

References

[1]FarmChat: A Conversational Agent to Answer Farmer QueriesMOHIT JAIN ,IBM ∗ Research, India and University of Washington, Seattle, WA, USAPRATYUSH KUMAR,Indian Institute of Technology, Madras, IndiaISHITA BHANSALIQ. VERA LIAO,IBM Research, Yorktown Heights, NY, USAKHAI TRUONG,University of

Toronto, Toronto, ON, CanadaSHWETAK PATEL,University of Washington, Seattle, WA, USA

[2]AgronomoBot: a smart answering Chatbot applied to agricultural sensor networks by GUSTAVO MARQUES MOSTAÇO 1, ÍCARO RAMIRES COSTA DE SOUZA 2 , LEONARDO BARRETO CAMPOS 2 , CARLOS EDUARDO CUGNASCA 1

[3]AGRICULTURE ADVANCEMENT USING ARTIFICIAL INTELLIGENCEKunal Verma1,Dinesh Pabbi2, Avnish Singh Ja