AI.Tomate — Drones, CV and AI for agriculture

Welcome to the fascinating world of artificial intelligence development! Today, I’m excited to introduce you to AI.TOMATE, an innovative project developed during the Saturday bootcamp using 4.0 technology for agriculture. This project combines technology, creativity, and a touch of humor to tackle a common problem: monitoring and caring for tomato plants.

What is AI.TOMATE?

AI.TOMATE is an intelligent system designed to help tomato growers optimize the care of their plants. Using a drone-mounted camera and advanced computer vision techniques, AI.TOMATE monitors tomato plants to detect, classify, and locate diseases. The data is sent to a large language model (LLM) via an API for detailed insights, which are then displayed in real-time on a Streamlit application.

1. Drone with Camera and Computer Vision
   AI.TOMATE uses a high-resolution camera-equipped drone to capture images of tomato plants. These images are processed using computer vision techniques to detect signs of diseases.

2. AI Platform and API
The AI platform is the brain behind AI.TOMATE. It uses machine learning algorithms and an LLM to analyze images and provide detailed information. Key components include:

Prediction Model: Trained with images of diseased plants to identify and classify diseases.
LLM: Provides additional information and recommendations based on analyzed data.
API: Acts as an intermediary between the computer vision system and the LLM.

3. Streamlit Application
To facilitate access to information and recommendations, AI.TOMATE includes a Streamlit application that allows users to:

• View real-time data.
  - Receive notifications and alerts about plant status.
  - Access personalized recommendations.
  - Visualize the location of diseased plants within the plantation.

Project Development During the Bootcamp

During the Saturday bootcamp, the AI.TOMATE team followed an agile approach to develop the project. Here’s a breakdown of the key stages:

1. Research and Planning
   The first step was researching tomato cultivation and the specific needs of the plants. System requirements were defined, and development stages were planned.

2. Data Collection
The team began collecting images of tomato plants, sourcing from urban gardens and open datasets. This phase was crucial for obtaining real data to train the AI model.

3. AI Model Development
Using computer vision and machine learning techniques, a prediction model was developed and trained. A combination of classification algorithms and neural networks was used to ensure model accuracy. Additionally, an LLM was integrated to provide detailed information and recommendations.

4. Streamlit Application Implementation
A Streamlit application was developed to display data and recommendations in real-time. The app was integrated with the AI platform and API to enable visualization of diseased plant locations and access to additional information.

5. Testing and Optimization
Finally, extensive testing was conducted to ensure all components functioned correctly. The system was also optimized to improve recommendation accuracy and user experience.

Installation and Configuration

To update and install the entire AI.TOMATE project, follow these steps:

Technologies

Streamlit:

A framework for designing a minimalist, lightweight, and high-performance frontend.

FastAPI:

A modern, high-performance web framework for building APIs with Python, based on OpenAPI and JSON Schema specifications.

Uvicorn:

An ultra-fast ASGI server for Python, designed to run asynchronous applications.

PIL:

Python Imaging Library offers powerful image processing capabilities.
TensorFlow:

An open-source library from Google for machine learning and AI, specializing in training and inference of deep neural networks.

Numpy:

A Python library providing support for multi-dimensional arrays and matrices, along with a collection of high-performance mathematical functions.

Ollama:

API service for LLM.

LLaVA:

A multimodal model combining a vision encoder and Vicuna for general-purpose visual and linguistic understanding, achieving impressive chat capabilities that mimic the spirits of GPT-4 multimodal.

New in LLaVA 1.6:

- Increased input image resolution up to 4 times more pixels, supporting resolutions of 672x672, 336x1344, 1344x336.
- Improved visual reasoning and OCR capability with an enhanced visual instruction tuning data mixture.
- Better visual conversation for more scenarios, covering different applications.
- Enhanced world knowledge and logical reasoning.

Clone the repository:

git clone https://github.com/josequijado/tomato.git

Install Dependencies:

Navigate to the cloned repository directory and run:

cd tomato
pip install -r requirements.txt

Configure Git LFS:

To handle large files, configure Git LFS:

git lfs install
git lfs pull

Run the API:

Start the FastAPI API:

uvicorn api:app — reload

Capture Image:

To capture an image with the drone, run:

python capture.py

The image will be automatically sent to the API, and the result will be displayed in the web app.

Update models and weights:

cd ModelsAndWeights
python update_models.py

Configure the API:

Edit `ConexionApi.py` to ensure the API endpoints and authentication keys are correctly configured.

Run the Streamlit application:

streamlit run capture_streamlit.py

The Team

Fernando Jose Moreno Molina
Isaac Ripoll Asenjo
Susana Aranda Carmena
Jose Quijado
Víctor Manuel Arbiol Martinez

References

Urban gardens that collaborated:

Tablao Compostura (Las Tablas)
Cuña verde Moratalaz
Huerto AVV Carabanchel Alto
Utopía Vallecas

Repository and Colab

Colab: [tomates.ipynb]
Colab2: [modeloTomates_version2.ipynb]
Colab Fine Tuning: [modeloTomates_version2-FineTuning.ipynb]
GitHub: [Identificación de enfermedades en plantación de tomateras]

Conclusion

AI.TOMATE is an excellent example of how artificial intelligence can be applied to agriculture to improve crop efficiency and yield. This project not only highlights the technical capabilities of the team but also their creativity and sense of humor. We hope AI.TOMATE inspires others to explore the potential of AI in agriculture and other fields!

Thank you for reading about AI.TOMATE! Remember: taking care of your plants has never been so fun and high-tech. Until the next harvest!