Smart trash bin with waste classification

1. Introduction

Nowadays, the global warming issue is one of the biggest problems in our world. The major changing might be too far for us to collaborate. But at least one thing that everyone can give a hand to this problem is sorting recycle waste. This project proposes a device that makes sorting recycle waste way easier. This device can automatically sort a recycled waste for everyone!

Figure 1. Waste sorting

2. Model development

2.1 Datasets

In this project, we use the open-source dataset from Kaggle as the main dataset and adjust them by adding external images and removing redundant images in order to improve the quality of the dataset for real use case scenarios.

Totally, we have 2892 images with 5 types of waste as figure 2 and 3.

Figure 2. Type of trash in datasets

Figure 3. Example of image in datasets

To prepare the dataset, we split our dataset into train, validation, and test data by this proportion.

• Train 70%
• Validation 20%
• Test 10%

With the great help of the TensorFlow Lite Model Maker library, There are functions that help us to import our dataset and split the data easily. For more detail, you can see some examples via this link.

2.2 Efficientnet

In this project, we use a Transferred learning technique for training our model. This technique is provided in the TensorFlow Lite Model Maker library. There are various pre-trained models supported in this library. To choose the best model, we need to consider the tradeoffs between various models. In this project, because of limitations of our device’s memory, we expect a model with good accuracy, low latency and small size . So, Efficientnet is a great candidate for our case.

EfficientNet is a convolutional neural network architecture developed by Google’s team. They propose a compound scaling method for increasing the efficiency and accuracy of the model. This method will uniformly scale network width, depth, and resolution with a set of fixed scaling coefficients. So, the strength point of EfficientNet is high accuracy but low parameter size.

Figure 4. Latency (ms) vs. Accuracy (Top 1)

Figure 5. Model Size vs. Accuracy (Top 1)

As we can see in the comparison graph, The EfficientNet model can perform with high accuracy even if their size are small. But choosing a model that is too accurate is not always good in every situation. Because of too many parameters in a high-accuracy model ,This factor can make our model overfit.

For this project we tried to train models with EfficientNet-Lite4 at the first time in order to get the high accuracy model. But when we test our model with real scenarios, The result from EfficientNet-Lite4 is worse than EfficientNet-Lite0 because of overfitting in EfficientNet-Lite4. So finally we choose EfficientNet-Lite0 as a main model for this project.

2.3 Model training

Because of the limitation of devices In this project, Our device supports the Tflite model. Tensorflow lite model maker is a great tool for developing models in Tflite format. The TensorFlow Lite Model Maker is a library that simplifies the process of training a model using a custom dataset. It uses transfer learning to reduce the amount of training data required and shorten the training time and it can export our model into Tflite format directly. So, this library is very useful and convenient for developing a model for deploying in small devices.

For how to use this library, you can see and follow the steps from this example.

Figure 6. TensorFlow Lite Converting

2.4 Result

We train our model using the Tensorflow lite model maker library with our dataset. For the structure of the model, we choose Efficientnet-lite0 as the main structure and set the parameter as a default setting from Efficientnet-lite0 model.

We trained our model for 30 epochs and got 0.93 accuracy and 0.59 loss from testing with test data and got the learning graph as follows.

Figure 7. A history log of model accuracy

Figure 8. A history log of model loss

Moreover, we test our model by predicting the image from test data and indicate the predicted result as below figure.

Figure 9. Result of model prediction

As the predicted result, we can see that there are some mistakes in predicting between plastic and glass, trash and paper. The cause of this problem might be the similarity of those objects that can make a model predict wrong sometimes.

3. Hardware

3.1 Electrical components

In order to make a smart trashcan portable, an embedded system with high processing power is required. Moreover, an actuator, camera, and sensor are also necessary to make this system possible. Below is the list of the hardware that we chose for this project.

• Main Computer

1. NanoPi M4 V2 Single Board Computer (SBC)
2. Google Coral TPU

• Servo Controller

1. STM32F103 Dev board

• Peripherals

1. MCAM400 MPIP Camera
2. MicroServo SG90
3. IR Proximity Sensor Module

The reason that we need a dedicated MCU for controlling the servo is that the NanoPi SBC cannot provide a fast enough PWM signal for controlling the servo. MCU and SBC are communicating with each other via UART

Figure 10. Hardware diagram

3.2 Mechanical component

Due to limited time constraints and budget, we decided to make a trashcan system that can be separated trash into two bins. The mechanism of the trashcan is very simple, it has a pivotable tray in the middle that can be rotated left or right. Then, the trash will fall down in either left or right bin according to the program.

We designed a servo mount and a rotating shaft using Fusion 360 CAD software. Then, 3D print these parts using an FDM 3D printer. After that, assembly of all parts on the plastic bin using some bearings, nuts, and bolts.

Figure 11. Designed servo mount and a rotating shift

Figure 12. Part of designed hardware by 3D printer

Other components and processing units are bundled together and mounted at 45° angle from the trash loading tray as below in figure 13.

Figure 13. Combining a processing unit with other component

4. Software

4.1 Overall

Our SBC is a Linux computer so we chose to develop our project using Python language. Each class is responsible for one task and it also runs on each individual thread.

Main libraries and frameworks that are being used in this project.

• TensorFlow and TensorFlow Lite
• OpenCV
• Flask
• MQTT
• WiringPi

Figure 14. Software diagram

As mentioned earlier, our SBC has built-in Google Coral Edge TPU. In order to utilize the performance of TPU, our machine learning model has to be converted into Edge TPU optimized TensorFlow Lite model.

Figure 15. Tensorflow Lite Convertor

Another part of the system that requires coding is the servo controller part. We picked the Arduino platform as a development tool for our STM32F103 dev board. Arduino platform has a variety of ready-to-use libraries which is very suitable for limited time constrain like this project.

The code is very simple, it just receives the data from UART and set the servo angle corresponding to the input data.

4.2 Dashboard

Figure 16. Dashboard data flow

NETPIE

NETPIE is an IoT cloud-based platform-as-a-service that helps connect your IoT devices together seamlessly by pushing the complexity from the hands of application developers or device manufacturers to the cloud.

MQTT Method

MQTT is a publish/subscribe protocol that allows edge-of-network devices to publish to a broker.

Figure 17. MQTT Architecture

Figure 18. Dashboard Interface

Our dashboard can show the following data

• Counting each type of the trash (Plastic, Glass, Paper, Metal, Trash)
• Status of a device (Online/Offline)
• Total pieces of trash in the trash bin
• Percentage of combustible and non-combustible trash

5. Result

The developed model for image classification can classify the type of each trash by the camera device working along with a servo motor which can receive the result of model prediction and then split the trash into the different sides of the trash bin. The dashboard in NETPIE can visualize the real-time result of the whole system by showing a number of totals and each type of trash.

6. Discussion

1. Various object in Trash class

According to our model, the trash class is the most inaccurate class because the objects in this class are quite various. There are many object’s shapes and many characteristics in this class. That can make our model confused when it predicts the object in trash class. In our opinion, we can solve this problem by separating an object in a trash class into a new specific class to make our model predict clearly.

2. Similarity of glass and plastic

Because our dataset has images in glass and plastic classes that are quite similar in shape and color. That can make our model predict wrong for these classes. In our opinion, we can solve this problem by increasing the images of these two classes with more features of each class to make a model that can predict them better.

3. Budget limitation

Because of the budget limitation of this project. Even though we can classify waste into 5 classes, we don’t have enough servo to separate waste into 5 types. For improvement, we can increase a servo and make a hierarchy to make a bin that can separate more classes.

4. Dashboard Designing

The dashboard interface is not adjusted with the user experience before. So, if any company or department needs a dashboard for visualizing the result of sorting recycled waste must change the interface to match their requirement.

References

• Model comparison

https://blog.tensorflow.org/2020/03/higher-accuracy-on-vision-models-with-efficientnet-lite.html

• EfficientNet model

https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html

• Garbage classification dataset

https://www.kaggle.com/datasets/asdasdasasdas/garbage-classification

• example code of tflite model maker for image classification

https://www.tensorflow.org/lite/tutorials/model_maker_image_classification

• TensorFlow models on Edge TPU

https://coral.ai/docs/edgetpu/models-intro/#compatibility-overview