Final Week — Project LEAFS

Published in

AIN311 Fall 2022 Projects

4 min readDec 30, 2022

Hi everyone,

Last week, we discussed the data augmentation method and provided more data to the model. Also, we discussed that we increased the epoch size but unfortunately, the model returned an early stopping response. This week, we will be discussing improving the model performance based on the experience of the previous versions. Also, we developed a GUI for our project to make user friendly.

Improving the Model Performance

As we mentioned in the previous week, we made arrangements for our data set such as relabeling images, adding classroom photos that we collect from our class and removing images which will make overfit the model. After that, we trained two models the first one contains 1892 images as the training set and 224 images as a validation set, and the other one contains 1954 images as the training set and 224 images as a validation set. Both of the models were set to train in 300 epochs but they returned early stopping at 184 and 284 respectively. The difference between these two data set is, the second one contains more classroom photos that we collected by hand from our class. We thought that adding more classroom photos will make our model more successful. As a result of these models, we chose our seventh model because of its good metrics as shown in Fig 1.

Fig 1. Precision-Recall Curve of Our Seventh Model

Assessment of Lecture Efficiency

We developed a semi-theoretical formula for the assessment of lecture efficiency by giving weights to classes as 1 to positive classes and between 0–0.75 to negative classes such as sleeping = 0, yawning = 0.50, eating or drinking = 0.75 and playing with phone = 0. The reason that we are giving these weights is, we thought that a person who yawns and eats or drinks something is still listening to the lecture but, a bit distracted, a person who sleeps and plays with a phone is fully distracted from the lecture. In Fig 2 we simply showed how the efficiency of the lecture is calculated as a semi-theoretical formula.

As an example calculation shown in Fig 3 for our formula, if we have 4 people in our class sleeping, listening, raising hand and yawning.

Fig 3. Example Calculation

Project LEAFS GUI

As a final task of our project, we created a Graphical User Interface(GUI) for the project. This GUI Contains the detections on the real-time camera frames, the number of class instances based on the real-time detections, and semi-theoretical lecture efficiency.

The images are shown above, as example testings of our project. Contains all different classes and calculates the efficiency according to these detections.

Conclusion

As a result of seven weeks, we finished our project after completing the steps:

Data Collection with different techniques such as web scraping, data augmentation and taking images of our classroom by hand.
Labelling all of these images.
Training models with small changes on data sets.
Choosing a final model based on the results from all models.
Determining a semi-theoretical formula to assess lecture efficiency.
Creating GUI.

Depending on these steps, our project is a comprehensive work of artificial intelligence. In other words, it is a work of Data Science, Data Mining, Computer Vision, Machine Learning, Deep Learning and GUI design.