Diatomist Droid: YOLOv3 for Intelligent Detection of Diatoms from Microscope Images
What is diatom? Diatom (Bacillariophyta) is single-celled algae composed of silica walls (valves). Here are some:
These images are all taken under microscope because the cell size of almost all the diatoms ranges from 2 to 500 µm (0.000002 ~ 0.0005 meters). Diatom individuals are not visible to bare human eyes. Diatoms are basic producers in food chain and is one of the most ecologically important algal groups. They are widely distributed in various aquatic environments including oceans, lakes, rivers and streams, as well as backyard ponds. They can also be found in many moist surfaces and soils. After many years of exploratory work by biologists, there are thousands of known diatom species so far and there are many more species which are still unknown. New species are discovered and reported every year. The total number of diatom species is estimated to range from 20,000 to 2 million. That is a full town of diatoms.
The taxonomy of diatoms is in some kind of chaos. You may observe that the names of diatoms in the above images are pretty long and rarely seen. Probably without the help of a dictionary (the thickest one), most people will never know what they mean. Many words are newly created as soon as a species is found. People who exclusively study diatoms call themselves diatomists. Similar with the stars, many diatom species are named by the name of diatomists from all over the world. That is why there are many German, Russian, and many other languages’ letters in the diatom names. However, there are many disputes over the naming of diatoms and the same species might has multiple names because they are discovered by different person at different places. There are many efforts in diatom community to unify their taxonomy but no global agreement is reached yet. There are some generally acceptable taxonomy by the community, like the taxonomy of Academy of Nature Science. The taxonomy updates every several years to include the newly discovered species and make corrections to old ones.
In such circumstances, recognizing diatoms is a huge challenge for everyone, not only because the diatom structure and shapes are various, but also there is no standard taxonomy. The diatoms in Europe are very different from the diatoms in U.S. Only a few people in the world can accurately identify and recognize diatoms using a “relatively correct” name. There are even some certificate program for training people to recognize diatom. There are way many things to learn before you can recognize the hundreds of common diatoms.
YOLOv3: Superman
In Star Wars, when they don’t want to do something (impossible) by themselves, they will order a droid to do it. Since diatom recognition is so hard, is there any “droid” who can take care of this? As the saying goes, “if you have any unrealistic requirements, go to a computer scientist” (quoted from nobody). This time, we find YOLO which seems very ideal to solve the diatom recognition problem. The demo video of YOLO is very cool. It can precisely identify the tie of James bond when 007 is moving. After watched that, my feeling is that YOLO is the superman to save all of us.
Diatom Droid: YOLO version
To make YOLO able to recognize diatoms, there is long way to go. Fortunately, there are many posts online which teach us in details how to train YOLO to recognize customized objects. The most useful post we find is:
Following the steps, we first labelled all our diatom images (~900) which we partially downloaded from the public website of ANSP of Drexel, and also observed some in lab by ourselves. The used labeling tool is labelImg. We used predefined classes generated from the ANSP taxonomy. The labeling takes us a lot of time because it requires diatom knowledge and great patience to distinguish diatoms based on their detailed differences. We must use predefined classes because there are too many diatom kinds and it is insufficient to type them in by hand.
We use the original YOLO neural network implementation in darknet. Git clone all the files of darknet from the Github repository.
git clone https://github.com/pjreddie/darknet.git
We created two configuration files: diatom.data and diatom.cfg. The content of diatom.data is:
classes= 1818
train = data/diatom/train.txt
valid = data/diatom/test.txt
names = data/diatom/classes.txt
backup = backup/
We used a python program to traverse the image list and split all the label diatom images into two sets: training and validation. The list of image names are saved into the train.txt and test.txt respectively.
The diatom.cfg is copied from yolov3-tiny.cfg. According to the above post, we made the following changes to diatom.cfg:
- Uncomment the lines 5,6, and 7 and change the training batch to 8 and subdivisions to 2.
- Change the number of filters for convolutional layer “[convolution]” just before every yolo output “[yolo]” such that the number of filters= #anchors x (5 + classnum)= 3 x (5+1818)= 5469. The number 5 is the count of parameters center_x, center_y, width, height, and objectness Score. So, change the lines 127 and 171 to “filters=5469”.
- Change the number of classes to 1818 in lines 135 and 177.
Then, download a pre-trained model as a start model. We use the recommand darknet53 (download link). Use the following command to kick off the training:
./darknet detector train cfg/diatom.data cfg/diatom.cfg darknet53.conv.74To speed up the training, we could use GPU which will be 500 times faster than CPU:
./darknet detector train cfg/diatom.data cfg/diatom.cfg darknet53.conv.74 -gpus 0The trained weights files will be saved in the backup folder. Within the first 1000 epochs, the weights will be saved every 100 times. After that, it will be saved every 1000 epochs.
After thousands epochs of training, the IOU (intersection over union) has reached over 77%, class accuracy over 97% and object accuracy over 95%. We tested the trained model on some images in the test dataset, and the results accuracy is relatively good considering the scale and size of the training dataset.
A little Discussion
Based on our results, YOLO is a very good tool to help automatically recognize diatoms. However, based on our experiments, some diatom classes which have only a few samples in the training dataset will not be recognized very well. This is understandable as many computer scientists recommend at least 100 samples for each classes. We have 1818 classess which mean we need at least ~180,000 unbiased samples in our training dataset to get a reasonable universal YOLO model which could possibly be used operationally in lab to help diatomist. We consider that as future work of the entire diatom community collaborating with computer scientists.
The author, Richard Hou, is a junior student in Mclean High School. This work is a summer intern project under the supervision of Dr. Ling Ren and Dr. Jensen Ziheng Sun from George Mason University. The text was revised and edited by them.
