[Week 6 — Facade Parsing Using Deep learning]
Team Members: furkan karababa, Onur Cankur, Javid Rajabov
This week, we run our basis model ( Fully Convolutional Network ). To do this, we had to make our own facade dataset in the format of the data as required in the our basis model’s code. And so, in this blog, we are going to explain step by step how we organized our dataset.
- Dataset format which is required in the code
Dataset must have two main folders of files, the first one is for images and the second one is for matlab files. Semantic classes are enumarated by numbers and matlab files of images are consist of two fields: label matrix and class names array. Label matrix is the pixelwise matrix form of the image which each cell of matrix contains numeric value of semantic class which, that pixel of image belongs. Another main point of this format of dataset is that images have to be in the pixel size of 256*256. And our aim was convert our each facade dataset to this format.
2. Converting eTrims and ECP datasets
As we said earlier, we have 3 main datasets for this facade project, and one of them is eTrims database, which includes two datasets. These two datasets differ from each other with respect to number of semantic classes which they have. This dataset has two main folders of files: images and annotations. As it’s supposed to be images folder contains facade images of different places and as you see in the below sample image, annotation folder contains images which are annotated(labeled) form of the original images with certain semantic classes.
In order to convert eTrims datasets to our dataset format, we had to convert annotated forms of the images to matlab format. And another thing we had to do is resizing images to 256*256. We did these two operations by coding and shared codes via github project under create data repository. This code consists of 3 main parts. We iterate our program over all original and annotated images and did 3 main operations over these. In each loop, firstly, program is resizing the obtained image, then iterating over all pixels of annotated image in order to enumarate every pixel according to rgb value with respect to semantic classes and store these values in the numpy matrix data structure. And at the end of the loop, program is saving each numpy matrix and 1d array which contains names of the semantic classes to matlab file. So using this code, we got each resized images and matlab files of these images, and our eTrims facade datasets are ready to use in the basis model code.
We did same operations while converting ECP dataset which is our another main dataset of facade project.
3. Converting Paris Art Deco dataset
The last dataset of our project, Paris Art Deco Facade dataset, also consists of two main folders which the first one contains images and the second contains pixelwise enumareted forms of each image in the txt file formats. While resizing images of this dataset to 256*256, we needed more effort than the previous ones. In this dataset we have labeled forms of images only in txt format, in order to resize them, firstly we convert them to png format according to certain rgb values and then resize them by coding program, we shared this code also in the github project. Sample operation figure is given below.
And finaly, after we got resized labeled images in the png format we did same operations as we did to previous datasets while converting them. So our last dataset is also ready to be used in the basis model code.
4. Running basis model in the Colab
Finally now, we have all of our datasets in the required form. So we run our basis model (fcn8) using these datasets in Google Colab.In order to do this, we needed upload all codes and datasets to Google Drive and import them Google Colab. And we get accuracies for each of datasets, we will analyse details of them in project report.
We are getting ready to prepare video presentation and then we will focus on improving our basis model implementation.
5. References
Etrims dataset — http://www.ipb.uni-bonn.de/projects/etrims_db/
ECP dataset — http://vision.mas.ecp.fr/Personnel/teboul/data.php
Paris Art Deco dataset — https://github.com/raghudeep/ParisArtDecoFacadesDataset
