Emotion Recognition Datasets

Fig 1-Facial Expressions

We express our emotions via facial cues, that can be identified by computers using the machine and deep learning algorithms. Detecting facial emotions is an important research area. This capability to detect emotions is called emotional intelligence. How machines detect emotions? Do they need anything?. Machines need plenty of datasets to learn emotions. There are datasets available as images or videos.

Here, we can have a look at some datasets that can be used for emotion recognition.

Images

01 — AffectNet

AffectNet is one of the popular datasets for detecting facial emotions. The dataset contains around one million facial images. They were collected from search engines with a thousand two hundred and fifty keywords from six languages altogether. Around four-hundred and fifty thousand images were manually annotated by 12 experts.

Fig 2 : Samples from AffectNet Dataset

You can download the dataset here.

02 — Extended Cohn-Kanade Dataset (CK+)

The Extended Cohn-Kanade Dataset (CK+) is used as a test-beds for many algorithms and it is widely used. The dataset contains 5,876 images of a hundred and twenty-three people. These images are labeled with seven emotions. All the images were taken with a constant background.

Fig 3 : Samples from CK+ dataset

You can download the dataset here.

03 — FER-2013

The FER-2013 is a widely used emotion dataset. The images are labeled with seven emotions: neutral, happy, surprise, sad, fear, disgust, and anger. The dataset contains 28,000 of training data, 3,500 of validation data, and 3,500 of test data. The images were collected from google. The images were collected in a way that they vary in the pose, age, and occlusion.

Fig 4 : Samples from Fer-2013

You can download the dataset here.

04— EMOTIC

EMOTIC database of images of people taken from the real environments, and annotated with their apparent emotions. They are labeled with 26 categories of emotion. They were labeled using Amazon Mechanical Turk (AMT) platform. The dataset contains 18,313 images and 23,788 annotated people. Some images were collected from Google as well.

Fig 5 : Samples from EMOTIC dataset

You can download the dataset here.

05 — Google Facial Expression Comparison Dataset

Google Facial Expression Comparison Dataset is an emotion dataset that is used on a large scale. The dataset contains triplet images with labels. Each triplet is labeled by the top six raters. Here, the dataset helps in identifying which of the two faces are similar in emotions. The dataset is used mainly for summarizing albums, classifying emotions, etc.

Fig 6 : Samples from Google Facial Expression Comparison Dataset

You can download the dataset here.

Videos

01 — CASME

CASME is a dataset of spontaneous micro-expressions. The images were annotated based on psychological studies. The dataset contains 195 micro-expressions that were taken at 60fps. The dataset samples were taken from thirty-five participants. Every clip is with a minimum length of 500ms.

You can download the dataset here: CASME, CASME II and CAS(ME)².

02 — SAMM

SAMM is a database of spontaneous micro-facial actions. The images represent a good distribution among different ethnicities and age distribution. Therefore, the dataset addresses the issues in limitations for demographical diversity. The dataset is annotated with seven categories of emotion and is recorded 200fps.

You can download the dataset here.

03 — SMIC

SMIC dataset contains spontaneous micro-expressions of 16 participants and 164 spontaneous micro-expressions. There are three datasets included in SMIC: SMIC-HS, SMIC-VIS, and SMIC-NIR. Videos were recorded with 100 fps (High Speed — HS), and the last ten subjects were recorded with cameras of 25 fps of both visual (VIS) and near inferred (NIR) light ranges.

You can download the dataset here.

04 — BAUM

BAUM is a video-based dataset. The dataset contains audio-visual clips of different languages and they are annotated. The clips represent real-world scenarios like different poses, lighting conditions, and subjects of varying ages. An image-based dataset is created with the peak frames from each clip.

You can download the dataset here: BAUM and BAUM2.

Miscellaneous

Apart from images and videos there are other 3D datasets like BU-3DFE, BU-4DFE, Bosphorus, and BP4D.

With the current increase in technology and the huge size of dataset, it is more feasible to identify emotions. But, we need datasets that could represent diversity in geography, ethnicity, gender and age limit. The above mentioned datasets may provide a way to get required datasets for emotion recognition related tasks.

Thankyou for your time to read my article.

References

1. https://analyticsindiamag.com/top-8-datasets-available-for-emotion-detection/
2. Mollahosseini, Ali, Behzad Hasani, and Mohammad H. Mahoor. “Affectnet: A database for facial expression, valence, and arousal computing in the wild.” IEEE .
3. Lucey, Patrick, et al. “The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression.”, IEEE, 2010.
4. Giannopoulos, Panagiotis, Isidoros Perikos, and Ioannis Hatzilygeroudis. “Deep learning approaches for facial emotion recognition: A case study on FER-2013.” Springer.
5. Kosti, Ronak, et al. “EMOTIC: Emotions in Context dataset.”, IEEE .
6. Yan, Wen-Jing, et al. “CASME database: a dataset of spontaneous micro-expressions collected from neutralized faces.”
7. Davison, Adrian K., et al. “Samm: A spontaneous micro-facial movement dataset.”
8. Erdem, Cigdem Eroglu, Cigdem Turan, and Zafer Aydin. “BAUM-2: a multilingual audio-visual affective face database.”