About the project
This project was a university assignment where the task was to develop a Large Language Model (LLM) capable of detecting emotions in sentences. It was a group project, and my primary focus was on dataset preparation and working with transformer models.
The emotions used during the project:
• Happiness
• Sadness
• Anger
• Surprise
• Fear
• Disgust
Since some of the datasets we used contained more than these six emotions, we applied the study by Ekman & Friesen to map all emotions to the six primary categories.
Dataset
To develop an accurate LLM (or any machine learning model), having a large, high-quality dataset is essential. Therefore, we used several open-source datasets, including: - GoEmotions - Friends emotion-labeld dialogues - MELD dataset - CARER dataset - Affective Text - Daily Dialogue - Affect data
In addition to these datasets, we used OpenAI’s language models to generate sentences along with their corresponding emotions. To further enrich our dataset, we used transcriptions from the TV series Survivor and had OpenAI's model annotate the emotional content of those sentences.
Additional information about the dataset
Approach
We experimented with different algorithms to tackle the problem, but state-of-the-art transformer models provided the best results by far. The highest performance was achieved using the RoBERTa transformer model, with which we achieved an F1 score of 0.91.
Result
Results table
| Sentence | Prediction |
|---|---|
| I sprained my ankle | Fear |
| I love waking up to a warm cup of coffee, knowing it's finally weekend. | Joy |
| When I do not get something right the first time, I get frustrated. | Anger |
| A clear blue sky lifts my mood. | Joy |
| I told you a thousand times before to leave me alone and mind your own business! | Anger |
| Paul started to resent Lola after all the broken promises. | Anger |
| The snake gave everyone a fright. | Fear |
| Hearing footsteps behind her in the deserted street made her heart race with fear. | Fear |
| My neighbor got nervous when there was a break-in at his house last night. | Fear |
| Insects are crawling under my skeen | Fear |