How To Build A Real-Time Emotion Detection Web App💻

Using Convolutional Neural Nets to detect up to 7 emotions

Photo by Matthew Fournier on Unsplash

Before we start, make sure you are familiar with the prerequisites.

Prerequisites

• Python
• Tensorflow + Keras
• Flask
• OpenCV

Here’s a preview of how the web app will look like once we finish😄

Here is the GitHub link for the project.

I’m not very expressive am I? 😐

1. Setting up

First, create a directory to work in and switch to it.

C:\Github>mkdir emotion-detection && cd emotion-detection
C:\Github\emotion-detection>

Now, we need to download the dataset and copy it into the directory.

Link for download: fer2013 dataset

Next, let’s set up a virtual environment and install the requirements.

C:\Github\emotion-detection>virtualenv venv1
C:\Github\emotion-detection>call venv1/Scripts/Activate
(venv1) C:\Github\emotion-detection> 

Here is the list of libraries we’ll need.

(venv1) C:\Github\emotion-detection>pip install -r requirements.txt

This is how the folder structure will look like once we finish.

Directory Structure:

• templates : folder containing HTML files
• app.py : .py file containing the code for the web app
• emotion_detection.py : neural network implementation
• fer.h5 : saved model architecture
• fer.json : saved model weights

After we’re done installing the packages, we can start training the model ^_^

Image Source: Google images

2. Training the model

Create a file emotion_detection.py to build and train the model. After training, we’ll save the model’s architecture and weights so that we can load it directly to get results.

Step 1 is to import all the needed libraries.

Next, let’s split the data into training and testing sets. Here, we have 7 categories of emotions and hence num_classes is 7.

Next, we perform some data augmentation on the images to get better results.

This step augments the data by performing operations such as rotation, zooming, rescaling, flipping the image, etc.

Awesome! Now its time to build the neural net‼

You can tweak the parameters or build your own network with a different number of layers, activation functions, etc.

Neural Net Architecture

All right! The next up is to fit the model to the dataset.

We’ve all been there at one point 😐

Note: If your model is taking too long to train then use platforms that allow access to free GPU or TPU for training (eg: Google Colab, Kaggle)

In the last part of the emotion_detection.py file, we’ll save the model’s architecture and weights.

After running emotion_detection.py, we should be having fer.json and fer.h5 in the directory.

3. Building the web app

We’ll use flask framework to build a lightweight web application. Before that, we’ll need OpenCV’s haarcascade_frontalface_default.xml to detect faces. Clone OpenCV’s repository from GitHub and copy the .xml file into the working directory.

Now we’ll code app.py . The first step here is to import the libraries.

Next, we’ll load the previously trained model into app.py

The webcam feed input is denoted by 0 and hence the parameter for cv2.VideoCapture() is 0. Next, we grab the frames from the video feed one by one and transform them to fit the input dimension size of our neural net. Then we make predictions on the image frame and return the frame along with a text box around it, with the emotion labeled on the box!

Code for detecting faces by Dhanush M via source.

Author : Dhanush M

Whew! Just one more step and we’ll be done🔥

4. Adding static files(HTML,CSS) to Templates folder

We’ll need to put the HTML files in the templates folder since that’s where Flask automatically looks.

That’s it!! Now we can run the code to test it!!

• Run app.py

(venv1) C:\Github\emotion-detection>python app.py

• Enter localhost address in the browser

All right, one last meme 😉

All Hail The King👑