All Projects

EmotionLens

Real-time face detection, emotion recognition, and multi-object tracking from a webcam feed — RetinaFace + a residual masking network + SORT.

Computer VisionArchivedApache 2.0RetinaFace7 (FER2013)SORTYOLOv5-nano
7
Emotion classes
RetinaFace
Face detector
224×224
Classifier input
SORT
Tracker
Source Code

Overview

Pipeline

Detection, classification, and context in one loop

Faces are detected in batches with RetinaFace (via the batch_face package, GPU-only), then each face is classified by a resmasking_dropout1 network — a Residual Masking Network variant adapted from phamquiluan/ResidualMaskingNetwork — trained on FER2013's 7 emotion classes: angry, disgust, fear, happy, sad, surprise, neutral.

A YOLOv5-nano pass runs alongside, filtered to person/cell-phone/laptop, to add engagement context around the faces rather than feed the emotion classifier itself.

Angry
Disgust
Fear
Happy
Sad
Surprise
Neutral
Stability

SORT smooths the flicker

Per-frame emotion predictions flicker on their own, so a vendored SORT (Simple Online Real-Time Tracking) implementation tracks each face across frames and takes a majority vote over its tracked history for the final displayed label. No FPS or accuracy numbers are published for this build — 'real-time' describes the design intent, not a measured benchmark.

Method

Every frame runs through four stages: detect faces, classify each face's emotion, detect surrounding objects for context, and track faces across frames to smooth the label.

  1. 01
    RetinaFace

    batched GPU face detection

  2. 02
    ResMaskingNet

    7-way emotion classification

  3. 03
    SORT

    track + majority-vote label

Results

Fig. 1Live detection — face box, emotion label, and tracking ID

Stack

RetinaFace (batch_face)GPU face detectionResMaskingNetemotion classifier, FER2013YOLOv5-nano (ultralytics)person/phone/laptop contextSORTmulti-face tracking + voteOpenCVcapture + overlay rendering

Reproduce it

Local (GPU required)
$ git clone https://github.com/baselhusam/EmotionLens.git
$ cd EmotionLens
$ pip install -r requirements.txt
$ python main.py
Back to all projectsView source →