A Research System for Facial Asymmetry Analysis and Stroke Risk Assistance Based on Images and Videos
In ambulance and emergency scenarios, this project leverages image- and video-based facial analysis to automatically extract facial motion and asymmetry features (key regions such as the mouth, eyes, and nose). By combining geometric features, deep learning models, and temporal trend analysis, the system provides auxiliary stroke (Stroke) risk signals to support early clinical judgment.
In the early stages of stroke, patients often exhibit observable facial abnormalities, including:
- Mouth corner drooping
- Asymmetric eyelid closure
- Nasal wing or facial midline deviation
- Abnormal changes in facial movements over time
In non-hospital environments such as ambulances, physicians typically rely on visual inspection and clinical experience . However, these signals are often:
- Highly subjective
- Difficult to quantify
- Hard to track consistently over time
The core objective of this project is:
To transform visually perceptible but hard-to-quantify facial asymmetry phenomena into computable, comparable, model-ready, and interpretable structured features .
Rather than being developed in a single step, this project follows a progressive three-phase approach , balancing medical plausibility and engineering controllability:
| Phase | Objective | Core Methods |
|---|---|---|
| Phase 1 | Interpretable feature modeling | Facial landmarks + geometric rules + lightweight classifiers |
| Phase 2 | Deep model exploration and reflection | CNNs (e.g., ResNet) + region constraints |
| Phase 3 | Dynamic trend analysis | Temporal features + time-series modeling |
The datasets used in this project are primarily obtained from Kaggle , and consist of the following components:
A subset of samples:
- Facial images or videos labeled as Stroke / Non-Stroke
- Limited in size and not strictly collected from clinical medical environments
Another subset (the majority):
- Artificially constructed symmetric / asymmetric facial datasets
- Including intentionally created mouth drooping, eye asymmetry, and facial deviation
- Mainly used to validate the model’s ability to perceive and distinguish facial asymmetry
Corresponding directory structure:
data/kaggle/2/ ├── stroke_data/ └── noStroke_data/
It is important to explicitly acknowledge that:
- The current dataset is not a purely medical Stroke / Non-Stroke dataset
- When trained on full-face images, deep learning models may:
- Learn visual symmetry differences
- Rather than true stroke-related medical features
As a result:
High accuracy does not imply medical validity.
This recognition directly motivated the methodological reflection and strategy adjustment in Phase 2 .
The goal of this project is not to achieve high leaderboard scores on Kaggle datasets, but to explore:
How to progressively approximate real medical scenarios through feature selection, region constraints, and temporal modeling under realistic data bias conditions.
In medical applications, interpretability and verifiability are critical.
Therefore, the project deliberately avoids deep learning in the initial phase and instead prioritizes:
- Interpretable geometric features
- Facial region analysis aligned with medical intuition
- Use MediaPipe to extract facial landmarks
- Perform distribution analysis between Stroke and Non-Stroke samples
- Identify facial regions and features with the strongest discriminative power
Through experimental and statistical analysis, the following regions were identified as most discriminative:
Mouth
- Height difference between left and right mouth corners
- Distance difference from mouth corners to the facial midline
- Overall mouth tilt angle
Eyes
- Difference in eye opening degree
- Eyelid height asymmetry
- Eye position relative to the facial midline
Nose
- Nasal bridge deviation
- Distance from nose tip to the facial midline
Corresponding modules:
- Facial landmark extraction:
face_keypoint_extractor.py - Facial completeness assessment:
face_completeness_judge.py - Symmetry computation:
face_symmetry_comparer.py - Functional feature extraction:
face_functional_feature_extractor.py
Outputs:
facial_asymmetry_symmetry.csvfacial_asymmetry_functional.csv
A lightweight classifier is trained on these features, and its performance is evaluated accordingly.
In the second phase, CNN models (e.g., ResNet) were trained directly on full-face images.
Experimental results showed:
- Training and testing accuracy approaching 100%
Further investigation revealed that the model primarily learned:
- Visual-level symmetry vs. asymmetry cues
- Secondary features such as lighting and texture
rather than genuine stroke-related medical characteristics.
This validates a critical conclusion:
Under dataset bias, high accuracy does not equate to medical effectiveness.
- Abandon full-face inputs
- Crop medically relevant facial regions:
- Eyes
- Nose
- Mouth
- Train deep models using only these regions
Related modules:
deep_asymmetry_models/ predictive_models/
Meanwhile, higher-quality medical datasets (e.g., Toronto stroke image/video datasets) are being investigated.
Stroke is not a static condition , but a dynamic process.
Therefore, relying solely on single-frame images is insufficient.
- Perform frame-by-frame or second-level analysis on videos
- For key regions (mouth, eyes, nose), track:
- Angle changes
- Distance variations
- Positional displacement trends
- Determine whether these changes continuously exceed predefined thresholds
- Estimate temporal Stroke / Non-Stroke risk trends
temporal_asymmetry_analysis/ temporal_mouth_asymmetry_detector.py
Future work (under higher-quality data conditions) includes:
- Temporal Graph Neural Networks (DGNN)
- More fine-grained temporal modeling techniques
FACIALASYMMETRYDETECTOR/ ├── backend/ │ └── src/ │ ├── data/ # Images / videos / Kaggle datasets │ ├── extractor/ # Landmark extraction / frame sampling / quality checks │ ├── metrics/ # Symmetry & functional features │ ├── pipeline/ # End-to-end pipelines │ ├── predictive_models/ # Classification / prediction models │ ├── deep_asymmetry_models/ # CNN-based models │ ├── temporal_asymmetry_analysis/ │ ├── data_io/ # CSV I/O and aggregation │ ├── utils/ │ ├── main.py # Entry point │ └── test/ ├── frontend/ ├── requirements.txt └── README.md
facial_asymmetry_symmetry.csvfacial_asymmetry_functional.csvfacial_asymmetry_functional_shap.png(model interpretability analysis)
- ✅ Clear medical motivation (emergency stroke risk assistance)
- ✅ Clean, extensible engineering architecture
- ✅ Highly interpretable features, avoiding pure black-box modeling
- ✅ Coverage of rule-based methods, machine learning, deep learning, and temporal analysis
- ✅ Solid foundation for future medical-grade datasets and models
This project is intended for research and engineering validation purposes only and does not constitute medical diagnostic advice.
The system outputs risk signals , not clinical diagnoses.
All results require further validation using real clinical data and medical environments.