Real-Time Face and Hand Landmark Detection Using OpenCV and Machine Learning

Abstract

This paper presents a real-time face and hand landmark detection system built using OpenCV and machine learning techniques, optimized for applications in human-computer interaction and gesture recognition. The system employs MediaPipe for initial landmark extraction combined with a lightweight CNN classifier for precise pose estimation and gesture classification. Trained on the Hand Gesture Recognition Database and custom augmented datasets, the model achieves 96.5% accuracy in detecting 21 hand landmarks and facial keypoints across diverse lighting conditions and skin tones. Deployed on resource-constrained devices, it processes 30+ FPS with minimal latency, making it suitable for real-time applications like sign language translation and augmented reality interfaces. Experimental results demonstrate robustness against occlusions and viewpoint variations. This work provides an accessible, high-performance solution for undergraduate-level computer vision projects with practical deployment capabilities.