Real-Time Pose Estimation Using Mediapipe for Gesture and Sentiment Analysis

Abstract

Signs and Visual Learnings are considered as the easiest ways to learn and interact with your surroundings and people. These interpretations are performed by the changes in direction of hand landmarks, face landmarks, and body landmarks. Technological advancement in the field of computer vision, the possibility to predict these tasks will progress through the combination of image processing, deep learning, and machine learning techniques. In this research, we will learn how to leverage Mediapipe to estimate both facial and body landmarks. With the data we will then be able to build custom pose classification models that allow you to decode what a person might be saying with their body language with fine grain accuracy. While learning about the project, we can also customize the suits based on the needs. These estimations are low -dimensional based on skeleton poses. To predict the whole notion and description of the body and face in real time, we make a model using the real-time pipeline. We will then evaluate the routine of our project on a dataset preprocessed by us and show it achieve high accuracy in decoding body language