Year : 2021
Mentors : Priyanka Sharma
Students involved : 18BCE260 - Vishva Patel, 18BCE214 - Aakash Shah
The project is based on providing a better solution for Sign language translation using computer vision and deep learning methods for gesture detection and classification. American Sign Language is one of the most widely used sign languages . One of the prominent issues faced by people with hearing and speech loss is the dearth of people actually knowing sign language as a way to communicate. This barrier frequently impedes communication for people with hearing or speech loss . Only solution available currently is reliance on an in person translator who has experience in sign language and can provide people with proper translation. But even this solutions has its own problem such as :
1. Loss of meaning or context because the translator is not well versed with the sign language
2. Lack of asl translators and people who can translate sign language when needed
This project aims to use state of the art deep learning models used for computer vision to implement a real time video to text sign language translator. The project will majorly deal with three parts :
1. Exploration and analysis of different computer vision methodologies and deep learning models feasible for implementing gesture detection and gathering , cleaning , labelling and pre-processing of the image dataset to be used to train the model.
2. Training the realized model with the gathered dataset and fine tuning the models hyperparameters for optimum precision and recall trade-off.
3. Implementation of the trained model after proper testing over a web app and mobile app for better accessibility.
This project can be implemented as a stand-alone translator for translating sign language to text for anyone to use for better communication.
This project can also be applied in many online video communication platforms as a plugin for giving the ability to people with hearing or speech loss to communicate more efficiently and express their thoughts more naturally.
This project can also be applied for the various public speaking forums , news presentations and miscellaneous formal events where traditionally people have relied on dedicated translators for providing translation.