A student of the Crimean Federal University has developed a gesture recognition system

CrimeaPRESS reports:

Denis Shcherbakov, a student of the Tauride College, a laboratory assistant at the FabLab Digital Creativity Laboratory of the Crimean Federal University, has developed a system for sign language recognition. The project includes three gesture recognition systems, one of which is based on a neural network.

I trained the neural network for a week, experimented with different architectures. Knowing the matter, I was able to choose the right architecture, make a dataset for the neural network and teach it everything that is needed— the press service of the university Denis Shcherbakov quotes.

The student used a dataset — a database on the basis of which the neural network is trained. With its help, he taught the neural network to recognize the dactyl alphabet — a letter-by-letter sign language.

For development, a system was used that consisted in comparing coordinates — the knuckles of the hand. Special functions were also used, descriptors of the polygon that the hand made up. Through mathematics and algorithms, a dataset was created with which you can identify a gesture.

Through the camera, the program recognizes gestures and displays letters on the screen. As the student notes, one of the most difficult in the development process was the creation of a dataset.

The peculiarity lies in the fact that the larger the dataset, the better the neural network can be trained, and the better the system will be. In the dataset on the basis of which our system was developed, for each letter there are from one hundred to two hundred examples of gestures from different people, from different cameras and angles. I asked people around me to make gestures of the dactyl alphabet on the camera in order to increase the datasets and teach the neural network— notes Denis Shcherbakov.

At the moment, the system is able to recognize more than thirty characters.

In addition to developing a gesture recognition system, Denis is interested in programming, modeling and cinematography, and also has experience in robotics and 3D printing.

source: KFU press service

News of the Crimea | CrimeaPRESS: latest news and events today