You are here

Belarusian programmers creating gesture interpreting app

A team of 20 Belarusian programmers has been working on a very prospective start up for six months. The team is trying to create an app which will be able to define gestures and then interpret them into words.

No one had before created such useful apps. It is not surprising that the project found its investor fairly quickly. The CEO of the project told what his team is currently working on.

“We first wanted to create something like GameDev does, than we started developing the app for the IOS function. With the help of tracking we managed to start developing the program for gestures. Then our team was asked to create a project of one more AR-game for a mobile platform with a virtual character with whom we could cooperate with the help of gestures. Currently we are working on creating a full defining of gestures for sign language”.

Ilya Lesun also emphasized that deaf people are not in the best of positions in life. They can’t hear those who speak, and those who speak can’t understand them.

“We can help many people by just creating an app which will translate gestures into text, voice, and vice versa”.

So, how will the app work? The app will use an ordinary RGB-camera, which any smartphone has. Through a certain filter the necessary data is then collected. After that a neural system will analyze it and a certain command will be launched.

“Till this moment so far we have been working on the math side of the project, now we will be moving on to the neural system. It will certainly improve the quality of our work”, explains Anton Milyutin, the data analyst of the team.

The app can already identify eyes, lip points. The aim of the app is to take in the emotions, to form phrases with the help of these emotions. The current aim of the app’s developer’s is to create a platform where deaf people would be able to teach new gestures to the system. Then a smartphone camera should be able to interpret a gesture into text.

At the moment nobody has developed anything like this app. Nevertheless, Google, Apple, Amazon and Microsoft are working in this direction.

Photo: dev.by

Belarusian programmers creating gesture interpreting app