Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion

Aurelijus Vaitkevičius; Mantas Taroza; Tomas Blažauskas; Robertas Damaševičius; Rytis Maskeliunas; Marcin Woźniak

doi:10.3390/app9030445

Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion

Aurelijus Vaitkevičius
, Mantas Taroza
, Tomas Blažauskas
, Robertas Damaševičius
, Rytis Maskeliunas
, Marcin Woźniak

Department of Artificial Intelligence Modelling (RMS/RMS2)

Kaunas University of Technology

Research output: Contribution to journal › Article › peer-review

110 Citations (Scopus)

Abstract

We perform gesture recognition in a Virtual Reality (VR) environment using data produced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) hand model by recognizing and tracking user's hands. From this model, the Leap Motion application programming interface (API) provides hand and finger locations in the 3D space. We present a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification (HMC) algorithm. We have achieved the gesture recognition accuracy (mean ± SD) is 86.1 ± 8.2% and gesture typing speed is 3.09 ± 0.53 words per minute (WPM), when recognizing the gestures of the American Sign Language (ASL).

Original language	English
Article number	445
Journal	Applied Sciences (Switzerland)
Volume	9
Issue number	3
DOIs	https://doi.org/10.3390/app9030445
Publication status	Published - 28 Jan 2019

Keywords

Data mining
Leap motion
Machine learning
Pattern recognition
Virtual reality
gesture recognition

ASJC Scopus subject areas

General Materials Science
Instrumentation
General Engineering
Process Chemistry and Technology
Computer Science Applications
Fluid Flow and Transfer Processes

Access to Document

10.3390/app9030445

Cite this

@article{14da001ca6d74405b4d6300a7c723669,

title = "Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion",

abstract = "We perform gesture recognition in a Virtual Reality (VR) environment using data produced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) hand model by recognizing and tracking user's hands. From this model, the Leap Motion application programming interface (API) provides hand and finger locations in the 3D space. We present a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification (HMC) algorithm. We have achieved the gesture recognition accuracy (mean ± SD) is 86.1 ± 8.2\% and gesture typing speed is 3.09 ± 0.53 words per minute (WPM), when recognizing the gestures of the American Sign Language (ASL).",

keywords = "Data mining, Leap motion, Machine learning, Pattern recognition, Virtual reality, gesture recognition",

author = "Aurelijus Vaitkevi{\v c}ius and Mantas Taroza and Tomas Bla{\v z}auskas and Robertas Dama{\v s}evi{\v c}ius and Rytis Maskeliunas and Marcin Wo{\'z}niak",

note = "Publisher Copyright: {\textcopyright} 2019 by the authors.",

year = "2019",

month = jan,

day = "28",

doi = "10.3390/app9030445",

language = "English",

volume = "9",

journal = "Applied Sciences (Switzerland)",

issn = "2076-3417",

publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",

number = "3",

}

TY - JOUR

T1 - Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion

AU - Vaitkevičius, Aurelijus

AU - Taroza, Mantas

AU - Blažauskas, Tomas

AU - Damaševičius, Robertas

AU - Maskeliunas, Rytis

AU - Woźniak, Marcin

PY - 2019/1/28

Y1 - 2019/1/28

N2 - We perform gesture recognition in a Virtual Reality (VR) environment using data produced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) hand model by recognizing and tracking user's hands. From this model, the Leap Motion application programming interface (API) provides hand and finger locations in the 3D space. We present a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification (HMC) algorithm. We have achieved the gesture recognition accuracy (mean ± SD) is 86.1 ± 8.2% and gesture typing speed is 3.09 ± 0.53 words per minute (WPM), when recognizing the gestures of the American Sign Language (ASL).

AB - We perform gesture recognition in a Virtual Reality (VR) environment using data produced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) hand model by recognizing and tracking user's hands. From this model, the Leap Motion application programming interface (API) provides hand and finger locations in the 3D space. We present a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification (HMC) algorithm. We have achieved the gesture recognition accuracy (mean ± SD) is 86.1 ± 8.2% and gesture typing speed is 3.09 ± 0.53 words per minute (WPM), when recognizing the gestures of the American Sign Language (ASL).

KW - Data mining

KW - Leap motion

KW - Machine learning

KW - Pattern recognition

KW - Virtual reality

KW - gesture recognition

UR - https://www.scopus.com/pages/publications/85061033111

U2 - 10.3390/app9030445

DO - 10.3390/app9030445

M3 - Article

AN - SCOPUS:85061033111

SN - 2076-3417

VL - 9

JO - Applied Sciences (Switzerland)

JF - Applied Sciences (Switzerland)

IS - 3

M1 - 445

ER -

Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this