Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion

Aurelijus Vaitkevičius; Mantas Taroza; Tomas Blažauskas; Robertas Damaševičius; Rytis Maskeliunas; Marcin Woźniak

doi:10.3390/app9030445

Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion

Aurelijus Vaitkevičius
, Mantas Taroza
, Tomas Blažauskas
, Robertas Damaševičius
, Rytis Maskeliunas
, Marcin Woźniak

Katedra Modelowania Sztucznej Inteligencji (RMS/RMS2)

Kaunas University of Technology

Wyniki badań: Wkład do czasopisma › Artykuł › recenzja

110 Cytowania z bazy Scopus

Abstrakt

We perform gesture recognition in a Virtual Reality (VR) environment using data produced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) hand model by recognizing and tracking user's hands. From this model, the Leap Motion application programming interface (API) provides hand and finger locations in the 3D space. We present a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification (HMC) algorithm. We have achieved the gesture recognition accuracy (mean ± SD) is 86.1 ± 8.2% and gesture typing speed is 3.09 ± 0.53 words per minute (WPM), when recognizing the gestures of the American Sign Language (ASL).

Język oryginału	angielski
Numer artykułu	445
Czasopismo	Applied Sciences (Switzerland)
Tom	9
Numer wydania	3
Identyfikatory DOI	https://doi.org/10.3390/app9030445
Status publikacji	Opublikowano - 28 sty 2019

Obszary tematyczne ASJC Scopus

Materiałoznawstwo ogólne
Instrumentacja
Inżynieria ogólna
Chemia i technologia procesów
Zastosowania informatyki
Procesy przepływu i przenoszenia płynów

Uzyskaj dostęp do dokumentu

10.3390/app9030445

Inne pliki i linki

Link do publikacji w Scopus

Cytowanie

@article{14da001ca6d74405b4d6300a7c723669,

title = "Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion",

abstract = "We perform gesture recognition in a Virtual Reality (VR) environment using data produced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) hand model by recognizing and tracking user's hands. From this model, the Leap Motion application programming interface (API) provides hand and finger locations in the 3D space. We present a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification (HMC) algorithm. We have achieved the gesture recognition accuracy (mean ± SD) is 86.1 ± 8.2\% and gesture typing speed is 3.09 ± 0.53 words per minute (WPM), when recognizing the gestures of the American Sign Language (ASL).",

keywords = "Data mining, Leap motion, Machine learning, Pattern recognition, Virtual reality, gesture recognition",

author = "Aurelijus Vaitkevi{\v c}ius and Mantas Taroza and Tomas Bla{\v z}auskas and Robertas Dama{\v s}evi{\v c}ius and Rytis Maskeliunas and Marcin Wo{\'z}niak",

note = "Publisher Copyright: {\textcopyright} 2019 by the authors.",

year = "2019",

month = jan,

day = "28",

doi = "10.3390/app9030445",

language = "English",

volume = "9",

journal = "Applied Sciences (Switzerland)",

issn = "2076-3417",

publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",

number = "3",

}

TY - JOUR

T1 - Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion

AU - Vaitkevičius, Aurelijus

AU - Taroza, Mantas

AU - Blažauskas, Tomas

AU - Damaševičius, Robertas

AU - Maskeliunas, Rytis

AU - Woźniak, Marcin

PY - 2019/1/28

Y1 - 2019/1/28

N2 - We perform gesture recognition in a Virtual Reality (VR) environment using data produced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) hand model by recognizing and tracking user's hands. From this model, the Leap Motion application programming interface (API) provides hand and finger locations in the 3D space. We present a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification (HMC) algorithm. We have achieved the gesture recognition accuracy (mean ± SD) is 86.1 ± 8.2% and gesture typing speed is 3.09 ± 0.53 words per minute (WPM), when recognizing the gestures of the American Sign Language (ASL).

AB - We perform gesture recognition in a Virtual Reality (VR) environment using data produced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) hand model by recognizing and tracking user's hands. From this model, the Leap Motion application programming interface (API) provides hand and finger locations in the 3D space. We present a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification (HMC) algorithm. We have achieved the gesture recognition accuracy (mean ± SD) is 86.1 ± 8.2% and gesture typing speed is 3.09 ± 0.53 words per minute (WPM), when recognizing the gestures of the American Sign Language (ASL).

KW - Data mining

KW - Leap motion

KW - Machine learning

KW - Pattern recognition

KW - Virtual reality

KW - gesture recognition

UR - https://www.scopus.com/pages/publications/85061033111

U2 - 10.3390/app9030445

DO - 10.3390/app9030445

M3 - Article

AN - SCOPUS:85061033111

SN - 2076-3417

VL - 9

JO - Applied Sciences (Switzerland)

JF - Applied Sciences (Switzerland)

IS - 3

M1 - 445

ER -

Recognition of American Sign Language gestures in a Virtual Reality using Leap Motion

Abstrakt

Obszary tematyczne ASJC Scopus

Uzyskaj dostęp do dokumentu

Inne pliki i linki

Fingerprint

Cytowanie