Recent advancement of the technology on hand gesture tracking devices provided new opportunities for Human Computer Interaction (HCI). Although great progress has been made by leveraging devices like Leap Motion, most of the focus is on the entertainment domain. This research project focuses towards virtual rehabilitation using serious games, by improving the cognitive and motor skills of the children with special needs. Proposed hypothesis included the implementation of 3D mouse to navigate using hand gestures and common interface to facilitate the play of serious games built in different technologies. A scoring system was implemented to quantify the user effort. Implementation methodology included capturing hand gesture tracking data from the Leap Motion controller. These gestures are used by users to play the serious games. The serious games are categorized according to the skills that they would improve. Time period of play and the score that the user achieved was stored in the database to track any improvement of the skills. Some training games also added to the LeapPlay system to targeted improvement of the skills. Training games are simple, which had minimal actions to perform than the standard serious games included, but they are target to practice specific skill, either cognitive or motor. Evaluation process captures the accuracy of the hand gesture recognition, while a scoring system determines the improvements made on user skills by considering game play duration and achievements. Phases in the application also includes serious game selection and training. Proof of concept received good usability feedback as well as positive results of the user improvements gained. By considering final outcome of the results, the selected hypothesis was identified as the suitable approach. This study helps children with special needs to avoid hardships while providing more accurate and quantifiable improvements to their rehabilitation with joy.
D. Djaouti, J. Alvarez, and J.-P. Jessel, “Classifying serious games: the G/P/S model,” in Handbook of Research on Improving Learning and Motivation Through Educational Games: Multidisciplinary Approaches, P. Felicia, Ed., pp. 118–136, IGI Global, 2011.
D. Djaouti, J. Alvarez, J.-P. Jessel, and O. Rampnoux, “Origins of serious games,” in Serious Games and Edutainment Applications, M. Ma, A. Oikonomou, and L. C. Jain, Eds., pp. 25–43, Springer, Berlin, Germany, 2011.
J. Alvarez and L. Michaud, Serious Games: Advergaming, Edugaming, Training, and More, IDATE, 2008.