The project

Active video game systems controlled through arm gestures and motions (like, Microsoft Kinect and Nintendo Wii) have become popular over the past years. Such systems have introduced a new style of physical interaction and have raised attention as a means for balance training and fitness improving for healthy persons as well as for medical purposes including rehabilitation protocols. However, the games that have been designed are not adapted for the elderly as there is often difficulty both in using and in understanding them, so the presence of a third more experienced person is always required to guide the users.

GAME2AWE aims to develop a platform for game-based exercise applications suitable for the elderly by integrating innovative features that combined are not provided by existing systems.

Objectives

The objectives of GAME2AWE are served by the following research and development activities:

Design of games suitable for the elderly following a human centered design approach regarding game mechanics specification by involving all relevant stakeholders (elderly, caregivers, medical experts, system and game engineers) in all phases of system development.

Combining multiple enabling technologies for the development of the GAME2AWE platform in order to support multimodal interactions (voice, gestures, movement and touch) which can facilitate the implementation of game scenarios in a way that motivates and enhances users’ autonomy while making the platform easy to use.

Design and construction of a robotic tile as a component for the assembly of a smart floor which will serve as an autonomous computing platform for the deployment of games that train motor and cognitive functions

Integration of measures in the game mechanics that have been proven to predict the risk of falling, such as the Choice Step Reaction Time providing the potential for using the platform as a diagnostic tool.

Application of machine learning algorithms to analyze the data collected by the platform in order to: a) define more effective exercise programs by adapting the difficulty level to the profile of each user; b) train a model to classify users at cognitive and motor levels according to their performance.

Evaluation of the game platform in three dimensions (motor, cognitive and technological) by applying a randomized controlled trial and using valid relevant scales.

Beyond state of the art

The innovation of the proposed platform is found in the combination of three technologies: the smart floor, the recognition of gestures and movements and virtual reality. Furthermore, through the use of innovative machine learning algorithms and modern artificial intelligence technology (e.g., MS Azure AI) to track the characteristics of the environment and the user, it will be possible to dynamically adapt the system to these characteristics (e.g., adjusting exercise difficulty). The above features make the GAME2AWE platform a unique platform for gaming exercises.

Methodology

Human centered design according to ISO 13407 for interactive applications is followed for the development of the exergame applications. The design is based on an iterative process where face-to-face meetings take place at the initial stage between the application development team, the elderly and health specialists (physiotherapists, orthopedics, psychologists) who are invited to identify possible interaction activities and the basic mechanisms related to the interventional goals of each game. A focus group is created to help refine the characteristics of each game and the body movements that will be used to empower the elderly during the game. By capturing the initial plan of the application, a quick prototype of the game concept is implemented, which is given to users for evaluation.

The system will incorporate virtual reality technologies and motion recognition using motion detection sensors. Virtual reality technology will offer the user a realistic open world environment that will be designed using a graphics development machine (Unity Graphics Engine). Also the combination of virtual reality with the motion sensor eliminates the need for additional remote controls providing more freedom of movement to the user.

The implementation and integration of machine learning algorithms aimed at training the system for the recognition of complex movements allows the creation of a variety of exercises to improve the physical condition and cognitive level of the user. In addition, the system will support its simultaneous use by two or more users, thus allowing cooperation for a better team result or fair competition.

To perform the exercises beyond the body movements that will be perceived by the motion sensor, a smart floor will be constructed which will consist of autonomous robotic tiles.