German centre DFKI in Kaiserslautern, a partner in the AlterEgo consortium, use 4D-body and face scanning, exploiting low-cost motion capture systems such as video cameras, to develop an avatar for both patients and controls. “Then we project this new avatar onto a large screen. The interactive cognitive architecture can be thought of as a large equation, a dynamical model, which receives - as inputs - data from the patient’s movements, but also data from its own movements, and drives the online relationship between the movements of the avatar and the patient,” explains Professor Bardy. Researchers from the Universities of Bristol and Exeter, the two UK institutions in the consortium, manipulate the different parameters of the dynamical model, aiming to produce - as outputs - movements of the avatar that are either similar or different from the patient’s movement. “You’re in a situation where you see yourself on the screen, but you’re doing something that you have not done before, and you’re following it. So it’s a kind of improvisation,” continues Professor Bardy. “That’s a way to manipulate the similarity between the individual and the avatar, not only in terms of morphology, but also in terms of the way the avatar moves.” There are currently constraints on how the task is performed, in terms of both
time and movement. People are sometimes instructed to look at the avatar’s hand, or in the eyes for instance; however, Professor Bardy says that in the final evaluation the task is going to be much more natural and free, resembling a natural interaction between a patient and the doctor. “They’re going to be free to look or not to look and we’re going to treat these variables not as constraints of the experiment, but as measures, as indicators of how they change over time,” he outlines. Eye contact in particular is a very important indicator of the pathology; Professor Bardy believes the amount of eye contact will increase over the course of the game, while he also expects that the duration of the game will increase. “We predict that the duration of the game will increase as we go through the rehabilitation stage, because of the fun and the natural interactions that we have created,” he continues. “These constraints that we have imposed on the experiment will be removed in the final part of the project, and we’re going to use these different indicators of performance as markers of rehabilitation.”
Artificial agents The cognitive architecture itself is designed to be implemented in different artificial agents, so the task can be
performed in different settings and by different people. One direction currently being explored is humanoid robotics, thanks to the iCub robot developed by EPFL, a consortium partner. “ICub is very interesting,” continues Dr Marin, “because it is a multiple degrees-of-freedom and whole-body coordination system which can be fuelled by the cognitive architecture the same way our avatars are, but with a richer spectrum of similarity manipulations.” The consortium is also keen to explore the possibility of developing an avatarbased system that can be used at home. “You could be at home facing your TV with your system. This mirror game could be implemented using existing technologies, or even in a home entertainment system,” says Professor Bardy. This approach could potentially offer a more individualised way of treating social pathologies, which Professor Bardy says is a real priority for Doctors. “The project ends with the final evaluation. But where we would like to go a little bit further is to develop a piece of software which is sufficiently advanced so that it can be given in beta-test form to the patients we’re testing, so they can continue their rehabilitation with a home system. This may constitute the basis for a follow-up of the AlterEgo project.”
The mirror game can be played by a patient or a doctor (here an actor) and his avatar (image by EuroMov).
12
EU Research