Games in Science and Technology

Major Research Project Developing Computer-Assisted Memory

An Old-Fashioned ReminderSix organizations, from six different countries are working together to develop a solution to the deterioration of the human memory. The three-year project named HERMES Cognitive Care and Guidance for Active Aging is looking at a combination of home-based and mobile solutions to help older people combat the natural reduction in cognitive abilities.

It is common knowledge that as humans age, their cognitive abilities begin to weaken and forgetfulness becomes a constant source of struggle. To tackle this issue, which is bound to cause growing social concerns among an aging population, the HERMES research initiative brings together expertise from varying fields, such as gerontology, speech processing, hardware integration and user-centered design. Read More...

The group of researchers is even developing a “home of the future” prototype, which would hold all the tools to assist an aging person with memory problems. For example, the house will be equipped with microphones and video cameras to record conversations and experiences at the user's will. All the information will be stored, processed and analyzed to help complement the person's memory. A mobile device that records conversations, experiences, location coordinates, dates and times outside the home, will provide functionality outside the home.

The prototype being designed will offer three main services. The first will help users remember what happened in the recent past. For example, you can ask the system “What did my daughter say to me yesterday about babysitting her kids?” The system will search its database of recordings according to the time window and the keywords used, and let you choose whether to play back the relevant conversations recorded in audio or on video.

The second service will give the person reminders or prompts to help manage their daily schedule. If you notice that the jar of coffee is almost finished, you can tell the system to add a reminder to buy coffee. The system will connect the terms and when you pass the store and your mobile device, noticing the store location, will remind you to buy coffee using a text to speech synthesis in a natural human voice.

The third aspect of the project is creating memory fitness exercises based on actual personal experiences. These games help jog the user's memory and are based on actual conversations or experiences that were recorded or entered as appointments. The system could ask you to organize them by category or quiz you by asking the exact time for next week's doctor's appointment.

“HERMES is helping realize the vision of lengthening the stage of independent living and helping aging populations remain active,” explains Professor Manfred Tscheligi, Director of the Center for Usability Research & Engineering (CURE) and coordinator of the HERMES project, in a news release.

CURE in Austria and INGEMA Foundation in Spain are working on the first stage of the project, which is to better understand the memory needs of the elderly. A portion of the work requires finding simple to use interfaces that elderly individuals with limited or no computer skills could control.

“Developing innovative yet non-intimidating technology to address the needs of this valuable population is a challenge that this community of researchers is eager to embrace,” added Tscheligi.

The technology aspect brings together the IBM Research Lab in Haifa (Israel), Athens Information Technology (Greece), Bradford University (UK) and TXT e-Solutions (Italy), who will work to develop the sensory and perceptual technologies.

Experts at the IBM Research Lab in Haifa, Israel are contributing technologies and research for speech to text transcription, speaker recognition, voice-based emotion detection, and text-to-speech synthesis.

“With HERMES, our research into multimedia technologies is taking a real leap forward in transforming how people will go about their daily lives,” says Ron Hoory, manager of speech technologies at the Haifa laboratory.

The concrete applications of this research project could help dramatically reduce the need for active care and support for the elderly and substantially increase their ability to cope.