Extraordinary tech to support ordinary life for people with dementia

A focus on flexibility and inclusivity lies at the heart of AUTONOMOUS, a digital solution to help people with dementia complete daily tasks safely and independently. From the multidisciplinary range of the team behind the tech, to their devoted pursuit of an affordable solution that meets different people’s needs, the project takes inclusivity and accessibility very seriously.  

It’s an approach that has helped propel the project from proof of concept to prototype in just nine months, and to win accolades: in October, AUTONOMOUS was selected as a finalist on the Longitude Prize on Dementia, receiving £300,000 and capacity-building support for the next 15 months. 

Being selected “was an exciting recognition for the team,” says Cristina Santos, project lead and senior researcher at Fraunhofer Portugal AICOS – which launched AUTONOMOUS alongside Carnegie Mellon University and the LUCA School of Arts in Belgium. “It was really important to understand that the methods and effort put into the solution was recognised, as well as [its] potential.” 

Extraordinary tech to aid ordinary tasks 

AUTONOMOUS runs on a smartwatch and smartphone and uses sensors to monitor household activities and data. The AI-powered system learns to recognise people’s patterns of life and sends alerts to help them with daily tasks such as cooking or washing up. It can also detect and alert users and caregivers to anomalies, such as a tap left running. As a person’s condition progresses, AUTONOMOUS becomes more proactive, offering real-time support and suggestions, and helping prevent emergencies and costly medical interventions. 

Pooling diverse expertise to solve problems 

A staggering range of complementary expertise feeds into AUTONOMOUS. “For instance, the partners at Carnegie Mellon University are very devoted to the AI aspect and modelling of the human activity recognition algorithms,” says Ana Vasconcelos, an AICOS scientist. LUCA has  rich experience in working with people with dementia “in a very longitudinal way.” Fraunhofer AICOS brings expertise in human-centred design and melding informatics engineering, electronics, telecommunications, and design, as well as expertise in psychology, healthcare, disability, and chronic illness. AUTONOMOUS also appointed a lived experience advisory board – “bringing the reality of people with dementia and caregivers’ difficulties into the development of the solution,” says Cristina.  

An openminded approach to testing 

Over the past year, their openminded approach has guided the team’s user research and development. The team has been testing the solution in Portugal, Italy, and Belgium: altogether, 120 participants (people with dementia, caregivers, care managers, and others) have taken part in co-development, using a range of strategies: focus group discussions, formal interviews, advisory sessions, and co-design workshops. 

After this, the team ran two-week field trials in four homes in Belgium and Italy – a deliberately broad range of settings to ensure data fairness and diversity. This enabled them “to stress test the system and assess its performance”, including how it monitors and classifies activities and provides interventions in real time. Essential, too, was an assessment of its acceptability and use by people with dementia. 

Feedback was both positive and robust. “People could understand the great potential of the solution,” and provided feedback on the kinds of stimuli and alerts they wanted, as well as the look and feel of the smartwatch, says Cristina. “Having people be active and participating in research that concerns their own issues is very important,” adds Ana. “They feel that contributing is already a very good experience in itself.”

Balancing different needs

Detailed user feedback has led to important technical refinements. Take the casing that surrounds the sensors, for example. From a technical perspective, it was essential that the sensors could see everything in the kitchen to capture data, explains Ana. At the same time, feedback from researchers and people living with dementia flagged that the device needed to be subtle and non-stigmatising, blending into the home environment without calling attention to the fact it is an assistive technology. The team had to “reach a middle ground,” coming up with a casing that would be technically sound and aesthetically pleasing but still ethically designed and used. They are now developing an ornamental floral display and vase-type casing for the sensors so that the device blends into a room’s decor, and will further co-design and refine this in the Finalist stage.

They also received feedback on the different prompts people want or need. The system can deliver alerts in different forms – images, sound, written messages, vibrations, voices, or a combination of these – which can be customised depending on likes, dislikes, and need. Different types can be nuanced further: for example, an image of a coffee machine could be a photograph, an elaborate drawing, or a simpler icon, and people can choose different voices for alerts. “It’s very, very subjective – it has to be personalised,” says Ana. 

“Hierarchy of stimuli” 

Another key lesson from the Discovery phase was the importance of using prompts appropriate to the daily fluctuations in people’s dementia symptoms, Cristina explains. The team was aware that people would have different needs and different types and stages of dementia – “but we didn’t account for the fact that the cognitive fluctuations of people with dementia occur throughout the day.” For example, if a tap is left on, the system must identify this and then relay the information to the user in a way they can understand. “And this can vary in terms of the prompt or the message that is conveyed, because sometimes the person can read, but possibly two hours after that, the person could have trouble reading,” Cristina explains. “So we had to find a way to make sure the system is responsive and adaptive to the needs of the person throughout the day.” 

“This is probably the most beautiful and challenging aspect of the solution,” she says. Using rules-based algorithms, different sequence combinations, and pre-programming, “the system is able to very quickly monitor and classify the activity and also when the person does not perform the activity or does not perform it correctly, to respond and to communicate with the user in a way that the person can actually understand in that specific moment.” If none of the strategies work, it then alerts the caregiver to provide support, if needed. 

A respectful approach to private space  

There is a balance to be struck here: safety is paramount but so is the need to avoid stigmatising people. “We didn’t want the system to shout or to alert that the person requires support – so we designed it to be very subtle and silent when not needed and then progressively to become more conspicuous and more capable of alerting the user,” over different timeframes, including the span of a day. 

Avoiding frustration is another priority. “If I’m experiencing  difficulty in completing an activity and the system is constantly telling me that I’m failing, that can be quite frustrating,” says Cristina. Instead, AUTONOMOUS integrates crucial communication strategies gathered from caregivers, such as: “Ok let’s redirect attention, let’s pause for a period of time, and then redirect or prompt the person to complete the activity [later on].”

Finetuning and expanding testing 

The team now plans to refine AUTONOMOUS to make its prompts more responsive to individual needs and rank them according to importance. The team are also working on an on-boarding process that can be done autonomously by the person with dementia and/or with the caregiver. “You can set your preferences, such as ‘I would like to be warned with a picture first and then with a sound and then with text,’ and you can have different hierarchies,” explains Ana. 

Month-long field trials with an expanded range of households will broaden the data collection, alongside co-development strategies used in the Discovery phase. The team has partnered with care centres and research networks across Portugal, Italy, and Belgium, and is considering expanding testing into the United States, to substantially broaden the data and help build a system responsive to diverse needs. 

The team will also focus on commercialisation – an area where support from the Longitude Prize on Dementia will play a crucial role. They are working on ways to make the solution as affordable and accessible as possible. For example, they have been testing different types of sensor hardware – “trying to find the cheapest solutions that work and to strip everything that is unnecessary” – and testing the system on both high- and low-spec market-ready devices, to see where it will be possible to bring prices down. 

“In the end, we are researchers not market players, so our goal is always to reach the largest number of people possible,” says Ana.