How a home robot could help people with memory problems find lost items

A new home robot that tracks objects could help you find lost items.

Engineers hope that the breakthrough will help people with memory issues, including dementia patients, find medicine, glasses, phones and other items they need but have misplaced.

They reprogrammed an existing robot to detect, track and keep a memory log of specific objects that it sees in its camera view through stored video.

With the robot capable of distinguishing one object from another, it can record the time and date objects enter or leave its view.

Researchers then developed a graphical interface to enable users to choose objects they want to be tracked and after typing the objects’ names, search for them on a smartphone app or computer.

Once that happens, the robot can indicate when and where it last observed the specific object.

‘Personalised companion robot’

Dr Ali Ayub, a fellow in electrical and computer engineering at Canada’s University of Waterloo, said: “The long-term impact of this is really exciting.

“A user can be involved not just with a companion robot but a personalised companion robot that can give them more independence.”

Dr Ayub and three colleagues began with a Fetch mobile manipulator robot, which has a camera for perceiving the world around it before adding an object-detection algorithm.

Tests have shown the system is highly accurate and while some individuals with dementia might find the technology daunting, Dr Ayub said caregivers could readily use it.

Moving forward, researchers will conduct user studies with people without disabilities, then people with dementia.

The paper on the project was presented at the recent 2023 ACM/IEEE International Conference on Human-Robot Interaction in Sweden.