by George Nott

Creating a ‘heartful robot’: UNSW and Fuji Xerox commence long term social robotics study

Feb 13, 20187 mins
Artificial IntelligenceCollaboration SoftwareDigital Transformation

abstract background light blue
Credit: Illus_man/Shutterstock

Despite the sci-fi inspired expectation that we will one day share our homes and workplaces with physical robots, there has been precious little real-life research into human-machine interaction over the long term.

Will our attitudes towards robots over time see them becoming an underclass of subservient slaves (which later revolt) as proposed in I, Robot? Perhaps cohabitation over a long enough period will see humans falling in love with their bots as is the case in Ex Machina?

There have been a small number of studies to date. In one human subjects shared their home with an expressive robotic head – the EMYS (EMotive headY System) – programmed with ‘emotions’ and able to do simple tasks like give weather updates and play music.

Considered a long term study, it lasted only ten days.

In other research participants have interacted with robots over the course of a year or more, but usually during hour long weekly sessions, or have been paired with basic robots like a vacuum cleaning Roomba.

There are very few social robots that have ever been implemented within a working environment or any environment really so we don’t have long term research data about what happens,” explains ProfessorMari Velonaki, director of UNSW’s Creative Robotics Lab.

Prof Mari Velonaki and humanoid robot Geminoid F

To find out, Velonaki, her colleagues and researchers from the Fuji Xerox Research Technology Group are busy building a prototype advanced bot for what will be one of the longest studies of day to day human interaction with a social robot.

The UNSW lab is now working on the design and ‘psychological programming’ of the robot, with technical capabilities such as its robo-navigation and artificial intelligence developed by the universities School of Computer Science

The bot – a prototype of which is expected to be ready within a year – will be placed with a group of 15 Fuji Xerox employees at the company’s research and development office in Yokohama, Japan, as part of a three year project.

Will the bot end up a valued member of the team, a tired toy or an annoyance?

“Our main goal is to make a companion for humanshellip; I think one of the differentiators will be understanding the person’s emotional requirements and acting not in a physical way, but in a subtle way that facilitates positive arousal,” says Dr Roshan Thapliya, research senior manager of the Research and Technology Group at Fuji Xerox.

“We want to create a heartful robot.”

Dr Roshan Tapliya. Photo: Quentin Jones

Creative mediator

When in situ at Yokohama, the robot will play a number of roles, Thapliya says.

“If you look at an average office worker at the moment, almost 70 to 80 per cent of their time they are not doing the work they should be doing. They are looking for documents or for the right person to ask about a particular problem. We would like a special type of robot that would fit right into the workplace so that people will not be disturbed by its presence but at the same time help them with their tasks,” he says.

The bot could act as a ‘creative mediator’ that interrupts at just the right moment and pairs people to collaborate, Velonaki, who has spent close to 20 years designing and building social robots and interactive art, adds.

“It sounds like an oxymoron, a machine to enhance human interaction, but sometimes you need a system that breaks the monotony. Someone else wants to join a circle because it seems collaborative and the robot can facilitate that,” she says.

Page Break

The robot will need to learn from body language cues if it is a good time to interrupt and if its interruption was welcome. With the help of the 15 employees – who will dedicate time in their day for training – the robot will over time refine its behaviours and movements.

“We have to provide something that can learn and adjust and start recognising people and learn the very basic things about a workspace: how people move, the speed of movement, when it’s a good time to slow down, what’s the acceptable body language, how close do you go, where do you stop,” says Velonaki.

There is an employee wellbeing function being built into the bot as well. It could encourage employees to take a break when tensions run high.

“It’s good to have some sort of a break. It’s good to leave a situation when it becomes too hectic and have some sort of pressure valve,” says Velonaki.

“We want a system that is not a threat – it doesn’t monitor your productivity but maybe your well being as a group,” she adds.

Velonaki’s robotic art installation – Diamandini – which exhibited at the VA in London.


A key challenge, given the length of the study, will be the robots need to maintain the interest and acceptance of its flesh and blood colleagues.

“For people to engage with it, it will need to learn and evolve and change. Otherwise it will get boring. And why would people at work spend time and work to teach a robot if they see that there’s no progress or they don’t see new patterns and it’s all a bit of a novelty,” says Velonaki.

Built in will be an element of surprise.

“You want an element of surprise. You don’t want this kind of totally subservient, happy robot. The question is when do you introduce the behavioural element of surprise? You don’t want it to be distracting you don’t want it to be something that looks like a fault or isn’t appropriate,” she adds.

Whether the study succeeds or fails (“even if the robot doesn’t coexist successfully we’re still learning” says Velonaki) its findings will be invaluable in designing future bots.

“If they’re going to be embedded in different social structures and institutions they have to be able to train and coexist and coexist in an interesting way,” Velonaki adds.

The robot’s design is, Velonaki puts it, “very different to anything that I’ve ever designed before” (and there have been some bizarre bots in the past). While its form and features are still under wraps, it is sure to be innovative, which is kind of the point, adds Thapliya.

“If you are really going to create something, you should create something that is really cutting edge. Get to the boundaries and see what it’s like,” he says.