“We certainly see relationships being created with our agents,” said Mortensen. “We can talk about what kind of relationships, but they're certainly existing at some level. 

“I bloody work there, but I still start out most of my requests with 'Amy, would you be so kind'.”

Humans developing intimate relationships with machines is nothing new. Anyone around in the mid-90s who saw how fixated children were with their tamagotchis can attest to that, but there was never any suggestion that a tamagotchi was anything other than a virtual pet. The stories Mortensen tells about people’s reactions to Amy and Andrew, however, point towards the development of different, more familiar interactions. 

“In 11% of all the meetings we set up, hundreds of thousands of meetings, at least one of the emails in that dialogue will have no other intent than gratitude, as in someone who knew this was a machine went back to it to say 'thank you very much, really appreciate you setting this up on Friday this week’,” said Mortensen.

These people are showing gratitude to a piece of software, which is a new phenomenon. But will we expect this to be the norm going forward? Will we expect our intelligent assistants to be able to pass the Turing Test and convince us of their consciousness?

“None of us really have much of a relationship with, say, Photoshop. If I need to work on an image, I’ll open up the application, use it and be happy,” said Mortensen.

“But as soon as you move to the conversation where within the UI, you can have these agents, whatever their job might be, converse with you in natural language, the ability to imagine and really inject emotion into these agents just becomes very easy. And the real question is: is it now a game of whether I can fool you into believing that this is a human?

“I'm not so sure that I believe that that's a game worth playing. But that doesn't mean that I don't think there's value in applying levels of human emotion into that dialogue. Not to fool you, but because that holds value in the dialogue.”

When Mortensen talks about injecting valuable emotions into the dialogue, what he means is using an emotion like sensitivity, not to trick a recipient but to benefit the conversation. He gives the example: if someone couldn’t make an appointment three times, then any reasonable person may think they’re being snubbed. A machine wouldn’t realise that, but a human may handle the rescheduling with increasing delicacy.

“Amy obviously has no real idea, and she can loop this seventeen times over and not care,” said Mortensen. “You care though, as though Dennis was kind of a nice guy, now I think he's a bit of an asshole, and you need to kind of allow yourself to figure out: do I even want that meeting? 

“What we have seen is if Amy recognises the fact that this is slightly unfortunate and starts out with a simple way of saying that, to a full-on apology on the third retry, that level of emotion from where she also knows this is unfortunate increases the probability of the meeting being kept intact. 

“Think about that: people know that this is a machine, they know the emotion we portray is not real, but just by the fact that it's visualised, has them saying 'perhaps Dennis isn't an asshole, he's just busy'.”

Despite not wanting to inject emotions into intelligent machines to trick users, the presence of these emotions inadvertently stokes emotional reactions from people.

X.ai has also given its intelligent agents the ability to relay all the conversations it has with third parties back to its boss, which reveals just how strong attachments to conversational machines have become, with people sometimes seemingly preferring them to other people. 

“If you were the boss of Amy you could see what she talks about to all of your guests, so you could just jump in and ask her 'hey what did you and John talk about?',” said Mortensen. 

“She'll send that back and you can see the whole dialogue. People use that as a backdrop to 'I think John is a little bit rude'. As in, I don't think I like John that much actually, in the way that he treated Amy, even though I don't care whether you triple click on any icon in Photoshop. 

“You can click all you want. I don't really care, but somehow there's a different relationship here where 'I don't like the way he talked to Amy'.”

For x.ai, even though the line is blurring, there’s no need to pretend that these relationships are anything more that of a human to a machine. “I'd much rather we tell people upfront 'hey this is a machine', but we do the job so well that in the end you say 'thanks, Amy',” said Mortensen.

But given that real relationships between humans and machines are beginning to blossom with the earliest iterations of conversational AIs, it’s no wonder that we can see these machines becoming increasingly involved in our lives.