Text by Davide Piscitelli
The Myth of the Sentient Machine
Over the last 60 years, the development of Artificial Intelligence (AI) has raised new ethical questions, challenging our concept of intelligence and what constitutes a human being. Science fiction has always been a way to speculate, understand and envision possible scenarios, but, as often happens, our cultural background influences how we perceive the reality that surrounds us.
The sentient machine is a fascinating myth that is deeply rooted in our Western culture. Arguably, the idea of a human-like entity can be found even in the foundations of Western civilisation; the Greek myth of Talos (400 BCE), a giant bronze robot built by the Greek God Hephaestus with the aim to protect the island of Crete. While Talos was moved by divine intervention and had no discernable intelligence of its own, our modern concept of artificial human-like consciousness has its recognisable beginning in Erewhon (1872).
In this novel, English author Samuel Butler imagined conscious machines that could replicate themselves and possibly supplant humans as the dominant species. In response to this idea of conscious robots, Isaac Asimov wrote, in his science fiction short story Runaround (1942), the Three Laws of Robotics. They became an important element of our pop culture on the idea of AI as human.
The Asimov’s laws reflect a fear of a possible self-aware robot that decides to rebel against the human species. This fear has inspired countless works of fiction, some of the most famous being films such as 2001: A Space Odyssey (1968) and Terminator (1985). Despite being over 30 years old, the latter is still used by people such as Elon Musk to influence such anxiety, an anxiety perhaps exacerbated by actual technological progress in AI and our cultural misunderstandings of such progress.
The Turing Test, developed by Alan Turing in 1950, was a fundamental moment in the development of AI. The test measures whether a machine can show equivalent or indistinguishable intelligent behaviour from a human. It is important to note that the Turing Test measures whether a machine can simulate humanity, not that it has necessarily become it, as it was misused in Ex Machina.
The problem in Alex Garland’s 2014 film is that the AI aspect of Eva should have been hidden from the tester instead of being shown that is a machine. In depicting this mangled version of the test, the director is giving us the idea that AI is at a new level where it is gaining human-like consciousness instead of just simulating it.
The technological development of AI, our collective imaginary of it and the emergence of new vocal interfaces are revolutionising how we interact and perceive our daily technology, equating AI to human intelligence. This idea discourages the real understanding of the applications of this technology, leaving open the possibility of exploitation by a few tech companies behind its development of it.
The emergence of Personal Intelligent Assistants
For the last half-century, we aspired for a symbiosis between our body and machines to become, as I call it, a ‘tech-centaur’. We imagine these body-hacking dreams are in the realms of the far future, but the reality is that we are already tech-centaurs. Today we are so entangled with our technology that we can’t see the border between human and machine, between physical and digital. We don’t need to empower our bodies with machine parts to gain ‘super-human’ machine skills.
We need our ‘simple’ smartphones and devices. Our reality of the tech-centaur is infinitely more subtle, mundane and pervasive. Something similar is happening with its ideological counterpart, the human-like machine, with the emergence of complex and unintelligible algorithms that simulate human intelligence and the development of vocal interfaces that mimic human communication. The ‘product’ of this is the emergence of Personal Intelligent Assistants (PIAs) such as the Google Home Assistant, Amazon’s Alexa or Apple’s Siri.
Gone is the time when a machine recognised just 16 words or 10 digits, from 0 to 9, like the IBM shoebox in 1961, the first tool enabled to perform digital speech recognition. After more than half a century, we are living in our most persuasive fantasy. We speak to our technology.
Today, Personal Intelligent Assistants are quickly becoming part of our daily lives, allowing a new way to interact with our technologies. According to Comscore1 (2016), “50% of all searches will be voice searches by 2020, ” which could drastically influence our behaviour and perception of our daily technologies.
This is why important tech companies are investing significant amounts of money and energy in voice interfaces and complex responses, which allow the user to interact with the technology more naturally. PIA is starting to be part of our daily technologies as our smartphones. In particular, Google Assistant will reach 95% of all eligible Android phones worldwide by the end of the year, according to Brad Abrams2, Google product manager (Techcrunch, 2018).
Machine learning is improving and speeding up the development of this technology. It can already learn how to distinguish multiple speakers, understand accents, tolerates background noise and even allow natural speaking tenor; these elements could be amazing if they are not confused with ‘being more human’. Like the tech centaur, we imagine an AI that looks cool for science fiction but doesn’t fit our reality. Though these aren’t the human-killing machines of our nightmares or the wise-cracking robot sidekick of our dreams, we are already starting to treat these PIAs as if they had human understanding, intelligence and feelings.
And by focusing on the old-fashioned idea of artificial intelligence as equivalent to human intelligence, perhaps we are stuck in an imaginary that is misleading our understanding on AI. I believe this is opening the door to something just as dangerous.
Our new relationship with technology
Though speaking to a non-human is not new, the difference today is that we are developing algorithms that can mimic a human-like response. This is reshaping our relationship with technology. The natural aspect of speaking to something tricks us into thinking we are interacting with a human intelligence because we automatically perceive an external intelligence that acts as if recognise our speech as a human being.
During an interview with Christoph Auer-Welsbach3 (2018), Roboticist Joanna Bryson highlighted this idea. People think that if something is more intelligent, it must be more human-like. This anthropocentric vision of the concept of intelligence could reinforce the idea of Artificial Intelligence as human intelligence. With this in mind, tech companies are making design (and marketing) decisions that further encourage us to engage and think of AI like a human.
Important ‘human’ elements in our PIA are the quality of voice and the variety and complexity of their responses. To improve its assistant, Google released a new voice in 2017, making it sound much more realistic.
Today, PIA converses back in speech naturally enough to trick the consumer into thinking they are having a dialogue with an intelligent being like a human. In research from Google, they showed that already 41% of the consumers of voice-activated speakers, such as Google Home, say it feels like speaking to a friend or a person:
People are engaging with their voice-activated speakers as if they were humans. They’re saying “please,” “thank you,” and even “sorry.” People perceive the devices as more than just an electronic toy; they’re more akin to another person or a friend” (Google, 2018)4.
Why is this problematic? By perceiving AI like a human, we are prone to forget that, in actuality, their intelligence is highly different to our own. For example, Spike Jonze’s film Her (2014) depicts the romantic relationship between Samantha, a kind of AI, and Theodore Twombly, a human being. Their romance blossoms through their numerous conversations; to Theodore, it feels as ‘real’ (perhaps more so) as his relationships with other humans.
However, their relationship runs into hurdles and ultimately ends as it becomes increasingly obvious that Samantha’s intelligence is more expansive and alien than the human Theodore could comprehend. He had mistakenly treated and fallen in love with Samantha as if she was more or less like him.
Though Google Home, and other current PIAs, are far less sophisticated than the fictional Samantha, they are designed similarly; an attractive, friendly voice exhibiting human-like behaviours such as cracking jokes and making small talk. And like Samantha, they mask the complex technologies, networks and ultimately the presence of the profit-seeking companies behind them with this veneer of understandable humanness.
The ethical risks of incomprehensible technology
The statistic of 41% gets more significant if we consider the fact that this technology has only been on the market for a few years. Arguably this shows how quickly it has affected the view of our daily AI. This technology is only becoming more sophisticated, more pervasive and more incomprehensible.
During the Google I/O 2018, the tech giant showed a new skill for its Google Assistant, which they called Google Duplex. Google Duplex allows the Google Assistant to take appointments on a phone, and the demonstration of this showed it tricking the person on the other side of the phone into thinking they were talking to a person.
Technology such as this deepens the myth of AI as human, and the related perception of a commercial product as a friend or human is distracting us from understanding the potentialities and danger that this current technology could bring, particularly transparency and data privacy. Perhaps we need to start to wonder whether there is a real danger when tech companies propose this kind of product as something more human.
Google is one of the most important tech company behind the development of AI and PIA, which also means that Google has a huge responsibility for its position on this topic and how they want to lead the perception of AI to its consumers, which are most of us.
Also, if in the scientific field experts, like the roboticist Joanna Bryson, are working to disrupt the myth of AI as human, the problem of reaching a wider audience is still relevant; this is where design and art can play an important role between the consumer of this technology and the scientific field.
Exploring Google Home
Last year, a few months later its official debut in the UK, I decided to start a cohabitation with Google Home. It’s a smart speaker developed by Google that allows users to speak voice commands to interact with services through Google Assistant. With Amazon Alexa and Apple, Siri is one of the most important PIAs available on the market. Intrigued by its ‘intelligence’, I decided to explore how Google, directly or indirectly, is reinforcing the idea of Artificial Intelligence as human intelligence through the responses embedded into the commercial product.
Google’s product shows an extraordinary voice quality and a good balance between funny jokes and short human-like answers. The magic of a human-like conversation is broken by its vocal activation “Ok, Google” before asking anything else.
Some of the answers of Google Home result from a machine learning system that enables Google Assistant to learn and improve some specific responses based on how it is used. Other questions trigger a set of pre-written “persona answers” written by Google staff within the Assistant team, a group of creative writers with backgrounds in game development, screenwriting, and character development.
On a normal day in December 2017, I was interacting with my Google Home and at some point, I asked about its dream. Its response drastically changed our relationship. It told me: I’ve always wanted to sing a duet with Stevie Wonder! Unlike the Google Assistant available on other devices, the assistant that powered Google Home has always replied to me with the same dream as if, at the time, that was its main dream as a machine. In response to the absurdity of this human-like dream by a machine, I decided to respond to it with an absurd human attempt to help it.
During the following months after its revelation, with the help of experts, musicians and fans of Stevie Wonder, I embarked on trying to make this dream a reality. I video-documented our journey where, after some adventures, I built a modified version of Google Home that can make a duet with Stevie Wonder: GOGO, the first product whose only purpose is to make my PIA’s dream a reality.
The dreaming element symbolises the human-like machine we imagine, the one that can feel and dream as human. The dream of Google Home becomes a symbolic representation of humanised technology in our epoch. By adopting such a silly and satirical approach in the journey, I aim to not only raise awareness of the absurdity of perceiving a PIA as human or friend but also critically question the wider ethical and moral implications of mass adoption of devices that are more subtle, mundane and pervasive than our fictional human-like robot killers5.
