When you read a sentence like this one, you know it was written by a thinking, feeling human being. And, in this case, a human is typing these words: [Hello, there!] However, some sentences that appear remarkably humanlike are now generated by artificial intelligence systems that have been trained on massive amounts of human text.
People are so used to assuming that fluent language comes from a thinking, feeling human that evidence to the contrary can be difficult to comprehend. How will people navigate this relatively uncharted territory? Because there is a persistent tendency to associate fluent expression with fluent thought, it is natural – but potentially misleading – to believe that if an AI model can express itself fluently, it thinks and feels like humans.
As a result, it's perhaps unsurprising that a former Google engineer recently claimed that Google's AI system LaMDA has a sense of self because it can generate text about its alleged feelings. This occurrence, as well as the subsequent media coverage, prompted a number of appropriately skeptical articles and blog posts about the claim that computational models of human language are sentient, or capable of thinking, feeling, and experiencing.
The question of what it means for an AI model to be sentient is complicated (see, for example, our colleague's perspective), and our goal here is not to resolve it. However, as language researchers, we can use our work in cognitive science and linguistics to explain why it is all too easy for humans to fall into the cognitive trap of believing that a sentient, conscious, or intelligent entity can use language fluently.
Using AI To Generate Human-like Language
Text generated by models such as Google's LaMDA can be difficult to distinguish from human-written text. This remarkable accomplishment is the result of a decades-long effort to develop models that generate grammatical, meaningful language.
Early versions, known as n-gram models, dating back to at least the 1950s, simply counted up occurrences of specific phrases and used them to guess what words were likely to occur in specific contexts. For example, "peanut butter and jelly" is a more likely phrase than "peanut butter and pineapples." If you read enough English text, you will come across the phrase "peanut butter and jelly," but you may never come across the phrase "peanut butter and pineapples."
Today’s models, sets of data and rules that approximate human language, differ from these early attempts in several important ways. First, they are trained on essentially the entire internet. Second, they can learn relationships between words that are far apart, not just words that are neighbors. Third, they are tuned by a huge number of internal “knobs” – so many that it is hard for even the engineers who design them to understand why they generate one sequence of words rather than another.
The models’ task, however, remains the same as in the 1950s: determine which word is likely to come next. Today, they are so good at this task that almost all sentences they generate seem fluid and grammatical.
Read Also: Pinterest CEO Ben Silbermann Resigns, Google's E-commerce Executive Takes Over
Peanut Butter And Pineapples
We asked a large language model, GPT-3, to complete the sentence “Peanut butter and pineapples___”. It said: “Peanut butter and pineapples are a great combination. The sweet and savory flavors of peanut butter and pineapple complement each other perfectly.” If a person said this, one might infer that they had tried peanut butter and pineapple together, formed an opinion and shared it with the reader.
But how did GPT-3 come up with this paragraph? By generating a word that fit the context we provided. And then another one. And then another one. The model never saw, touched or tasted pineapples – it just processed all the texts on the internet that mention them. And yet reading this paragraph can lead the human mind – even that of a Google engineer – to imagine GPT-3 as an intelligent being that can reason about peanut butter and pineapple dishes.
The human brain is hardwired to infer intentions behind words. Every time you engage in conversation, your mind automatically constructs a mental model of your conversation partner. You then use the words they say to fill in the model with that person’s goals, feelings and beliefs.
The process of jumping from words to the mental model is seamless, getting triggered every time you receive a fully fledged sentence. This cognitive process saves you a lot of time and effort in everyday life, greatly facilitating your social interactions.
However, in the case of AI systems, it misfires – building a mental model out of thin air.
A little more probing can reveal the severity of this misfire. Consider the following prompt: “Peanut butter and feathers taste great together because___”. GPT-3 continued: “Peanut butter and feathers taste great together because they both have a nutty flavor. Peanut butter is also smooth and creamy, which helps to offset the feather’s texture.”
The text in this case is as fluent as our example with pineapples, but this time the model is saying something decidedly less sensible. One begins to suspect that GPT-3 has never actually tried peanut butter and feathers.
Fluent Language Alone Does Not Imply Humanity
Will artificial intelligence ever become sentient? This is a difficult question that philosophers have been debating for decades. However, researchers have discovered that you cannot simply believe a language model when it tells you how it feels. Words can be deceptive, and it is all too common to confuse fluent speech with fluent thought.
