> GPT-3 doesn't have any knowledge of how the world actually works.
I think this is a philosophical question. There is a view that, basically, there is no such thing as knowledge, just language (or, at least, there is no distinction between knowledge and language). In this view, all there really is is language, which is mostly composed of metaphors and, ultimately, metaphors only refer to other metaphors, i.e. language is circular. In this view, not only is the ultimate, physical, concrete world beyond us but also we can't even talk about it. From this perspective, GPT-3 is not substantively different than what our minds are doing.
That view makes some strong claims (I don't find it convincing), but it's out there. A slightly different claim, though, is that "knowledge of how (we think) the world actually works" is encoded in language. To me, that seems trivially true. So, again, how you take this quote from LeCun depends on what you think knowledge is and your view of the relationship between knowledge and language.
Tell that to all the other animals on Earth. Do they not also have knowledge? Do you really think they encode their knowledge in language?
Do you really think that humans are so special as to encode all their knowledge in language? Watch a movie. Listen to a song. Examine a piece of art. Feel sculpture. Play a guitar. Dance.
There is a segment of the software community that is highly language centric/adept. But that community is often blind to other forms of understanding.
Just look at the language of Shakespeare. Much of the language is visual and experiential. How much would you actually understand without your senses and imagination? Your knowledge encompasses your being.
>Tell that to all the other animals on Earth. Do they not also have knowledge? Do you really think they encode their knowledge in language?
Well, actually, yes, they do. Many animals have elaborate languages encompassing many concepts. Crows can explain to one another what a human looks like, for example.
If GPT-3 has a consistent position on anything, it's only because the corpus it was trained on was consistent about it. So, for example, it will reliably autocomplete Jabberwocky because there are a lot of copies of this poem in the corpus and they are all the same.
If there were two versions of this poem that started the same way, it would pick between the variations in the corpus randomly. In other cases it might choose based on the style of prose or other stuff like that.
GPT-3 can get some trivia right, but it's only because the editors of Wikipedia already came to consensus about it and Wikipedia was weighted more. It doesn't have a way of coming to a consistent conclusion on its own.
Without consistency, how can it be said to know or believe anything? You might as well ask what a library believes. Sure, the authors may have believed things, but it depends which book you happen to pick up.
I agree with you in that I would make a strong distinction between what a model like GPT-3 does and whatever it is that humans do.
But I do think you're missing the point just a bit. When we speak and think, we use all kinds of metaphors that express judgements about the world, usually without realizing it. In other words, the way we use language encodes concepts in a deep way.
To borrow an example from George Lakoff, we, in English, use war-metaphors to talk about arguments. Of arguments and of wars you can say things like "he's marshalling his forces," "they're ceding their territory," or "she's girding her defenses". In fact, almost anything you can say about a war you can also say about an argument. In American politics, with regard to partisan squabbling and the filibuster, we talk about "the nuclear option". The fact that these metaphors make sense to us indicates a judgement, something like "arguments are like wars". That judgement shows up in billions of lines of English scraped from the internet and can be fed into a model, allowing GPT-3 to "make that connection" via purely statistical methods.
Yes, this is a bit like asking "what a library believes". But a lot of these metaphors show up in our languages and, in a way, they express judgements, which is something akin to a belief. Does that mean a library has beliefs? Is this all knowledge is? I wouldn't go that far. But the argument is an interesting one and worth raising.
Well, it's certainly interesting that it can learn metaphors, and this can be useful for creative purposes, so it's fun to play with.
But a sophisticated understanding of metaphors could be used to tell the truth or to lie. In the case of GPT-3, it doesn't know the difference. Telling the truth and lying come out of the same autocompletion process.
If you consider the use of a metaphor to be showing judgement, it means that a particular metaphor seems to be appropriate to use in a particular context.
At the risk of reigniting the perpetual war about how to characterize machine intelligence, and by extension how to characterize the risk they pose, Yann has been (and still is AFAIK) more in the "existential AI risk is a long-term problem" group. In a 2016 interview LeCun said [1]:
> We’re very far from having machines that can learn the most basic things about the world in the way humans and animals can do. Like, yes, in particular areas machines have superhuman performance, but in terms of general intelligence we’re not even close to a rat. This makes a lot of questions people are asking themselves premature. . That’s not to say we shouldn’t think about them, but there’s no danger in the immediate or even medium term. There are real dangers in the department of AI, real risks, but they’re not Terminator scenarios.
That's pretty measured overall, but he doesn't know that there's no existential AI risk in the medium term. No one does, and that's the problem. Experts simply suspect that it's unlikely. Stuart Russell and him have debated similar topics [2].
To tie back to your point: I keep seeing LeCun brush over tricky questions like yours and the ones at [2] with an arrogant confidence. I wish that he would be more careful, and I hope that I have a skewed view of him.
He's not wrong, we're very far. And looking at past "progress" it seems that we'll get there very slowly. So it seems long-term.
Except people are bad at exponential processes. Yet when economics drives us we are suddenly good at making them happen. And this combo seems to be what makes these existential risks. (Like climate change, or other manifestations of the coordination problem.)
I think an important distinction to make is your use of the word "language", and how we think of language as it concerns human minds, and as it concerns GPT-3.
In our heads, language is a combination of words and concepts, and knowledge can be encoded by making connections between concepts, not simply words. If there is no concept or idea backing up the words, it can hardly be called knowledge. Consider the case of the man who did not speak French, yet memorised a French dictionary, and subsequently went on to win a Scrabble competition. Just because he knows the words, would you say he knows the language?
A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
> In our heads, language is a combination of words and concepts, and knowledge can be encoded by making connections between concepts, not simply words. If there is no concept or idea backing up the words, it can hardly be called knowledge.
Great point.
> A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
Are you sure? Aren't "concepts" encoded in how language is used, at least to some degree?
LeCun does say that models that explicitly attempt represent knowledge perform better than GPT-3 in terms of answering questions. I'm no expert but I believe him.
>Aren’t “concepts” encoded in how language is used, at least to some degree?
Good point and I think this shows up to the extent different languages might affect how we express particular concepts.
However I think it is more accurate to say that language solidifies and gives form to how we express concepts and the “concepts” themselves are independent of languages. Only our “expression” of these “concepts” depends on language.
For anyone interested in art and art history, this distinction was the central focus of the French surrealist painter Rene Magritte.
Language is how we store our knowledge, and language is a system of words. If a language model contains all the possible sentences you can say, it will complete any of your sentences, don't you think it knows what you know? The input is sequence of characters, so you can say it may or may not operate on words. It can operate on subwords, words or phrases where it see fit.
I like to think intelligence as clouds. If you dig deep down, they are just droplets, there are so many of them, they can appear to be so many different shapes. And they look complete different. Maybe intelligence is the same.
Personally I do not find the whole "language = knowledge" argument convincing. But if you're interested in reading writers who make that argument (and perhaps I'm vulgarizing the argument a bit), Nietzsche makes it in On Truth and Falsity in their Extra-Moral Sense and George Lakoff makes it in Metaphors We Live By.
I'd also suggest Wittgenstein's
Tractatus Logico-Philosophicus, a seminal work of the logical positivist movement. Influenced by Frege's predicate calculus, the aim of the Tractatus was to determine an isomorphic relationship between language, thought, and external states of affairs. An axiomatic attempt to reveal a potentially ideal logical language, that is not interested in meaning per se, but merely an accurate reflection of the world. A closed system that essentially excludes non-falsifiable metaphysical question. Famously concluding with the instruction: "Whereof one cannot speak, thereof one must be silent." Part of Wittgenstein's project, even in its early aggressively logical form, was philosophy as a therapeutic. That is, the metaphysical questions concerning god, being, essence, and forms that had inspired thousands of years worth of fevered conversation, could be finally be quieted. That's not to say they couldn't be meditated on, but were not in the domain of his logical language, and so silence. Again, I think early Wittgenstein sometimes gets misinterpreted, "...therefore one cannot speak" does not, to me, mean that it can't be considered or one must forgo spirituality, just that it couldn't be spoken of within the project of the Tractatus.
Logical empiricism was ultimately a dead end as the criteria for even verifying empirical truth has long been contentious philosophically, and was further critiqued by contemporaries such as Quine who attacked the premise of the analytic/synthetic distinction (think Hume's fork, which Kant tried to solve) and Popper who cited the problem of induction to critique the fundamental premises of the positivists verificationism.
Wittgenstein is an interesting case, as the Tractatus is considered an early work of his, profoundly influential to analytic philosphy at the time, yet his later work, Philosophical Investigations is sometimes seen to retract the dogmatism found in the Tractatus. I tend to take the view that it's a continuation of his thought, rather than a retraction of his earlier work. Crudely, whereas his former thought represented a narrowly axiomatic definition of language and its truth value, PI investigates, among many other ideas, language as an activity, or game, that has meaning dependent on the context of its use, languages as families. Granted, Wittgenstein is a complex thinker and these are simply my interpretations.
It's also curious to note that as positivism was beginning to fall out of favor around the time of the second world war, a continental thinker such as Heidegger, whose thought luxuriated in the kind of metaphysical questions the positivists necessarily eschewed, rose to prominence and was infamously sanctioned by the NSDAP to philosophize about their presumed "destiny". Bit of a tangent, but I think the historical context is relevant, as often philosophical movements are birthed from pre- and post-war attitudes.
Are animals as intelligent as human? What language does to us is to be able to pass on knowledge to generations, knowledge can be accumulated. Animals may be able to pass on simple concept to next generation, they won't be able to accumulate knowledge without language and writing.
As a response to leftyted, erispoe's point neither depends on nor implies that animals are as intelligent as humans, it is simply bringing up a counter-example: what appears to be knowledge in animals lacking language.
Of course, you could always attempt to define knowledge such that it is purely verbal, or alternatively define whatever is going on in the brain of an animal to be language, but is either approach useful? In common usage, we recognise, as knowledge, various things that cannot be communicated by language, such as knowing how to ride a unicycle on a tightrope (I doubt you can learn it just from a book) and the infamous qualia which supposedly prove that the mind is dualistic. And what about the knowledge of how to use language? How does that get bootstrapped?
The knowledge of riding a unicycle on a tightrope does not give you the ability to ride a unicycle on a tightrope. Yes, you have to learn it through experiencing it, because you need to map it to your motors. Animal has instincts, they are able to trace water, for example, without teaching. It is also knowledge, but for our discussion, knowledge is what we obtained after birth, not something encoded in our DNA. This knowledge is stored in the format of language. Apparently some believes that that language is part of our DNA. That's why you can have Tazan, but not an animal can speak human language even if a human raise it since it was born because language cannot be learnt without support in code in DNA.
If we define knowledge that way, leftyted's original claim (knowledge is language) becomes true by definition.
This can lead to some confusion. In Frank Jackson's Knowledge Argument[1], dualists say that qualia are knowledge, knowledge is language, and so if Mary could not learn qualia by studying science, then materialism is false. This argument trades on being inconsistent over what it means to have knowledge of something.
In important ways yes, and in important ways no. Yes: can deal with dynamic spatial environments, can act towards a goal (intentionality), some almost certainly can plan how to reache these goals. No: Use of symbols and recursive language.
I don't know if i'd go as far as to agree that "there is no knowledge, only language" .. but I 100% agree one of the key insights from GPT-3 -- why training on language is so effective in the first place -- is that language is tightly coupled to reality
I'm not sure that you can assert that language is tightly coupled to reality, unless you're using the term "reality" to mean something akin to "as one perceives the world" (regardless of whether that perception is correct or not).
Most expressions of language that survived from a few thousand years ago are centered around myths, and while those myths may have contained certain moral or ethical lessons (that were and are subject to interpretation) they certainly weren't tightly coupled to reality in an objective sense.
Training on expressions of language (I separate the concept of language itself from its expression in the form of writing, speaking, etc) certainly has use cases but can GPT-3 recognize a previously unknown analogy and correlate it with the proper piece of applicable "knowledge" it has? If not then it really has no understanding.
2. yes, language is bottlenecked by human perception, as are all things
3. even the notion of myth and fiction is encoded in language. language is self-descriptive and self-aware and you can separate sense from non-sense.
4. i'm not talking about knowledge or understanding, but of addressing the question of why training on language let's GPT-3 make human-like predictions as if it knows about reality? either it's a fluke, or it's because language as a whole is a model that approximates reality.
Why does it make human-like predictions as if it knows about reality? Because it's essentially pattern-matching the consensus of the literature it was trained on. Literature as a whole will tend to settle on a consensus sentiment, albeit one that is probably significantly behind the current consensus sentiment (it takes time to accumulate enough mass to move the weights). If your interactions with GPT-3 fall into the rather sizeable consensus that most people either subscribe to or are familiar with then it will certainly prove a decent mimic of understanding. If, however, you attempt to teach it a novel concept or if you dig into its interactions long enough to test for depth of understanding you run into the gaps and GPT-3 either begins to mimic that bullshit artist everyone knows that claims to know things but is only regurgitating platitudes and buzzwords or it begins to mimic behavior that would make you question whether it was sober and/or sane.
There's little doubt that GPT-3 could hold its own quite well in a bout of polite conversation and/or small talk that features in many social situations but that's more a commentary on the limited area of knowledge and behavior that etiquette expects for interactions in such settings.
It could also be trained on the canon of Shakespeare and behave as a prior work of Shakespeare, but if you left one play out of that canon and then attempted to have GPT-3 generate that missing work it wouldn't... at all. It doesn't approximate how Shakespeare thought, it approximates the literature he produced that you used for training.
None of this is to denigrate the achievement that GPT-3 represents, it is merely to point out that it is unfair to GPT-3 to attempt to hold it to the standards of AGI.
> metaphors only refer to other metaphors, i.e. language is circular.
Except we (humans) have real-life experience that gives meaning to those metaphors.
Humans are very good at visual memory, thanks to millions of years of evolutionary pressure to hunt game and gather for food. Hence why the best memory champions in the world mostly use the concept of "memory palaces." When you think of a car, do you see a car in your head or does a dictionary definition (i.e. purely language knowledge) of a car pop in your mind?
It's not that philosophical. If you visualize the GPT-3 embeddings using an embeddings projector then you can see the knowledge with your own eyes. That bypasses the need for GPT-3 to use language to communicate its knowledge to you. Computers aren't a black box like we've traditionally thought of brains.
If it's only philosophical, then me saying that Hacker News Website itself has 'knowledge' of everything we discuss about is also philosophical. Same can be applied to plain paper books.
How about any web application? A for loop? anything which can generate something for you?
> GPT-3 doesn't have any knowledge of how the world actually works.
I think this is a philosophical question. There is a view that, basically, there is no such thing as knowledge, just language (or, at least, there is no distinction between knowledge and language). In this view, all there really is is language, which is mostly composed of metaphors and, ultimately, metaphors only refer to other metaphors, i.e. language is circular. In this view, not only is the ultimate, physical, concrete world beyond us but also we can't even talk about it. From this perspective, GPT-3 is not substantively different than what our minds are doing.
That view makes some strong claims (I don't find it convincing), but it's out there. A slightly different claim, though, is that "knowledge of how (we think) the world actually works" is encoded in language. To me, that seems trivially true. So, again, how you take this quote from LeCun depends on what you think knowledge is and your view of the relationship between knowledge and language.