It should not come to anyone’s surprise that science and technology critics and linguistic scholars are very particular about the words used to describe products like ChatGPT.

Language informs. Language positions. Language influences. Language betrays. Language matters.

On this topic, Simon Willison recently wrote a thoughtful work of reflection called, What I should have said about the term Artificial Intelligence. It begins:

With the benefit of hindsight, I did a bad job with my post, It’s OK to call it Artificial Intelligence a few days ago.

I constructed the post in a confrontational way, reacting against a strawman argument. This wasn’t necessary, and it angered people who justifiably felt I was attacking them.

Reading it again now, the message I conveyed was not what I intended. People interpreted it as saying that their time spent arguing that AI is not intelligent was unproductive—a reasonable interpretation, given that I said “… I still don’t think this argument is a helpful contribution to the discussion.”

That was rude and unfair. I’m sorry. I regret writing and including that. I should have done better, more careful work with this.

Here’s a much improved version of what I wanted to say:

There are many good arguments for why the term Artificial Intelligence is problematic. I’ve been uncomfortable with it myself. Despite those arguments, I’ve decided to give in to the larger cultural momentum and invest in helping people understand what modern AI can do and how it works, rather than spending any time convincing them to call it something else.

This is (and should be) a personal decision! My big error here was implying that other people should do the same thing.

The rest of my original post was meant to justify my decision, focusing mainly on the academic history of the term and the confusion I felt around more accurate but less widely understood terminology like Large Language Models.

What I should have said about the term Artificial Intelligence, Simon Willison’s Weblog, 9th January 2024.

While most of the general public tends to refer to all generative large language models as chatgpt, I will continue to be a member of the pedantic web and use phrases like ‘large language models’ and ‘machine learning’ instead of ‘general artificial intelligence’ or AI.

I am on Team Stochastic Parrot. I am not on Team AI Rights.

I continue to do this because something important is at stake. The framing of the discussion of these nascent technological systems hold political consequence, and as such will hold economic and social consequences of significance.

Yesterday, I stumbled upon an article which had a title that was pure catnip to this librarian blogger: Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMs. Here is the abstract:

Are LLMs cultural technologies like photocopiers or printing presses, which transmit information but cannot create new content? A challenge for this idea, which we call bibliotechnism, is that LLMs often do generate entirely novel text. We begin by defending bibliotechnism against this challenge, showing how novel text may be meaningful only in a derivative sense, so that the content of this generated text depends in an important sense on the content of original human text. We go on to present a different, novel challenge for bibliotechnism, stemming from examples in which LLMs generate “novel reference”, using novel names to refer to novel entities. Such examples could be smoothly explained if LLMs were not cultural technologies but possessed a limited form of agency (beliefs, desires, and intentions). According to interpretationism in the philosophy of mind, a system has beliefs, desires and intentions if and only if its behavior is well-explained by the hypothesis that it has such states. In line with this view, we argue that cases of novel reference provide evidence that LLMs do in fact have beliefs, desires, and intentions, and thus have a limited form of agency.

arXiv:2401.04854 [cs.CL]

But before I dove into the text of the work, I did some lateral searching and I spent a few moments learning more about the authors and if anyone I knew had already noted their work. In doing, I found out that Prof. Emily M. Bender had already taken the authors of this work to task for not understanding her work as betrayed by the context of the citations of Bender and Koller 2020 in their text.

I’m going to take Dr. Bender’s word that the paper Are Language Models More Like Libraries or Like Librarians? is not worth the time to parse. But I still would like to raise the question: Are large language models like libraries or like librarians?

Are large language models cultural artifacts or are they agents?

So much rides on these answers. But before I explain why, first let me pull you quietly aside and make the case of why you really should join Team Cultural Artifact.

If you want to start where I did, you would come across a recommendation to read ChatGPT is an engine of cultural transmission from Memex 1.1 by John Naughton. And if you read that long and worthwhile essay by Henry Farrell, he will recommend that you spend some time with the work of psychologist Alison Gopnik on the “learning” of large language models as compared to children. Why is it so important to compare learning models between children and LLMs? Let me give the most straightforward reason first.

In what’s known as the classic “Turing test,” Alan Turing in 1950 suggested that if you couldn’t tell the difference in a typed conversation between a person and a computer, the computer might qualify as intelligent. Large language models are getting close. But Turing also proposed a more stringent test: For true intelligence, a computer should not only be able to talk about the world like a human adult—it should be able to learn about the world like a human child.

Gopnik, A. (2022, Jul 15). What AI still doesn’t know how to do; artificial intelligence programs that learn to write and speak can sound almost human—but they can’t think creatively like a small child can. Wall Street Journal. Online

While Gopnik’s Wall Street Journal article is paywalled, the research paper that it draws from is open access:

Our study has two components: an “imitation” component (making an interpolative judgment from existing knowledge about objects) and an “innovation” component (making an extrapolative judgment about the new ways that objects could be used). In the innovation part of the study, we present a series of problems in which a goal has to be executed in the absence of the typical tool (e.g., drawing a circle in the absence of a compass). We then provide alternative objects for participants to select: (a) an object that is more superficially similar to the typical tool and is associated with it but is not functionally relevant to the context (e.g., a ruler), (b) an object that is superficially dissimilar but has the same affordances and causal properties as the typical tool (e.g., a teapot that possesses a round bottom), and (c) a totally irrelevant object (e.g., a stove). In the imitation part of the study, we present the same sets of objects but ask participants to select which of the object options would “go best” with the typical tool (e.g., a compass and a ruler are more closely associated than a compass and a teapot).

Yiu, E., Kosoy, E., & Gopnik, A. (2023). Transmission Versus Truth, Imitation Versus Innovation: What Children Can Do That Large Language and Language-and-Vision Models Cannot (Yet). Perspectives on Psychological Science, 0(0). https://doi.org/10.1177/17456916231201401

I’m going to borrow Henry Farrel’s description of the outcome of this experiment:

You can see some of the consequences if you provide LLMs and humans with descriptions of real world physical problems, and ask them to describe how these problems might be solved without the usual tools. For example, Gopnik and her co-authors have investigated what happens when you ask LLMs and kids to draw a circle without a compass? You could ask both whether they would be better off using a ruler or a teapot to solve this problem LLMs tend to suggest rulers – in their maps of statistical associations between tokens, ‘rulers’ are a lot closer to ‘compasses’ than ‘teapots.’ Kids instead opt for the teapot – living in the physical universe, they know that teapots are round…

… But this has much broader and more interesting implications than making Sam Altman sad. Human beings learn in two kinds of ways – by imitating and by innovating. Gopnikism argues that LLMs are incapable of innovating, but they are good at imitating, and for some purposes at least, they are much better at it than human beings.

That is why we can think of LLMs as a cultural technology….

ChatGPT is an engine of cultural transmission, Henry Farrell, Programmable Mutter, Jan 11, 2024

I am planning to do more reading about this idea of LLM as cultural artifact and to follow some of the lines of that Alison Gopnik invites us to explore, as in this this short 15 minute talk from a year ago called Large Language Models as a Cultural Technology.

Ok, the side-track is over.

What is the harm in using the language that non-academics have given to use? Why am I banging on about my refusal to call LLMs a form of artificial intelligence?

I am writing this because I was moved to do after reading AI rights and human harms: From judiciary hearings in the US to a miscarriage of justice in the UK – should silicon have rights? by Helen Beetham, imperfect offerings, Jan 17, 2024:

With judiciary hearings on technology and journalism in the US Senate last week, it was not a week for tech or media commentators to stay quietly at home. Cue Jeff Jarvis, director of a Centre for Entrepreneurial Journalism, who sees many opportunities for collaboration between AI and print entrepreneurs, if only the Gutenburg relic of copyright law could be consigned to history. He took the opportunity to make this argument for ‘AI rights.’

What indeed. Jarvis has since claimed he meant many of these terms ‘metaphorically’, and he is of course free to use words as he chooses. But it is hardly an original or an innocent metaphor. What is at stake in these hearings is whether a particular kind of innovation –generative transformer models – offer such extraordinary opportunities that companies should be allowed to develop them regardless of legal frameworks, cultural conventions and evidence of harm. And in this dispute, ‘learning’ is one of the key terms that has been co-opted by the AI industry to make their case.

AI rights and human harms: From judiciary hearings in the US to a miscarriage of justice in the UK – should silicon have rights? by Helen Beetham, imperfect offerings, Jan 17, 2024

Helen put on my radar the development of something that proponents want us to call “The AI Rights Movement” (which I will only repeat using scare quotes).

Silicon lives matter

This is why we are seeing apparently rational, or at least sentient people calling for an AI rights movement, and tech advocates like Ray Kurzweil getting on board. For example, in thinking about AI consciousness, we are urged to reject ‘human bias’, ‘species chauvanism’ and ‘anthropocentrism’, terms taken directly from arguments for animal rights. The case for animal rights is a sophisticated and super interesting one, and I should get around to posting about it in relation to theories of ‘post-humanism’ (theories that also see technologies as having some kind of agency). But luckily I don’t have to step up to that here, because the tech guys are happy to use much less sophisticated comparisons.

When Nick Bostrom asserts ‘the principle of substrate non-discrimination’, he tells us that the idea of human beings having a different moral status to silicon systems is equivalent to ‘racism’. The difference between silicon and biology, like differences of skin tone (his definition of ‘race’, not mine), is something we should just get over…

AI rights and human harms: From judiciary hearings in the US to a miscarriage of justice in the UK – should silicon have rights? by Helen Beetham, imperfect offerings, Jan 17, 2024

You really should read the whole piece because Helen’s conclusion is that if we don’t try to embed human rights into into every stage of the AI lifecycle, those rights will not be learned.

Big tech has so far blasted through anti-trust legislation to establish its vast concentrations of capital. It is attempting to overturn copyright law, to ensure it can capture and process any content that it likes. I predict that it will come for human rights and the laws that defend them. Why? Because there is mounting evidence that generative models, like all data models at scale, like all inexplicable algorithms and black box systems in the hands of powerful states and corporations, present grave threats to human rights. This is borne out in an excellent report by the Council of Europe, AI in Education: a critical view through the lens of Human Rights, Democracy, and the Rule of Law. the sections on human rights and democracy in particular should be read by everyone working in education.

AI rights and human harms: From judiciary hearings in the US to a miscarriage of justice in the UK – should silicon have rights? by Helen Beetham, imperfect offerings, Jan 17, 2024

Likewise, I also think it’s worth reading Henry Farrell and Cosma Shalizi’s June 21st 2023, article in The Economist called, Artificial intelligence is a familiar-looking monster. Farrell and Shalizi are also Team Cultural Artifact, but their vision of LLMs are more like a Lovecraftian monster than a library catalogue.

But what such worries fail to acknowledge is that we’ve lived among shoggoths for centuries, tending to them as though they were our masters. We call them “the market system”, “bureaucracy” and even “electoral democracy”. The true Singularity began at least two centuries ago with the industrial revolution, when human society was transformed by vast inhuman forces. Markets and bureaucracies seem familiar, but they are actually enormous, impersonal distributed systems of information-processing that transmute the seething chaos of our collective knowledge into useful simplifications…

… Likewise, the political anthropologist James Scott has explained how bureaucracies are monsters of information, devouring rich, informal bodies of tacitly held knowledge and excreting a thin slurry of abstract categories that rulers use to “see” the world. Democracies spin out their own abstractions. The “public” depicted by polls and election results is a drastically simplified sketch of the amorphous mass of opinions, beliefs and knowledge held by individual citizens.

Lovecraft’s monsters live in our imaginations because they are fantastical shadows of the unliving systems that run on human beings and determine their lives. Markets and states can have enormous collective benefits, but they surely seem inimical to individuals who lose their jobs to economic change or get entangled in the suckered coils of bureaucratic decisions. As Hayek proclaims, and as Scott deplores, these vast machineries are simply incapable of caring if they crush the powerless or devour the virtuous. Nor is their crushing weight distributed evenly.

This is why I will not describe products like ChatGPT as Artificial General Intelligence. This is why I will avoid using the word learned when describing the behaviour of software, and will substitute that word with associated instead.

Your LLM is more like a library catalogue than a library but if you call it a library, I won’t be upset. I recognize that we are experiencing the development of new form of cultural artifact of massive import and influence.

But an LLM is not a librarian and I won’t let you call it that.

If anything, it is a Shoggoth with a librarian face.