Creativity and the problem of the improbable

> The universe is no narrow thing and the order within it is not constrained by any latitude in its conception to repeat what exists in one part in any other part.

Even in this world more things exist without our knowledge than with it and the order in creation which you see is that which you have put there, like a string in a maze, so that you shall not lose your way. For existence has its own order and that no man's mind can compass, that mind itself being but a fact among others.¹

It is often asked: how much better can LLMs get? This piece started as an answer to that question, although it has since ended up concerning a more specialised aspect. My sense of language models is that—apart from any broader limitations in terms of their exclusively modelling language—their limits will ultimately be those of language itself. Here I want to explain what I mean by this, in which I will consider specifically the difference between solidifying and extending performance.² The former is more important for integration into existing society, the latter for more radical visions.

Supposing that unsupervised learning works by acquiring primitives, the key is that they acquire reliable access to a flexible set of these. This allows for their sampling more or less creatively from the space of all possible sentences. There are two desires in terms of improving performance here, either to ensure their reliability for tasks which are already within their ambit—thus solidifying their performance—or to extend the tasks of which they are capable, an aim which may take several forms.

I am concerned here less with their augmentation but rather such extension as occurs in their own terms. The theory of flexible priors thus has a second aspect, at which we encounter the specific sense of extension in these models: the models learn not only priors but also the shape of their distribution; or really these cannot be separated, as meaning inheres in the wider shape—there are no isolated units in such systems.

The problem of creativity follows from this structuring of movement in logical space. Language models currently learn to move through this space in terms of probability, that this allows them to acquire the shape of meaning. They provide what an answer would look like, and the form of prompting here has proved surprisingly powerful. Based solely on this set of primitives we have been able to direct them into new areas.

The trouble is that their movement tends to follow the probability distribution, so when it comes to creative activity they will tend to reproduce the patterns that are already well-established. This is excellent for learning, even essential, but it is much less impressive for actual intelligence. We can examine this dynamic with reference to the elements that constitute a tree: heartwood, xylem, cambium, phloem, and bark.³

The heartwood is dead, what was once xylem has become calcified with compounds that ensure its resistance to decay. The xylem serves to transport water and nutrients from the roots to the cambium, which is the thin layer where growth occurs. The bark protects the structure as a whole, while the phloem transports sugars from the leaves.⁴

Applying this to language, we see that the bulk of language is heartwood and serves a structural purpose. This is the land of dead metaphor and literal language. All of this is obviously essential for the functioning of society—and hence the continuation of language as species—but this is not the place where we ought to expect any creativity.

We can see the xylem and phloem in turn as providing the structures which prepare the cambium, and here these must be related to the individual. The xylem can be considered akin to the institutions which support creativity, that these are the ties between the structure and the living individual. The phloem, in contrast, might be considered as akin to the specific experience of the individual. This is the qualitative source of nutrients for growth: their life—all that they have felt, witnessed, been.

The creative individual, nourished thus in life and mind, exists only at the very edges of this structure.⁵ This is the case in every field, and it ought to be so: the whole structure depends on this balance.⁶ The primary value is the survival of the species.

When it comes to creativity the specific area which we want to emulate is altogether rare. The actual creative action is entirely improbable, for its occurrence definitionally occurs at the edges.⁷ While the results of this creativity have come to serve ultimately as new rings in the heartwood, they do not originate there; nor can they thus be found.

The task, in other words, is to somehow make the improbable probable. This can be done with careful prompting, but even then I have noticed a tendency for inference to be torn between shapes that it recognises. Where there is a novel movement, it cannot follow the line smoothly; moves instead in a jagged manner. There is even a sort of schizophrenia here, a split between the two aspects that it cannot know as one.

We will undoubtedly see incredible advantages in terms of solidifying of performance, but whether this will translate into true creativity is unclear. The sort of experimental tests concerning creativity which are oft touted really reflect nothing of the sort, they test only the creativity we would expect from a relatively intelligent servant. The real question is whether it can be creative in the highest sense of human ingenuity.⁸ This is yet to be proven, and it is reasonable to believe that they will struggle with this task.

The sum of human science has been an interplay between solidifying and extending the fruits of its labour. While the bulk of the effort may well have been scientific and often enough algorithmic, the cutting edge of this process has always been poetic and inherently heuristic. These aspects are equally necessary for all that has followed from this joint enterprise—and it is perhaps more than anything the fact that this has been a joint enterprise which characterises the peculiarity of mankind. Whatever the major names of history, they have been supported by the whole edifice of human effort in all its necessary aspects: from the specifics of food, clothing, and shelter through to the whole material and spiritual structure that raises each infant to the heights of history.

This general structure has been our particular advantage, but it exists only as filtered in each instance through the particularity of an individual. Here is the contradiction at the heart of this process: that the particular becomes an individual only through being subsumed by the universal, that this assimilation is necessary for individuality in a truly human sense. Those wild children forgotten by the world or otherwise born of neglect are not individuals in this sense, are merely echoes of biology in an inert environment. The miracle of humanity is that we become individual through imitation.

---

1. Cormac McCarthy, Blood Meridian: “The truth about the world, he said, is that anything is possible. Had you not seen it all from birth and thereby bled it of its strangeness it would appear to you for what it is, a hat trick in a medicine show, a fevered dream, a trance bepopulate with chimeras having neither analogue nor precedent, an itinerant carnival, a migratory tentshow whose ultimate destination after many a pitch in many a mudded field is unspeakable and calamitous beyond reckoning.”

2. Here I am primarily talking about naked LLMs, with only a tangential interest in agents. I have written on the broader situation elsewhere, in which I argue that more important than creativity will be basic improvements that allow networks of LLMs to act as bureaucracies. There is the further possibility that these may be connected in unexpected ways, with results depending largely on the nature of training data. This is not dealt with here.

3. The key to this metaphor is the fact that the vast majority of the tree is dead, only the edge is alive. This is precisely the case for language; that growth occurs only at the edges of use.

4. I am not so certain what to do with bark here, since there are not really external threats in this metaphor. There is a role for an immune system, but that is different.

5. This is not to deny creativity to the ordinary, to the average. My considerations of creativity here relate to a retrospective point of view, thus consider only those that extend the broader structure in a historical sense. The individual on this view tends to operate within the space bounded by these structures, a fact which is most apparent in Kuhn’s treatment of science as a collective enterprise structured and bounded by its prevailing paradigms. There is work to be done within these areas, even plenty of creative work; but this is not what I mean.

6. We can see historical shifts in the relations here, especially if our image is broadened from merely considering language to thinking of culture as a whole, with the specific advantage of the United States seemingly to have come from its openness to creativity. The United Kingdom, in contrast, had stricter notions concerning the proper points for growth.

7. Temperature is not a true solution to this, since the aim is for the entire movement to be improbable rather than simply that of the next token. This requires a sustained creativity that really amounts to an improbable worldview, something which can only be imprecisely simulated by careful prompting at this point. Temperature, in contrast, generates variability up until a threshold and then seems to undergo a phase change; at which point we end up with something more like Brownian motion than the logical movement of language. The task is to maintain the tension between probability in terms of meaningful movement with this second-order improbability in terms of the angle and direction of this movement.

8. It is worth noting here that this is a problem for humans also, but this tendency towards a more conservative science has been accepted more or less as par for the course—depending on the dynamics at play, whether there is any institutional or egregoric interference. The whole is necessarily inertial, with this allowing for its continuance; and yet the limits are equally real, and equally problematic. My argument is intended for those who would herald a super-intelligence which is meant to transcend all of our merely human limitations.