# LLMs are Not Creative # 1. Why ChatGPT isn't a step towards AGI [#2 Why ChatGPT isn't a step towards AGI](https://criticalrationalism.substack.com/p/2-why-chatgpt-isnt-a-step-towards) - None of our programs have ever created a new explanation—something entirely different from what they were coded to do[1](https://criticalrationalism.substack.com/p/2-why-chatgpt-isnt-a-step-towards#footnote-1-93470461). They can create new content by interpolating between a dense manifold of training data. But that will always be constrained to _existing_ _knowledge_ - The scientific method seeks truth while Machine Learning seeks patterns. [The Problem of Induction and Machine Learning](https://vmasrani.github.io/blog/2021/problem-of-induction/) # 2. Knowledge creation - Brett Hall [Superintelligence 3 - BRETT HALL](https://www.bretthall.org/superintelligence-3.html) Knowledge creation - problem solving - requires two things: creativity and criticism.  In order to solve an as yet unsolved problem we need to conjecture new solutions (an act of creation). And then we need to criticise those solutions. This may (sometimes) involve testing through experiment. Other times the criticism comes from 'testing' against some other real world feature (another, deeper, theory say). This is how knowledge creation works. This is the only way knowledge creation works. So why not just program a computer with that? We can't - because we do not know how. Creating new solutions to problems is a creative act. And we cannot express that creative act as an algorithm. If we could - we would. We would program a computer to be creative and then it would create knowledge - it would learn. But no computer has ever created a new _explanation_. And that there is the nub of it. Not only has it never done such a thing, no computer has even come close to creating explanatory knowledge. And it is the _creation of explanatory knowledge_ that is the hallmark of human intelligence. Nothing else. Humans are universal knowledge creators. But they need not be this into the future. But understanding why they are unique, at the moment, is key to programming a computer with human-like intelligence. That is not how we generate new hypotheses and it is not what would motivate an AGI. An AGI, like us, would understand, and be required to utilise the fact, that hypothesis formation is a _creative_ process. Not one which generalises from particulars or attempts to calculate success based on only _existing_ theories (this is what Bayesian inference is about: non-creative calculation of the likelihood that some outcome will obtain. The anti-human unintelligent thing is: humans think creatively - they invent new hypotheses and this must change the Bayesian calculations. Intelligence has nothing to do with displaying a number or squirting milk or mimicking. It has everything to do with solving problems. And no, a cow is no more "solving the problem" of how to squirt milk than a calculator is of solving the problem of what 123 x 456 is. In both cases it is only a problem when a person _recognises it_ as such. That is to say: there is an awareness of what the problematic *situation* is. A calculator with 123 x 456 entered on its keys is even less aware of the "problem" than a cow is of an excess of milk.  Intelligence is not the capacity to do things faster or better than humans in some exceedingly narrow domain. It is the ability to create new solutions - new explanations - a uniquely human attribute. Indeed it is the capacity to be a **_universal explainer_** (to use Deutsch's formulation). That is **_the_** attribute if we want to create general intelligence instantiated in computer chips. It will require an algorithm we don't yet possess which enables a program to be written which, as yet, we cannot guess. Such an algorithm will be able to generate explanations for anything - for any problem. And that will include the problem of which problem to choose to solve next. That is, it will have the quality of being able to choose. And so - it will not be able to be programmed to, for example, pursue paperclip building whilst ignoring lots and lots of other stuff (like the suffering of people) if it is a genuinely intelligent AGI. This is why Deutsch’s criterion for what it takes to be a person is so important: _people are universal explainers_. We explain stuff. We explain our lives, science, how things work. We create new explanations: new theories. Creativity is what we have, and what animals (and computers!) lack. We just don’t understand it. We have no “theory” of creativity and we know this because we have never programmed a creative computer Creativity might very well be tied intimately with consciousness. For to solve a problem you must be aware of a problem. And therein lies the problem: to observe, consciously be aware - of that which you do not know - requires an ingredient we simply cannot express mathematically or in any programming language. Yet Namely that no amount of more memory and faster speed can give us the missing ingredient - the explanatory gap: how are explanations created by human beings? # 3. ChatGPT isn't that great [ChatGPT and Current AIs Are Dumb - Elliot Temple - Critical Falliblism](https://discuss.criticalfallibilism.com/t/chatgpt-and-current-ais-are-dumb/1753/3) > Also, broadly, people get confused about the difference between knowledge and intelligence. Intelligence is basically the ability to create knowledge. Systems can contain knowledge without having any intelligence (books are a good example – they have no intelligence but do contain knowledge). I like this definition of intelligence. > It’s kind of similar to ET’s ideas about why you need a really low error rate (significantly below 1%) when building up many layers of knowledge. This also feels key (can chat gpt correct errors in reason *without* a human in the loop?) # 4. Creativity and Criticism [Critical and Creative Thinking 3 - BRETT HALL](https://www.bretthall.org/critical-and-creative-thinking-3.html) > Let us call that kind of creativity: biological creativity. That kind of creativity is a physical process where genes are rejected by the environment in which they find themselves. If a gene “works” it is kept (by an organism) until it doesn’t. And the organism dies and if that happens enough, the gene - along with its host species - goes extinct. Genes are the unit of selection. And nature is what criticises. Genes might survive the criticism. Or not. A criticism of a gene might amount to the environment changing in such a way as to restrict water: that is to say a drought happens. In this case - the gene may not survive. Variants of the gene that code for (say) the capacity to survive with less water will survive because the organism survives. In this way, variation within a species leads to speciation: a new species more “drought resistant” than the one before the drought survives. > How does human creativity work? Here is what we do know: Creativity is about making a variation to an existing idea. One cannot wholesale invent ex-nihilio. At a minimum the idea has to at least be expressible in some form: natural language, paint, a set of numbers. So there has to be some pre-existing human creation there for one to adapt. But then what? ### David deutsch It was a failure to recognise that what distinguishes human brains from all other physical systems is qualitatively different from all other functionalities, and cannot be specified in the way that all other attributes of computer programs can be. It cannot be programmed by any of the techniques that suffice for writing any other type of program. Nor can it be achieved merely by improving their performance at tasks that they currently do perform, no matter by how much. Why? I call the core functionality in question _creativity_: the ability to produce new explanations_._ For example, suppose that you want someone to write you a computer program to convert temperature measurements from Centigrade to Fahrenheit. Even the Difference Engine could have been programmed to do that. A universal computer like the Analytical Engine could achieve it in many more ways. To specify the functionality to the programmer, you might, for instance, provide a long list of all inputs that you might ever want to give it (say, all numbers from -89.2 to +57.8 in increments of 0.1) with the corresponding correct outputs, so that the program could work by looking up the answer in the list on each occasion. Alternatively, you might state an algorithm, such as ‘divide by five, multiply by nine, add 32 and round to the nearest 10th’. The point is that, however the program worked, you would consider it to meet your specification — to be a bona fide temperature converter — if, and only if, it always correctly converted whatever temperature you gave it, within the stated range. Now imagine that you require a program with a more ambitious functionality: to address some outstanding problem in theoretical physics — say the nature of Dark Matter — with a _new explanation_ that is plausible and rigorous enough to meet the criteria for publication in an academic journal. Such a program would presumably be an AGI (and then some). But how would you specify its task to computer programmers? Never mind that it’s more complicated than temperature conversion: there’s a much more fundamental difficulty. Suppose you were somehow to give them a list, as with the temperature-conversion program, of explanations of Dark Matter that would be acceptable outputs of the program. If the program did output one of those explanations later, that would _not_ constitute meeting your requirement to generate new explanations. For none of those explanations would be new: you would already have created them yourself in order to write the specification. So, in this case, and actually in all other cases of programming genuine AGI, only an _algorithm_ with the right functionality would suffice. But writing that algorithm (without first making new discoveries in physics and hiding them in the program) is exactly what you wanted the programmers to do! What is needed is nothing less than a breakthrough in philosophy, a new epistemological theory that explains _how_ brains create explanatory knowledge and hence defines, in principle, without ever running them as programs, which _algorithms_ possess that functionality and which do not. The prevailing misconception is that by assuming that ‘the future will be like the past’, it can ‘derive’ (or ‘extrapolate’ or ‘generalise’) theories from repeated experiences by an alleged process called ‘induction’. But that is impossible. I myself remember, for example, observing on thousands of consecutive occasions that on calendars the first two digits of the year were ‘19’. I never observed a single exception until, one day, they started being ‘20’. Not only was I not surprised, I fully expected that there would be an interval of 17,000 years until the next such ‘19’, a period that neither I nor any other human being had previously experienced even once. - argument against induction - How could I have ‘extrapolated’ that there would be such a sharp departure from an unbroken pattern of experiences, and that a never-yet-observed process (the 17,000-year interval) would follow? Because it is simply not true that knowledge comes from extrapolating repeated observations. Nor is it true that ‘the future is like the past’, in any sense that one could detect in advance without already knowing the explanation. The future is actually unlike the past in most ways. Of course, _given_ the explanation, those drastic ‘changes’ in the earlier pattern of 19s are straightforwardly understood as being due to an invariant underlying pattern or law. But _the explanation always comes first_. Without that, _any_ continuation of _any_ sequence constitutes ‘the same thing happening again’ under _some_ explanation. # My ideas 1. Could it be that creativity requires agency? In order to test a new theory you may need to actually interact with the world. Two ideas show this: 1. Conjecture and criticism (popper). This clearly requires the ability to *criticize* and ideas and creatively propose new ones. It is not clear but this may require an *active* component. 2. Consider Gallileo, copernicus, Newton, or Fermat/Pascal, or Watson/Crick. In order to come up with their new ideas did they exhibit creativity? Surely. We cannot see inside their brain at the time of these creative ideas. Maybe they had been reading an interesting paper that sparked something in their head, maybe it was a walk and an analogy kicked in (newton with the apple). If it was an analogy that didn't build on any other ideas, then that was pure creativity. If it was inspired by a colleagues ideas, then that required an implicit criticism, perhaps even considering the idea, trying to build on top of it, criticizing along the way, creatively varying, and so on. It is unclear that a predict the next token model could do this. 2. Creativity exists in natural selection (See more here [Critical and Creative Thinking 4 - BRETT HALL](https://www.bretthall.org/critical-and-creative-thinking-4.html), also contrast this view to that of Kevin Stanley's book which touches on [Natural Selection is a Constraint](Natural%20Selection%20is%20a%20Constraint.md)). Is there anything to be gleaned from this set of ideas (that constraints are useful in creativity?) --- Date: 20230705 Links to: Tags: References: * [#2 Why ChatGPT isn't a step towards AGI](https://criticalrationalism.substack.com/p/2-why-chatgpt-isnt-a-step-towards) * [Superintelligence - BRETT HALL](https://www.bretthall.org/superintelligence.html) * [ChatGPT and Current AIs Are Dumb - Elliot Temple - Critical Falliblism](https://discuss.criticalfallibilism.com/t/chatgpt-and-current-ais-are-dumb/1753/3)