# Active - Philosophy and Artificial Intelligence General thoughts: * [Problem](Problem.md) / [Problem Driven Epistemology](Problem%20Driven%20Epistemology.md) * [Problems Create a Logic of Reasoning](Problems%20Create%20a%20Logic%20of%20Reasoning.md) * [Explanationless Prediction Is Impossible](Explanationless%20Prediction%20Is%20Impossible.md) * [Reach and Constraints](Reach%20and%20Constraints.md) * [Scientific Measurements Use Chains of Proxies](Scientific%20Measurements%20Use%20Chains%20of%20Proxies.md) / [Grading Your Own Homework](Grading%20Your%20Own%20Homework.md) * ### Categorize * [Scientific Measurements Use Chains of Proxies](Scientific%20Measurements%20Use%20Chains%20of%20Proxies.md) * [Explanationless Prediction Is Impossible](Explanationless%20Prediction%20Is%20Impossible.md) * [Looking Under the Lamppost Because it is Brighter There](Looking%20Under%20the%20Lamppost%20Because%20it%20is%20Brighter%20There.md) * [Grading Your Own Homework](Grading%20Your%20Own%20Homework.md) ### Creativity Creativity, knowledge and explanation are fundamentally impossible to define. This is because once you have defined them, then you can setup a formal system in which they are confined. If you had a system that met that definition, then it would be confined to that and it could never produce anything outside of the system. For example, if it knew about arithmetic to the level of the postulates of Peano, it could *never* produce Godel's Theorem. This is because Godel's Theorem involves going *outside* of that system and explaining it. You could say that it is *not* defining something and executing the algorithm. Because it would always be an algorithm once it was in a framework. Could you then say that it is the ability to go outside the framework? TODO: **Why can't we define explanation? ** * [ChatGPT](https://chatgpt.com/share/e/677add06-b2d8-8006-99ba-5eb233de5610) ### Explanation is impossible to define David Deutsch does discuss the difficulty of formulating a closed definition of explanation in _The Beginning of Infinity_. His argument ties into his broader philosophy that knowledge and progress are potentially unbounded. He emphasizes that the concept of “explanation” cannot be fully captured by a strict formal definition because our capacity for creating new modes of explanation is itself unbounded. Deutsch argues that good explanations are characterized by their ability to be hard to vary while still accounting for the phenomena they address. This means that a good explanation must not only fit the data but also resist arbitrary modification without losing its explanatory power. However, he acknowledges that the process of creating explanations is an open-ended and creative endeavor, which means any attempt to rigidly define “explanation” would fail to accommodate the potentially infinite variety of explanatory frameworks humanity might develop in the future. This idea aligns with his views on fallibilism and the nature of scientific inquiry: our understanding and the frameworks we use to explain the universe are always provisional and subject to improvement. Thus, defining explanation in a closed way would contradict the infinite potential for discovery and innovation that Deutsch sees as fundamental to human progress. We need a system that can go outside itself. But wait, how can there be a universal explainer (and we know that something can universally explain), if we can't define explanation? Universal computation made sense because we can rigorously define computation and come up with a proof (good explanation) that universal computation is legit. But how can we have a universal explainer if we don't have a definition of explanation? Could there not be explanations we don't know of that couldn't be explained by "universal explainers"? Reasoning vs cached ideas * deriving planetary motion based on existing explanation + algorithm is not reasoning * coming up with explanation required reasoning + creativity Can AI advance if it does not have a problem to solve? * what does it mean to have a problem? We will always be grading our own homework * Why all this talk of goal post moving? Did people complain this much in past points of time in science? --- Okay, so two new arguments that I like. The first would be if you had an LLM that was, let's just say, at the level of like a, well we didn't even just say AGI for the purposes of this argument, it should be able to have an opinion. Again, right now it is just some retriever of information. If it was able to actually be some sort of a, you know, Doi-Chien, what you can't even say, or you know, you can call it a super intelligence for all I care. If it actually was that, then I should be able to ask it and say, you know, which of these theories of epistemology actually provides the best explanation? And that is not an arbitrary question. That is not a question that is, you know, purely, I guess it's some arbitrary relativeness that is a concrete objective question of which there is an actual answer. So I think the way out of that argument is to put it back on to the people who are effectively arguing, saying that this thing is, you know, smarter than anyone alive right now. You're left asking like, okay, well if you have all this information, like what is your opinion of it? You should be able to tell me what is the best explanation and I should be able to effectively, independently, vary and criticize that. And clearly we're nowhere near that right now. Now, in a sense, I think this does come back to something about, I like the idea that, you know, as it stands today, LLMs are quite poor at synthesis. You know, I think that that is effectively an objective fact. You know, if I were to say, synthesize, and not a cache synthesis, but again, two reason about these things. It has to form opinions. It has to have an internal problem and it can't use a cache reasoning strategy. And I think the challenge right now is like these things, again, they don't and can't synthesize. So it's very hard for it to actually have an opinion. Now, if you ask that question and then someone responds with a long trajectory and is able to, someone responds to the long trajectory and basically provides the mode of reasoning. Well, then again, that's a different story that could totally get you out of this situation. So I think that that's probably just something worth considering and keeping in mind here. Is if suddenly, if I pose this as a problem and and like, if we were to suddenly say, open AI, here's what we want to see that this is actually a, you know, it's, it can have opinions and it's genuinely intelligent. Well, what we're going to end up doing is like, you know, it can be, you're going to just have people training it with this, these trajectories. And you're going to have people slowly giving it trajectories to work on. Now, again, a key will be if these trajectories are noticeably garbage. Like, you know, if it comes back with all sorts of shit about moral and cultural relativism, then I'll know. Okay. Again, here's the problem. That constitutes a prediction. But again, an explanation is predictions impossible. If you understand something's reasoning, you have to actually understand what is going on beneath, beneath the surface. And right now we, we don't have that. So effectively, without, without some degree of explanation and just general explanatory power, what we're going to be left with is for the system. You're always in a basically the Chinese room, paradox or experiment or whatever it was called. And you won't be able to get away from that. --- So, one of the things that Alicia actually says is that you can't have a formal definition of explanation because that is fundamentally closed and explanation is fundamentally unbounded. I get that, but I think there's a confusion, a couple points for me that I need to clear up. One is like, why, if there's a concept of a universal explainer, how can we know that humans are universal explainers given that we don't have a definition, like a closed form definition of explanation? So, I think I need to better understand that. I also need to understand how, if we say we have a definition of computation that is universal, and now maybe the idea is there are computations that we can't perform. And so, in a sense, computation, again, this is probably why you need to get to chapter six, in a sense, computation is something that's not universal, but computation is not it's universal in the sense that a Turing machine is universal in the sense that it can, any Turing machine can perform any possible, a physically possible computation. So, again, we're back to this notion of physically possible and physically universal. So that might be something that helps us get around it, but I think you're still left with another problem, like, again, why, what is the reasoning for not being able to have a closed form open-ended definition of explanation? It feels like, again, is this a self-reference issue? Is there something I'm missing? You know, just not entirely sure there. --- # Hard To Vary * Okay, an interesting thought that's coming to mind is um if you are writing where you are coming up and needing to come up with a hard to very solution, large language models do a bad job. What is interesting is I just think a lot of what we do on a day to day basis is not hard to very. I think I would grant that. I think that that is actually true. It's like if I look at everything that we do it's like a lot of this stuff you're like okay yep you could have swapped in any word there. You know one of the reasons I think LLMs are so useful, I think it's kind of like we have this weird conflation. I think in terms of programming they are only, they're useful just because we have so many examples and this text is so unbelievably constrained in these examples. So again it's like it's kind of the perfect use case for a large language model. And then in terms of text of which you may write with it's so unbelievably frequently a lot of what we write, myself included, is so frequently easy to vary. It's easy to manipulate, it's easy to move around, you can swap in different words. You know LLMs constantly just kind of spitball a ton of stuff at you. When you try to get them to write something that is hard to vary, I mean I don't even think I literally feel like they're as useful as a random number generator spitting out text on the screen. That is how useless I have found it. And you could say that that's user error but then again it's like so then where's the intelligence lie? The intelligence lies in me. So I think basically it comes down to they're really good at two things. If they have pre-cached patterns, I think they're good at retrieving those. So that's just kind of like a text-based look up, if you will. And you may then say that so that's thing one. If you can just kind of retrieve a preset pattern and then I think honestly with this notion of temperature and kind of combining different things, a lot of stuff is just very easily combined. So a lot of stuff just, you're not actually producing good explanations. If you do, it's a pre-cached one and if you want an LLM to come up with a good explanation it is incredibly rare that it will be able to do so if it doesn't have a pre-cached pattern for it. The reason being good explanations are very hard to vary. So they're frequently going to have all sorts of things that are wrong. And again, so the counter argument to that may be well wait but like don't we have a kind of pre-cached pattern, they're sorry, don't we have good explanations? Wouldn't you say a program is a good explanation of something that you want to model? I would say yes but again that's a pre-cached pattern. I think that's just an interesting thing to riff on. * Hard to vary vs overfit? Think about cover parameters (if only one parameter did well, is that a hard to vary explanation or an overfit one)? Description vs explanation --- # Bad Arguments An interesting thing about the argument of more training data will just kind of create more, will lead to like an AGI is again, this actually goes back to a weird misleading form of kind of, it looks like first principles thinking, but it's really not. What you're really doing is you're just looking at what is logically possible. You're not looking at what it actually comprises a good explanation. So what I mean by that is like, you're basically saying, well, like we have all, we only know of one kind of generally intelligent thinking system, the human mind, human brain. That's the only one we know of. And we have a good understanding of how it works. Sorry, we don't have a great understanding of how it works, but the pieces that we do understand the philosophy, some under some pieces related to like interaction with the world, et cetera. We know that these systems do not have. And you could say, well, like, what if there's another means into creativity besides the one that we know of? Like, what if this is a, you know, this is one path, but it was kind of a, you know, there's a path dependence and, you know, kind of this, we have a parochial conception of what creativity, not what it is, but how it could have been created, how we could have arrived at it. And the counter-argument therapy, well, you could, again, you could basically say it with anything. Um, you know, you, without having a good explanation for it, it's like, you could literally do that with fucking anything. And that therefore is like a, a bad and an irrational way of reasoning. Now if it showed that, if it had showed that we were in somehow on our trajectory towards creativity or, or anything, but like there's just this big jump, um, that is also going against all that we know about the only system that is creative in the universe. We are basically going against the way that it ended up being creative. In which case you need an explanation for why. You can't just say, well, couldn't it be? Yes. You could say that about fucking anything though. Again, you could say that, you could, the same exact argument is used to say, well, uh, couldn't it be that all that really exists is my own mind and everything else is a figment of my imagination? So like solipsism? That is the exact same argument. It's couldn't it be? Isn't it technically possible? In other words, logically possible. In other words, it doesn't, there's not some contradiction in there. I wish there's not. But is it a good explanation? No. # **[Gell-Mann amnesia effect - Wikipedia](https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect)** # Response to Sasha: Why do I believe that this would be rewriting epistemology? Respond to Sasha: For problems of engineering I’d be hard pressed to bet against Elon. But this is a problem of epistemology—and his track record there is no better than that of a traffic cone. With that said there are obviously ways he could retreat on his object level claims and try to swap this out to be a problem of engineering and declare it a “success”. That is what I’m expecting to happen. I believe that for Elon to be correct, he would have to have discovered new epistemology? What did vaden say (AI podcast)? # Epistemology Focused Argument: Problems One idea that I'm floating is that in order to actually achieve AGI we would need a breakthrough in epistemology. Based on Poppers arguments, knowledge is created via *problems*. He has described a problem as a *clash* or a *conflict*. I have described them as a *violation of an expectation*. An expectation is based on a mismatch between your world model and the world (this can be the physical world or the world of abstractions). But, in order to have this violation you must have an expectation. And so we can see that in order to create new knowledge you must have expectations, and you must have problems. And you must be able to be surprised. And, more importantly, you must be able to be *curious*. But just what is curiosity? In some ways it is an internal decision to seek improvements to your world model—an admittance that your world model is incomplete. Consider how *I* update my world model. I start by considering it to be *weak* in some area. I try and then learn about a new idea there. As I'm learning, I am constantly checking—explicitly and implicitly—that his new idea fits in why my other ideas. This process is not perfect. Sometimes I miss something. But a theme is present: I am always trying to ensure that it is *consistent* with the other ideas that I hold. If it is not, I have to question whether it is my other idea that must be updated, or the new idea must be tweaked, or I am not understanding it properly, and so on. Under the hood this is a constant *trial and error*, *conjecture and criticism*. No new knowledge is created without this. Now consider modern LLMs. Does it have a world model? That is unclear, but for the sake argument, suppose it does. Two questions then arise: 1. Can it *update its world model to account for new knowledge that is handed to it*? This would mean *updating its weights*. The answer is that currently, it cannot unless it is retrained. Note: it is unclear if a small new piece of knowledge could be incorporated via a world model. For example: maybe it would just take it as *fact* and then stuff it on the side of some preexisting world model. In other words: it would not consistently integrate it (side note: this could be an interesting research question. How could you investigate this?) 1. Experiment: Train an LLM on a some data set with a set of ideas explicitly held out. Show that there is world model. Then, provide a small piece of knowledge that it never saw—it could retrain for all I care. Will that small piece of knowledge *be* connected to the other knowledge if I don't explicitly write a paragraph connecting it? 2. Can it *generate new knowledge* and then integrate it into it's world model? This is an even harder task than (1), for it must *generate the knowledge* and then integrate it. Remember that integration will require criticism. 1. Does integration require criticism? In one sense *no*. If I we posit a *fixed, static* world model, then the LLM could learn all it needs at training time (no criticism required). It would just be entirely based off of the corpus of text/knowledge it was learning from. Note that at this point it may not even have a world model. But for a dynamic, updatable world model yes criticism is required. Consider the two cases: (1) I give the LLM the new knowledge and tell it "this is 100% accurate and you should integrate it". Even in this case, it must be integrated *correctly*! Even if I tell you something is true, you still need to determine how to slot it into your world model. I think AI roughly tacks on an idea and then tries to come up with the most cohesive *looking* sentence/paragraph to follow. It is not properly slotting anything into its world model. Hence why you need to constantly provide criticism for it. Note that people who are not encountering this may not be on the bleeding edge of doing research, or working in domains that are more bespoke. (2) The LLM must generate the knowledge on its own. In this case criticism is clearly required. But what is interesting is that criticism can be *faked*. LLMs may have learned to mimic criticism—it looks like criticism, but it doesn't actually update the world model. 2. This would explain why LLMs so often can be prompted to have inconsistent views on things. It is also why I can Could you *program* [Conjecture and Criticism](Conjecture%20and%20Criticism.md)? Certainly! It is *physically possible* (human brains are a type of computer and [Universal Computers](Universal%20Computer.md) are physically possible). But are we programming it right now? Not even close.