So yeah, so intelligence explosion, I'm sure your family has the idea, but it's the idea that if you were to build general AI problem solving algorithms, well, the problem of building such an AI, that itself is a problem that could be solved by your AI. And maybe it could be solved better than what humans can do.

So your AI could start tweaking its own algorithm, could start being a better version of itself. And so on iteratively in a recursive fashion. And so you would end up with an AI with exponentially increasing intelligence. And I was basically questioning this idea, first of all, because the notion of intelligence explosion uses an implicit definition of intelligence that doesn't sound quite right to me.

It considers intelligence as a property of a brain that you can consider in isolation, like the height of a building, for instance. But that's not really what intelligence is. Intelligence emerges from the interaction between a brain, a body, like embodied intelligence, and an environment. And if you're missing one of these pieces, then you cannot redefine intelligence anymore.

So just tweaking a brain to make it smarter and smarter doesn't actually make any sense to me. So, first of all, you're crushing the dreams of many people, right? So there's a, let's look at like Sam Harris, actually a lot of physicists, Max Tegmark, people who think, you know, the universe is a information processing system.

Our brain is kind of an information processing system. So what's the theoretical limit? Like, it doesn't make sense that there should be some, it seems naive to think that our own brain is somehow the limit of the capabilities of this information. I'm playing devil's advocate here. This information processing system.

And then if you just scale it, if you're able to build something that's on par with the brain, you just, the process that builds it just continues and it'll improve exponentially. So that's the logic that's used actually by almost everybody that is worried about super human intelligence. - Yeah.

- So you're trying to make, so most people who are skeptical of that are kind of like, this doesn't, their thought process, this doesn't feel right. Like that's for me as well. So I'm more like, it doesn't, the whole thing is shrouded in mystery where you can't really say anything concrete, but you could say this doesn't feel right.

This doesn't feel like that's how the brain works. And you're trying to, with your blog posts and now making it a little more explicit. So one idea is that the brain doesn't exist alone. It exists within the environment. So you can't exponentially, you would have to somehow exponentially improve the environment and the brain together almost yet in order to create something that's much smarter in some kind of, of course we don't have a definition of intelligence.

- That's correct. That's correct. I don't think, if you look at very smart people today, even humans, not even talking about AIs, I don't think their brain and the performance of their brain is the bottleneck to the expressed intelligence, to their achievements. You cannot just tweak one part of the system, like of this brain body environment system and expect capabilities like what emerges out of this system to just, you know, explode exponentially.

Because anytime you improve one part of a system with many interdependencies like this, there's a new bottleneck that arises, right? And I don't think even today for very smart people, their brain is not the bottleneck to the sort of problems they can solve. Right? In fact, many very smart people today, you know, they're not actually solving any big scientific problems.

They're not Einstein. They're like Einstein, but you know, the patent clerk days, like Einstein became Einstein because this was a meeting of a genius with a big problem at the right time. Right? But maybe this meeting could have never happened. And then Einstein would have just been a patent clerk.

Right? And in fact, many people today are probably like genius level smart, but you wouldn't know because they're not really expressing any of that. Wow. That's brilliant. We can think of the world, earth, but also the universe as just as a space of problems. So all of these problems and tasks are roaming it of various difficulty.

And there's agents, creatures like ourselves and animals and so on that are also roaming it. And then you get coupled with a problem and then you solve it. But without that coupling, you can't demonstrate your quote unquote intelligence. Exactly. So intelligence is the meeting of great problem solving capabilities with a great problem.

And if you don't have the problem, you don't really express an intelligence. All you're left with is potential intelligence, like the performance of your brain or, you know, how high your IQ is, which in itself is just a number. Right. Yeah. Thank you. Thank you. Thank you.