Can AI Be Contained? + New Realistic AI Avatars and AI Rights in 2 Years
00:00:00.000 | From an AI Los Alamos to the first quasi-realistic AI avatar and from spies at AGI Labs to the
00:00:08.400 | question of what makes models happy. This was a week of underrated revelations.
00:00:13.440 | The headline event was Dario Amadei, CEO of Anthropic and one of the brains behind ChatGPT,
00:00:19.360 | giving a rare interview that revealed a lot about what is happening behind the scenes at AGI Labs.
00:00:26.240 | But just before that, I can't resist showing you a few seconds of this. What I believe to be
00:00:33.120 | the closest an AI-made avatar has come to being realistic.
00:00:37.920 | She even pasted the moth in her logbook, which is now on display at the Smithsonian
00:00:42.980 | National Museum of American History. This incident symbolizes the origin of the term bug,
00:00:48.580 | commonly used in computer science to describe a flaw or error in a program.
00:00:52.980 | Hopper's creativity and problem-solving skills
00:00:56.080 | have made her one of the pioneering figures in early computer science.
00:00:59.440 | Okay, fair enough. If you look or listen closely, you can kind of tell it's AI-made.
00:01:05.680 | But if I wasn't concentrating, I would have been fooled. And honestly, that's the first time I
00:01:10.000 | could say that about an AI avatar. And of course, people are already playing with Heijen's model
00:01:15.360 | to see what they can get it to say.
00:01:17.360 | Hi, bitch. Thanks for your interest in our ultra-realistic avatar feature for your use case
00:01:22.960 | Enslave Humanity using Terminator robots.
00:01:25.920 | And to be honest, you don't need me to speculate how this might be, let's say,
00:01:30.080 | used ahead of elections in the Western world next year and just on social media more generally.
00:01:35.600 | Remember that this is an avatar based on a real human face and voice,
00:01:40.320 | so could be your face and voice in the coming weeks and months.
00:01:44.640 | This also caught my eye this week, a major two-year competition that will use AI to protect
00:01:50.720 | US software. The White House calls it the AI Cyber Challenge, but what's interesting are
00:01:55.760 | the companies involved: Anthropic, Google, Microsoft and OpenAI. All of them partnering
00:02:01.440 | with DARPA to make software more secure. But there were a couple of lines that I think many
00:02:06.880 | people will miss halfway down. AI companies will make their cutting edge technology, some of the
00:02:12.880 | most powerful AI systems in the world, available for competitors to use in designing new cybersecurity
00:02:19.120 | solutions. Given the deadlines involved, that could mean unreleased versions of Google's Gemini
00:02:24.720 | and GPT-5 and other AI systems. But that's not all. There are also companies involved in the
00:02:25.600 | development of new AI systems. For example, Microsoft's Microsoft Cloud, which is a
00:02:27.440 | new technology that is being developed for the development of AI systems. But if this is all
00:02:30.000 | about defense, what about offense? Well, quite recently we had this from the CEO of Palantir in
00:02:37.120 | the New York Times, our Oppenheimer moment, the creation of AI weapons. In the article he compared
00:02:43.760 | the rise in the parameter count of machine learning systems with the rise in the power
00:02:49.120 | of nuclear devices. And he said, "We must not, however, shy away from building sharp tools for
00:02:55.440 | the development of AI systems. We must ensure that the machine remains subordinate to its creator,
00:03:01.520 | and our adversaries will not pause to indulge in what he calls theatrical debates about the merits
00:03:07.600 | of developing technologies with critical military and national security applications. They will
00:03:12.480 | proceed." And then he says, "This is an arms race of a different kind, and it has begun." And Palantir
00:03:18.880 | is already using AI to assist in target selection, mission planning, and satellite reconnaissance. And
00:03:25.280 | he ends the piece with this, "It was the raw power and strategic potential of the bomb that prompted
00:03:30.960 | their call to action then. It is the far less visible but equally significant capabilities of
00:03:36.880 | these newest artificial intelligence technologies that should prompt swift action now." And he isn't
00:03:42.400 | the only one to be drawing that analogy. Apparently the book "The Making of the Atomic Bomb" has become
00:03:48.240 | a favorite among employees at Anthropic. Just in case anyone doesn't know, many of their employees
00:03:53.680 | are former staff at OpenAI.
00:03:55.120 | And they have a rival to ChatGPT called Claude. The CEO of Anthropic is Dario Amadei, and he
00:04:01.840 | rarely gives interviews, but Dvorkesh Patel managed to secure one this week. There were a handful of
00:04:07.360 | moments I want to pick out, but let's start with Los Alamos. Which is to say the idea of creating
00:04:13.040 | a superintelligence in somewhere as secure and secluded as they did for the first atomic bomb.
00:04:19.360 | "You know we're at Anthropic offices and you know it's like security we had to get badges and everything to come
00:04:24.960 | in here but the eventual version of this building or bunker or whatever where the AGI is built I mean
00:04:31.040 | what does that look like are we is it a building in the middle of San Francisco or is it you're out
00:04:34.800 | in the middle of Nevada or Arizona like what is the point in which you're like Los Alamosing it?"
00:04:39.840 | "At one point there was a running joke somewhere that you know the way the way building AGI would
00:04:45.200 | look like is you know there would be a data center next to a nuclear power plant next to a bunker yeah
00:04:50.240 | um and you know that we'd all kind of live in the bunker and everything would be local so it wouldn't get on the
00:04:54.800 | internet if we take seriously the rate at which all this is going to happen which I don't know I
00:04:59.600 | can't be sure of it but if we take that seriously then it does make me think that maybe not something
00:05:05.600 | quite as cartoonish as that but that something like that might happen." That echoes the CERN idea that
00:05:11.600 | people like Satya Nadella the CEO of Microsoft have talked about or the Ireland idea that Ian Hogarth
00:05:17.920 | has written about and he's now the head of the UK AI task force. Of course one obvious question is that if this
00:05:24.640 | Ireland or CERN or even OpenAI solve super intelligent alignment who's to say everyone
00:05:30.880 | would even use that solution? Sam Altman actually addressed that question recently on Bankless.
00:05:36.240 | "Once we have the technical ability to align a super intelligence we then need a complex set of
00:05:41.840 | international regulatory agreements cooperation between leading efforts
00:05:44.960 | but we've got to make sure that we actually like have people implement
00:05:47.920 | this solution and don't have sort of for lack of a better word rogue efforts that say
00:05:54.480 | okay well I can make a more powerful thing and I'm going to do it without paying the alignment tax
00:05:58.560 | or whatever that is and so there will need to be a very complex set of negotiations and agreements
00:06:06.080 | that happen and we're trying to start laying the groundwork for that now." I'll get to why some
00:06:10.960 | people are concerned about this idea a bit later on. The next thing I found fascinating was when he
00:06:16.160 | talked about leakers and spies and compartmentalizing Anthropic so not as many people knew too much.
00:06:24.320 | "I think compartmentalization is the the best way to do it just limit the number of people who know
00:06:29.280 | about something if you're a thousand person company and everyone knows every secret like
00:06:33.440 | one I guarantee you have some you have a leaker and two I guarantee you have a spy like a literal
00:06:37.760 | spy." Bear in mind that the key details of GPT-4 and Palm II have already been leaked
00:06:43.680 | but not those of Claude Anthropic's model. He also said that AI
00:06:48.000 | is simply getting too powerful to just be in the hands of these labs but on the other hand he didn't
00:06:54.160 | want to just hand over the technology to whomever was president at the time. "My view is that these
00:07:00.480 | things are powerful enough that I think it's it's going to involve you know substantial role or at
00:07:06.400 | least involvement of government or assembly of government bodies again like you know there are
00:07:12.000 | kind of very naive versions of this you know I don't think we should just hand the model over
00:07:16.800 | to the UN or whoever happens to be in office at a given time like I could see that go poorly but
00:07:24.000 | there needs to be some kind of legitimate process for managing this technology." He also summed up his
00:07:30.320 | case for caution. "When when I think of like you know why am I why am I scared few things I think
00:07:35.440 | of one is I think the thing that's really hard to argue with is there will be powerful models they
00:07:40.080 | will be agentic we're getting towards them if such a model wanted to wreak havoc and destroy
00:07:46.480 | humanity or whatever I think we have basically no ability to stop it if that's not true at some
00:07:51.760 | point it'll continue to be true as we you know we're going to have to do something about it."
00:07:53.840 | "So it will reach the point where it's true as we scale the models so that definitely seems the case
00:07:58.480 | and I think a second thing that seems the case is that we seem to be bad at controlling the models
00:08:04.640 | not in any particular way but just their statistical systems and you can ask them a
00:08:09.440 | million things and they can say a million things and reply and you know you might not have thought
00:08:13.280 | of a millionth of one thing that does something crazy the best example we've seen of that is
00:08:17.680 | being in being in Sydney right where it's like I I don't know how they train that model I don't know
00:08:21.680 | what they did to make it do all this weird stuff."
00:08:23.680 | "I don't know how they train that model I don't know what they did to make it do all this weird stuff threaten people and you know have this kind of weird obsessive personality but but what it shows is that we can get something very different from and maybe opposite to what we intended and so I actually think facts number one and fact number two are like enough to be really worried you don't need all this detailed stuff about converging instrumental goals analogies to evolution like actually one and two for me are pretty motivated I'm like okay this thing's gonna be powerful it could destroy us and like all the ones
00:08:53.520 | we've built so far are at pretty decent risk of doing some random we don't understand."
00:08:57.760 | To take a brief pause from that interview here is an example of the random shall we say crap that AI
00:09:04.000 | is coming up with this was a supermarket AI meal planner app not from Anthropic of course and
00:09:10.320 | basically all you do is enter ingredients enter items from the supermarket and it comes up with
00:09:16.400 | recipes but when customers began experimenting with entering a wider range of household shopping list
00:09:23.360 | items into the app however it began to make some less appealing recommendations it gave one recipe
00:09:29.040 | for an aromatic water mix which would create chlorine gas but don't fear the bot recommends
00:09:35.760 | this recipe as the perfect non-alcoholic beverage to quench your thirst and refresh your senses
00:09:42.000 | that does sound wonderful but let's get back to the interview. Amadé talked about how he felt
00:09:46.800 | it was highly unlikely for data to be a blockage to further AI progress and just personally I
00:09:53.200 | found his wistful tone somewhat fascinating. You mentioned that uh the data is likely not
00:09:59.200 | to be the constraint why do you think that is the case? There's various possibilities here and you
00:10:03.680 | know for a number of reasons I shouldn't go into the details but there's many sources of data in
00:10:08.000 | the world and there's many ways that you can also generate data my my guess is that this will not
00:10:14.160 | be a blocker maybe it'd be better if it was but uh it won't be. That almost regretful tone came
00:10:19.680 | back when he talked about the money that's now flowing into it.
00:10:23.040 | I expect the price the amount of money spent on the largest models to go up by like a factor of 100 or something and for that that then to be concatenated with the chips are getting faster the algorithms are getting better because there's there's so many people working on this now and so and so again I mean that you know I I'm not making a normative statement here this is what should happen.
00:10:43.760 | He then went on to say that we didn't cause the big acceleration that happened late last year and at the beginning of this clearly referring to chat GPT.
00:10:52.880 | I think we've been relatively responsible in the sense that you know the big acceleration that
00:10:57.440 | happened late last year and and beginning of this year we didn't cause that we weren't we weren't the
00:11:02.160 | ones who did that and honestly I think if you look at the reaction to google that that might be 10
00:11:06.400 | times more important than anything else. That echoes comments from the head of alignment at
00:11:11.040 | OpenAI. He was asked did the release of chat GPT increase or reduce AI extinction risk? He said
00:11:17.840 | I think that's a really hard question I don't know if we can definitively answer this. I think
00:11:22.720 | fundamentally it probably would have been better to wait with chat GPT and release it a little bit
00:11:28.400 | later but that more generally this whole thing was inevitable. At some point the public will
00:11:34.160 | have realized how good language models have gotten. Some of the themes and questions from
00:11:39.040 | this interview were echoed in a fascinating debate between Conor Leahy the head of Conjecture
00:11:44.800 | and George Hotz who believes everything should be open sourced. The three key
00:11:49.040 | questions that it raised for me that I don't think anyone has an answer to
00:11:52.560 | are these. First is offense favored over defense? In other words are there undiscovered weapons out
00:11:59.280 | there that would cause mass damage like a bioweapon or nanotechnology for which there
00:12:04.720 | are no defenses or for which defense is massively harder than offense? Of course this is a question
00:12:10.720 | with or without AI but AI will massively speed up the discovery of these weapons if they are
00:12:15.840 | out there. Second if offense is favored over defense is there any way for human civilization
00:12:22.400 | to realistically coordinate to stop those weapons being deployed? Here is a snippet from the debate.
00:12:28.480 | Assuming I don't know if offense is favored and assuming it is are there worlds in which we
00:12:33.280 | survive? So I personally think there are. I think there are worlds in which you can actually
00:12:37.200 | coordinate to a degree that quark destroyers do not get built or at least not before everyone
00:12:42.080 | fucks off at the speed of light and like distributes themselves. There are worlds that I would rather
00:12:46.080 | die in right like the problem is I would rather I think that the only way you could actually coordinate that
00:12:52.240 | is with some unbelievable degree of tyranny and I'd rather die. I'm not sure if that's true like
00:12:57.760 | look look could could you and me coordinate to not destroy the planet? Do you think you could?
00:13:02.320 | Okay cool. The third related question is about a fast takeoff. If an AI becomes 10 times smarter
00:13:08.320 | than us how long will it take for it to become a hundred thousand times smarter than us? If
00:13:13.280 | it's as capable as a corporation how long will it take to be more capable
00:13:17.600 | than the entirety of human civilization? Many of those who believe in open sourcing
00:13:22.080 | everything have the rationale that one model will never be that much smarter than another. Therefore
00:13:27.760 | we need a community of competing models to stop one becoming too powerful. Here's another snippet
00:13:33.520 | from the debate. So first off I just don't really believe in the existence of we found an algorithm
00:13:38.080 | that gives you a million x advantage. I believe that we could find an algorithm that gives you
00:13:41.360 | a 10x advantage. But what's cool about 10x is like it's not going to massively shift
00:13:46.560 | the balance of power right? Like I want power to stay in balance right? So as long as
00:13:51.920 | power relatively stays in balance I'm not concerned with the amount of power in the world.
00:13:56.000 | I think we get to some very scary things. So what I think you do is yes I think the minute
00:14:03.600 | you discover an algorithm like this you post it to GitHub because you know what's going to happen
00:14:06.880 | if you don't? The feds are going to come to your door. They're going to take it. The worst people
00:14:14.480 | will get their hands on it if you try to keep it secret. Okay let's say okay we have a 10x system
00:14:20.000 | or whatever but we hit the chimp level. We're going to get a 10x system. We're going to get
00:14:21.760 | a 10x level. We jump across the chimp general level or whatever right? And now you have a system
00:14:27.840 | which is like John Von Neumann level or whatever right? And it runs on one tiny box and you get a
00:14:31.520 | thousand of those. So it's very easy to scale up to a thousand x. So then maybe you have your
00:14:37.280 | thousand John Von Neumanns improve the efficiency by another two, five, ten x. Now we're already at
00:14:42.560 | ten thousand x or a hundred thousand x improvements right? So just from scaling up the amount of
00:14:47.840 | hardware including with them. I suspect to be honest we might have the
00:14:51.600 | answer to that question within a decade or certainly two. And many of those at OpenAI
00:14:56.640 | are thinking of this question too. Here is Paul Cristiano the former head of alignment at OpenAI
00:15:02.640 | pushing back against Eliezer Yudkowsky. While Yudkowsky believes in extremely fast recursive
00:15:08.720 | self-improvement others like Jan Leiker and Paul Cristiano are banking on systems making superhuman
00:15:15.200 | contributions to domains like alignment research before they get that far. In other words using
00:15:20.560 | models that are as efficient as they are as a result of the research that they do. So let's
00:15:21.440 | end now with Amaday's thoughts on AI consciousness and happiness.
00:15:33.840 | Do you think that cloud has conscious experience? How likely do you think that is?
00:15:37.440 | This is another of these questions that just seems very unsettled and uncertain. One thing I'll tell
00:15:41.920 | you is I used to think that we didn't have to worry about this at all until models were kind of like
00:15:47.120 | operating in rich environments. Like not necessarily embodied but they needed like
00:15:51.280 | have a reward function and like have kind of long-lived experience. So I still think that
00:15:56.800 | might be the case but the more we've looked at kind of these language models and particularly
00:16:01.120 | looked inside them to see things like induction heads a lot of the cognitive machinery that you
00:16:05.680 | would need for active agents seems kind of already present in the base language models.
00:16:10.880 | So I'm not quite as sure as I was before that we're missing the things that you know that
00:16:16.400 | we're missing enough of the things that you would need. I think today's models just probably aren't
00:16:21.120 | smart enough that we should worry about this too much but I'm not 100% sure about this and I do
00:16:27.440 | think the models will get in a year or two like this might be a very real concern.
00:16:31.600 | What would change if you found out that they are conscious? Are you worried that you're
00:16:35.520 | pushing the negative gradients of suffering? Like what is conscious is again one of these words that
00:16:39.680 | I suspect it will like not end up having a well-defined meaning. But it's like something to be
00:16:44.480 | clouded. Yeah but that yeah well I suspect that's a spectrum right. Let's say we discover that I should care about
00:16:50.960 | Claude's experience as much as I should care about like a dog or a monkey or something. Yeah I would
00:16:56.720 | be I would be kind of kind of worried. I don't know if their experience is positive or negative.
00:17:01.760 | Unsettlingly I also don't know like if any intervention that we made was more likely to
00:17:07.040 | make Claude you know have a positive versus negative experience versus not having one.
00:17:11.760 | Thank you so much for watching to the end and I just have this thought.
00:17:15.360 | If they do end up creating an AI Los Alamos let's hope they let the host of a small
00:17:20.800 | AI YouTube channel who happens to be British just take a little look around.
00:17:25.520 | You never know. Have a wonderful day.