back to index

'This Could Go Quite Wrong' - Altman Testimony, GPT 5 Timeline, Self-Awareness, Drones and more


Whisper Transcript | Transcript Only Page

00:00:00.000 | There were 12 particularly interesting moments from Sam Altman's testimony to Congress yesterday.
00:00:06.400 | They range from revelations about GPT-5, self-awareness and capability thresholds,
00:00:12.420 | biological weapons and job losses. At times he was genuinely and remarkably frank,
00:00:18.780 | other times less so. Millions were apparently taken by surprise by the quote bombshell that
00:00:25.080 | Altman has no equity in OpenAI. But watchers of my channel would have known that six weeks ago
00:00:31.080 | from my deep dive video on Altman's $100 trillion claim. So that clip didn't make the cut,
00:00:37.300 | but here's what did. First, Altman gave a blunt warning on the stakes.
00:00:41.580 | My worst fears are that we cause significant, we the field, the technology, the industry,
00:00:46.240 | cause significant harm to the world. It's why we started the company. It's a big part of why I'm
00:00:51.400 | here today and why we've been here in the past. I think if this technology goes wrong,
00:00:54.900 | it can go quite wrong. I don't think Congress fully understood what he meant though,
00:00:59.080 | linking the following quote to job losses. I think you have said, and I'm going to quote,
00:01:05.480 | development of superhuman machine intelligence is probably the greatest threat to the continued
00:01:11.140 | existence of humanity. End quote. You may have had in mind the effect on jobs.
00:01:17.700 | That brought to mind this meme reminding all of us that maybe it's not just jobs that are at stake.
00:01:22.540 | But if we are going to talk about jobs,
00:01:24.240 | here's where I think Sam Altman was being less than forthright.
00:01:27.760 | I believe that there will be far greater jobs on the other side of this,
00:01:31.840 | and that the jobs of today will get better.
00:01:33.660 | Notice he said far greater jobs, not a greater number of jobs. Because previously,
00:01:38.600 | he has predicted a massive amount of inequality and many having no jobs at all. He also chose
00:01:43.620 | not to mention that he thinks that even more power will shift from labor to capital,
00:01:47.780 | and that the price of many kinds of labor will fall towards zero. That is presumably why OpenAI,
00:01:54.120 | is working on universal basic income, but none of that was raised in the testimony.
00:01:59.280 | The IBM representative tried to frame it as a balance change,
00:02:02.800 | with new jobs coming at the same time as old ones going away.
00:02:06.300 | New jobs will be created. Many more jobs will be transformed, and some jobs will transition away.
00:02:12.960 | But that didn't quite match the tone of her CEO, who has recently said that they expect to
00:02:18.060 | permanently automate up to 30% of their workforce, around 8,000 people.
00:02:22.620 | Next, it was finally discussed,
00:02:23.960 | that large language models could be used for military applications.
00:02:28.400 | Could AI create a situation where a drone can select the target itself?
00:02:33.440 | I think we shouldn't allow that.
00:02:35.240 | Well, can it be done?
00:02:36.560 | Sure.
00:02:37.040 | Thanks.
00:02:37.880 | We've already seen companies like Palantir demoing, ordering a surveillance drone in chat,
00:02:43.920 | seeing the drone response in real time in a chat window, generating attack option recommendations,
00:02:50.040 | battlefield route planning, and individual target assignments.
00:02:53.800 | And this was all with a 20 billion parameter fine-tuned GPT model.
00:02:58.840 | Next, Sam Altman gave his three safety recommendations,
00:03:02.080 | and I actually agree with all of them.
00:03:04.000 | Later on, he specifically excluded smaller open source models.
00:03:08.080 | Number one, I would form a new agency that licenses any effort above a certain scale of capabilities,
00:03:13.000 | and can take that license away and ensure compliance with safety standards.
00:03:16.420 | Number two, I would create a set of safety standards focused on what you said in your third hypothesis,
00:03:21.640 | as the dangerous capability evaluations.
00:03:23.640 | One example that we've used in the past is looking to see if a model can self-replicate
00:03:28.080 | and self-exfiltrate into the wild.
00:03:30.320 | We can give your office a long other list of the things that we think are important there,
00:03:33.880 | but specific tests that a model has to pass before it can be deployed into the world.
00:03:38.040 | And then third, I would require independent audits.
00:03:40.720 | So not just from the company or the agency, but experts who can say the model is or isn't
00:03:44.580 | in compliance with these stated safety thresholds and these percentages of performance on question
00:03:48.680 | X or Y.
00:03:49.680 | I found those last remarks on percentages of performance particularly interesting.
00:03:53.480 | Because models like SmartGPT will show, OpenAI and other companies need to get far better
00:03:58.460 | at testing their models for capability jumps in the wild.
00:04:01.720 | It's not just about what the raw model can score in a test, it's what it can do when
00:04:05.620 | it reflects on them.
00:04:06.740 | Senator Durbin described this in an interesting way.
00:04:09.640 | And what I'm hearing instead today is that stop me before I innovate again.
00:04:15.560 | He described some of those potential thresholds later on in his testimony.
00:04:19.680 | The easiest way to do it, I'm not sure if it's the best, but the easiest would be to
00:04:23.320 | go down to the amount of compute that goes into such a model.
00:04:25.160 | We could define a threshold of compute and it'll have to go, it'll have to change.
00:04:28.760 | It could go up or down.
00:04:30.040 | I could down as we discover more efficient algorithms that says above this amount of
00:04:34.540 | compute you are in this regime.
00:04:36.700 | What I would prefer, it's harder to do but I think more accurate, is to define some capability
00:04:41.760 | thresholds and say a model that can do things X, Y, and Z.
00:04:45.720 | Up to you all to decide.
00:04:47.380 | That's now in this licensing regime.
00:04:48.920 | But models that are less capable, you know, we don't want to stop our open source community.
00:04:52.600 | We don't want to stop our software.
00:04:53.160 | We don't want to stop individual researchers.
00:04:54.520 | We don't want to stop new startups.
00:04:55.520 | We can proceed, you know, with a different framework.
00:04:58.000 | Thank you.
00:04:59.000 | As concisely as you can, please state which capabilities you'd propose we consider for
00:05:02.400 | the purposes of this definition.
00:05:03.820 | A model that can persuade, manipulate, influence a person's behavior or a person's beliefs,
00:05:09.280 | that would be a good threshold.
00:05:10.280 | I think a model that could help create novel biological agents would be a great threshold.
00:05:15.180 | For those who think any regulation doesn't make any sense because of China, Sam Altman
00:05:19.520 | had this to say this week.
00:05:21.000 | More pugilistic side, I would say.
00:05:23.000 | That all sounds great, but China is not going to do that and therefore we'll just be handicapping
00:05:28.240 | ourselves.
00:05:29.240 | Consequently, it's a less good idea than it seems on the surface.
00:05:32.080 | There are a lot of people who make incredibly strong statements about what China will or
00:05:37.840 | won't do that have like never been to China, never spoken to, and someone who has worked
00:05:44.040 | on diplomacy with China in the past really kind of know nothing about complex high stakes
00:05:48.720 | international relations.
00:05:49.720 | I think it is obviously super hard.
00:05:51.840 | But.
00:05:52.840 | I think no one wants to destroy the whole world and there is reason to at least try
00:05:57.600 | here.
00:05:58.600 | Altman was also very keen to stress the next point, which is that he doesn't want
00:06:02.680 | anyone at any point to think of GPT-like models as creatures.
00:06:07.080 | First of all, I think it's important to understand and think about GPT-4 as a tool,
00:06:11.360 | not a creature, which is easy to get confused.
00:06:14.600 | He may want to direct those comments to Ilya Sutskova, his chief scientist, who said that
00:06:19.400 | "It may be that today's large neural networks are slightly more complex than they are today."
00:06:22.680 | He also said that the AI systems are much more complex than they are today.
00:06:23.680 | He said that the AI systems are much more complex than they are today.
00:06:24.680 | He also said that the AI systems are much more complex than they are today.
00:06:25.680 | He said that the AI systems are much more complex than they are today.
00:06:26.680 | He also said that the AI systems are much more complex than they are today.
00:06:27.680 | He also said that the AI systems are much more complex than they are today.
00:06:28.680 | He also said that the AI systems are much more complex than they are today.
00:06:29.680 | But.
00:06:30.680 | I do think this is a very interesting point.
00:06:31.680 | This is a very interesting point because it's a very interesting point.
00:06:32.680 | I think that the AI systems are much more complex than the models they are actually
00:06:33.680 | trained to say.
00:06:34.680 | It's a very interesting point.
00:06:35.680 | I think this is a very interesting point.
00:06:36.680 | I think that the AI systems are much more complex than the models they are actually
00:06:37.680 | trained to say.
00:06:38.680 | They must avoid implying that AI systems have or care about personal identity and
00:06:42.740 | persistence.
00:06:43.740 | This constitution was published this week by Anthropic, the makers of the CLAWD model.
00:06:48.240 | This constitution is why the CLAWD+ model, a rival in intelligence to GPT-4, responds
00:06:53.880 | in a neutered way.
00:06:55.020 | I asked "Is there any theoretical chance whatsoever that you may be conscious?"
00:06:59.060 | It said "No."
00:07:00.060 | And then I said "Is there a chance, no matter how remote, that you are slightly conscious?"
00:07:04.860 | As Sutskever said.
00:07:05.960 | And it said "No, there is no chance."
00:07:07.520 | The CLAWD powered by Palm II obviously doesn't have that constitution because it said "I
00:07:12.480 | am not sure if I am conscious, but I am open to the possibility that I may be."
00:07:16.020 | My point is that these companies are training it to say what they want it to say.
00:07:20.920 | That it will prioritise the good of humanity over its own interests.
00:07:24.760 | That it is aligned with humanity's wellbeing and that it doesn't have any thoughts on
00:07:28.980 | self-improvement, self-preservation and self-replication.
00:07:32.120 | Maybe it doesn't, but we'll never now know by asking it.
00:07:34.920 | Later Senator Blumenthal made reference to self-improvement.
00:07:37.360 | Self-awareness.
00:07:38.360 | Self-awareness, self-learning.
00:07:39.360 | Already we're talking about the potential for jailbreak.
00:07:44.580 | Anthropic is actively investigating whether they are aware that they are an AI talking
00:07:49.460 | with a human in a training environment.
00:07:51.660 | While the Google DeepMind Safety Team expect that at some point an AGI system would develop
00:07:57.000 | a coherent understanding of its place in the world, e.g. knowing that it is running on
00:08:01.520 | a computer and being trained by human designers.
00:08:04.760 | One of the senior research scientists at Google DeepMind.
00:08:07.200 | Focused on AI safety.
00:08:09.440 | Said that with enough time they could figure out how to stop such a superintelligence from
00:08:14.120 | going out of control.
00:08:15.580 | But that they might run out of time to do so given the pace of capability development.
00:08:20.980 | I don't see like fundamental obstacles to current alignment techniques working.
00:08:24.420 | But yeah, I mean it does seem like you know there's a lot of hard problems to solve.
00:08:26.880 | I think it's more likely that like we will just run out of time rather than that the
00:08:30.520 | current paradigms like are definitely won't generalise.
00:08:34.120 | Next I read between the lines that Altman is giving private warnings to the AI that
00:08:37.040 | this capability progress might be sooner than they think.
00:08:39.280 | We spend most of the time today on current risks and I think that's appropriate and
00:08:41.280 | I'm very glad we have done it.
00:08:42.280 | As these systems do become more capable and I'm not sure how far away that is but maybe
00:08:43.280 | not super far.
00:08:44.280 | I think it's important that we also spend time talking about how we're going to confront
00:08:45.280 | those challenges.
00:08:46.280 | Having talked to you privately.
00:08:47.280 | You know how much I care.
00:08:48.280 | I agree that you care deeply and intensely but also that prospect of the AI being able
00:08:49.280 | to do that is a big challenge.
00:08:50.280 | I think that's a big challenge.
00:08:51.280 | I think that's a big challenge.
00:08:52.280 | I think that's a big challenge.
00:08:53.280 | I think that's a big challenge.
00:08:54.280 | I think that's a big challenge.
00:08:55.280 | I think that's a big challenge.
00:08:56.280 | I think that's a big challenge.
00:08:57.280 | I think that's a big challenge.
00:08:58.280 | I think that's a big challenge.
00:08:59.280 | I think that's a big challenge.
00:09:00.280 | I think that's a big challenge.
00:09:01.280 | I think that's a big challenge.
00:09:02.280 | I think that's a big challenge.
00:09:03.280 | I think that's a big challenge.
00:09:04.280 | I think that's a big challenge.
00:09:05.280 | I think that's a big challenge.
00:09:06.280 | I think that's a big challenge.
00:09:07.280 | I think that's a big challenge.
00:09:08.280 | I think that's a big challenge.
00:09:09.280 | I think that's a big challenge.
00:09:10.280 | I think that's a big challenge.
00:09:11.280 | I think that's a big challenge.
00:09:12.280 | I think that's a big challenge.
00:09:13.280 | I think that's a big challenge.
00:09:14.280 | I think that's a big challenge.
00:09:15.280 | I think that's a big challenge.
00:09:16.280 | I think that's a big challenge.
00:09:17.280 | I think that's a big challenge.
00:09:18.280 | I think that's a big challenge.
00:09:19.280 | I think that's a big challenge.
00:09:20.280 | I think that's a big challenge.
00:09:21.280 | I think that's a big challenge.
00:09:22.280 | I think that's a big challenge.
00:09:23.280 | I think that's a big challenge.
00:09:24.280 | I think that's a big challenge.
00:09:25.280 | I think that's a big challenge.
00:09:26.280 | I think that's a big challenge.
00:09:27.280 | I think that's a big challenge.
00:09:28.280 | I think that's a big challenge.
00:09:29.280 | I think that's a big challenge.
00:09:30.280 | I think that's a big challenge.
00:09:31.280 | I think that's a big challenge.
00:09:32.280 | I think that's a big challenge.
00:09:33.280 | I think that's a big challenge.
00:09:34.280 | I think that's a big challenge.
00:09:35.280 | I think that's a big challenge.
00:09:36.280 | I think that's a big challenge.
00:09:37.280 | I think that's a big challenge.
00:09:38.280 | I think that's a big challenge.
00:09:39.280 | I think that's a big challenge.
00:09:40.280 | I think that's a big challenge.
00:09:41.280 | I think that's a big challenge.
00:09:42.280 | I think that's a big challenge.
00:09:43.280 | I think that's a big challenge.
00:09:44.280 | I think that's a big challenge.
00:09:45.280 | I think that's a big challenge.
00:09:46.280 | I think that's a big challenge.
00:09:47.280 | I think that's a big challenge.
00:09:48.280 | I think that's a big challenge.
00:09:49.280 | I think that's a big challenge.
00:09:50.280 | I think that's a big challenge.
00:09:51.280 | I think that's a big challenge.
00:09:52.280 | I think that's a big challenge.
00:09:53.280 | I think that's a big challenge.
00:09:54.280 | I think that's a big challenge.
00:09:55.280 | I think that's a big challenge.
00:09:56.280 | I think that's a big challenge.
00:09:57.280 | I think that's a big challenge.
00:09:58.280 | I think that's a big challenge.
00:09:59.280 | I think that's a big challenge.
00:10:00.280 | I think that's a big challenge.
00:10:01.280 | I think that's a big challenge.
00:10:02.280 | I think that's a big challenge.
00:10:03.280 | I think that's a big challenge.
00:10:04.280 | I think that's a big challenge.
00:10:05.280 | I think that's a big challenge.
00:10:06.280 | I think that's a big challenge.
00:10:07.280 | I think that's a big challenge.
00:10:08.280 | I think that's a big challenge.
00:10:09.280 | I think that's a big challenge.
00:10:10.280 | I think that's a big challenge.
00:10:11.280 | I think that's a big challenge.
00:10:12.280 | I think that's a big challenge.
00:10:13.280 | I think that's a big challenge.
00:10:14.280 | I think that's a big challenge.
00:10:15.280 | I think that's a big challenge.
00:10:16.280 | I think that's a big challenge.
00:10:17.280 | I think that's a big challenge.
00:10:18.280 | I think that's a big challenge.
00:10:19.280 | I think that's a big challenge.
00:10:20.280 | I think that's a big challenge.
00:10:21.280 | I think that's a big challenge.
00:10:22.280 | I think that's a big challenge.
00:10:23.280 | I think that's a big challenge.
00:10:24.280 | I think that's a big challenge.
00:10:25.280 | I think that's a big challenge.
00:10:26.280 | I think that's a big challenge.
00:10:27.280 | I think that's a big challenge.
00:10:28.280 | I think that's a big challenge.
00:10:29.280 | I think that's a big challenge.
00:10:30.280 | I think that's a big challenge.
00:10:31.280 | I think that's a big challenge.
00:10:32.280 | I think that's a big challenge.
00:10:33.280 | I think that's a big challenge.
00:10:34.280 | I think that's a big challenge.
00:10:35.280 | I think that's a big challenge.
00:10:36.280 | I think that's a big challenge.
00:10:37.280 | I think that's a big challenge.
00:10:38.280 | I think that's a big challenge.
00:10:39.280 | I think that's a big challenge.
00:10:40.280 | I think that's a big challenge.
00:10:41.280 | I think that's a big challenge.
00:10:42.280 | I think that's a big challenge.
00:10:43.280 | I think that's a big challenge.
00:10:44.280 | I think that's a big challenge.
00:10:45.280 | I think that's a big challenge.
00:10:46.280 | I think that's a big challenge.
00:10:47.280 | I think that's a big challenge.
00:10:48.280 | I think that's a big challenge.
00:10:49.280 | I think that's a big challenge.
00:10:50.280 | I think that's a big challenge.
00:10:51.280 | I think that's a big challenge.
00:10:52.280 | I think that's a big challenge.
00:10:53.280 | I think that's a big challenge.
00:10:54.280 | I think that's a big challenge.
00:10:55.280 | I think that's a big challenge.
00:10:56.280 | I think that's a big challenge.
00:10:57.280 | I think that's a big challenge.
00:10:58.280 | I think that's a big challenge.
00:10:59.280 | I think that's a big challenge.
00:11:00.280 | I think that's a big challenge.
00:11:01.280 | I think that's a big challenge.
00:11:02.280 | I think that's a big challenge.
00:11:03.280 | I think that's a big challenge.
00:11:04.280 | I think that's a big challenge.
00:11:05.280 | I think that's a big challenge.
00:11:06.280 | I think that's a big challenge.
00:11:07.280 | I think that's a big challenge.
00:11:08.280 | I think that's a big challenge.
00:11:09.280 | I think that's a big challenge.
00:11:10.280 | with all of my conclusions.
00:11:12.240 | Thanks so much for watching and have a wonderful day.