WEBVTT

1
00:00:00.000 --> 00:00:05.040
In October over 850 experts, including yourself and other leaders like Richard Branson and Jeffrey

2
00:00:05.040 --> 00:00:11.040
Hinton, signed a statement to ban AI superintelligence as you guys raised concerns of potential human

3
00:00:11.040 --> 00:00:16.240
extinction. Because unless we figure out how do we guarantee that the AI systems are safe,

4
00:00:17.280 --> 00:00:21.600
we are toast. And you've been so influential on the subject of AI, you wrote the textbook.

5
00:00:21.600 --> 00:00:25.440
At many of the CEOs who are building some of the AI companies now would have studied on the

6
00:00:25.440 --> 00:00:33.520
subject of AI. So do you have any regrets? Professor Stuart Russell has been named one of

7
00:00:33.520 --> 00:00:39.200
Time Magazine's most influential voices in AI. After spending over 50 years researching,

8
00:00:39.200 --> 00:00:44.080
teaching and finding ways to design AI in such a way that humans maintain control.

9
00:00:44.080 --> 00:00:48.480
We talk about this gorilla problem as a way to understand AI in the context of humans.

10
00:00:48.480 --> 00:00:52.960
Yeah, so a few million years ago, a human line branched off from the gorilla line in evolution.

11
00:00:52.960 --> 00:00:56.880
And now the gorillas have no say in whether they continue to exist because we are much smarter than

12
00:00:56.880 --> 00:01:01.200
they are. Intelligence is actually the single most important factor to control planet Earth.

13
00:01:01.200 --> 00:01:03.840
Yep, but we're in the process of making something more intelligent than us.

14
00:01:04.640 --> 00:01:09.280
Exactly. Why don't people stop then? Well, one of the reasons is something called the Midas Touch.

15
00:01:09.280 --> 00:01:13.840
So King Midas is his legendary king who asked the gods, can everything I touch turn to gold.

16
00:01:13.840 --> 00:01:18.320
And we think of the Midas Touch as being a good thing, but he goes to drink some water, the water

17
00:01:18.320 --> 00:01:22.640
is turned to gold and he goes to comfort his daughter, his daughter turns to gold. So he dies in

18
00:01:22.640 --> 00:01:27.600
misery and starvation. So this applies to our current situation in two ways. One is that

19
00:01:27.600 --> 00:01:32.160
greed is driving these companies to pursue technology with the probabilities of extinction being

20
00:01:32.160 --> 00:01:36.800
worse than playing Russian roulette. And that's even according to the people developing the technology

21
00:01:36.800 --> 00:01:41.440
without our permission and people are just ruling themselves if they think it's naturally going

22
00:01:41.440 --> 00:01:47.680
to be controllable. So, you know, after 50 years I could retire, but instead I'm working 80 or

23
00:01:47.680 --> 00:01:52.320
100 hours a week trying to move things in the right direction. So if you had a button in front of

24
00:01:52.320 --> 00:01:55.920
you, which would stop all progress in artificial intelligence, would you press it?

25
00:01:57.840 --> 00:02:03.440
Not yet. I think there's still a decent chance to guarantee safety and I can explain more of what

26
00:02:03.440 --> 00:02:11.200
that is. Just give me 30 seconds of your time. Two things I wanted to say. The first thing is a

27
00:02:11.200 --> 00:02:15.840
huge thank you for listening and tuning into the show. Week after week means the world to all of

28
00:02:15.840 --> 00:02:20.000
us and this really is a dream that we absolutely never had and couldn't have imagined getting to

29
00:02:20.000 --> 00:02:25.200
this place. But secondly, it's a dream where we feel like we're only just getting started. And if

30
00:02:25.200 --> 00:02:30.640
you enjoy what we do here, please join the 24% of people that listen to this podcast regularly

31
00:02:30.640 --> 00:02:35.840
and follow us on this app. Here's a promise I'm going to make to you. I'm going to do everything in

32
00:02:35.840 --> 00:02:40.720
my power to make this show as good as I can now and into the future. We're going to deliver the

33
00:02:40.720 --> 00:02:44.880
guests that you want me to speak to and we're going to continue to keep doing all of the things you

34
00:02:44.880 --> 00:02:47.040
love about this show. Thank you.

35
00:02:54.640 --> 00:02:59.760
Professor Stuart Russell, OBE. A lot of people have been talking about AI for the last

36
00:02:59.760 --> 00:03:05.040
couple of years. It appears you've this really shocked me. It appears you've been talking about AI

37
00:03:05.040 --> 00:03:11.520
for most of your life. Well, I started doing AI in high school back in England. But then I did

38
00:03:11.520 --> 00:03:19.680
my PhDs starting in 82 at Stanford. I joined the Faculty of Berkeley in 86. So I'm my 40th

39
00:03:19.680 --> 00:03:25.600
year as a professor at Berkeley. The main thing that the AI community is familiar with in my work

40
00:03:27.200 --> 00:03:34.960
is a textbook that I wrote. Is this the textbook that most students who study AI are likely

41
00:03:34.960 --> 00:03:42.640
learning from? Yeah. So you wrote the textbook on artificial intelligence 31 years ago.

42
00:03:43.440 --> 00:03:48.240
You actually probably started writing it because it's so bloody big in the year that I was born.

43
00:03:48.240 --> 00:03:53.040
So I was born in 92. Yeah, it took me about two years to mean your book at the same age,

44
00:03:53.040 --> 00:03:58.800
which just a wonderful way for me to understand just how long you've been talking about this.

45
00:03:59.600 --> 00:04:04.080
And how long you've been writing about this. And actually, it's interesting that many of the

46
00:04:05.200 --> 00:04:09.760
CEOs who are building some of the AI companies now probably learnt from your textbook.

47
00:04:10.720 --> 00:04:16.320
You had a conversation with somebody who said that in order for people to get the message that

48
00:04:16.320 --> 00:04:21.200
we're going to be talking about today, there would have to be a catastrophe for people to wake up.

49
00:04:22.240 --> 00:04:26.640
Can you give me context on that conversation and adjust of who you had this conversation with?

50
00:04:27.200 --> 00:04:35.760
So it was with one of the CEOs of a leading AI company. He sees two possibilities as do I, which is

51
00:04:38.800 --> 00:04:45.200
either we have a small or let's say small scale disaster of the same scale as Chernobyl.

52
00:04:45.840 --> 00:04:51.600
But nuclear meltdown in Ukraine? Yeah, so this nuclear plant blew up in 1986,

53
00:04:52.400 --> 00:04:59.600
killed a fair number of people directly and maybe tens of thousands of people indirectly through

54
00:05:00.400 --> 00:05:09.600
radiation. Recent cost estimates more than a trillion dollars. So that would wake people up,

55
00:05:09.600 --> 00:05:14.880
that would get the governments to regulate. He's talked to the governments and they won't do it.

56
00:05:15.440 --> 00:05:23.120
So he looked at this Chernobyl scale disaster as the best case scenario because then the governments

57
00:05:23.120 --> 00:05:32.320
would regulate and require AI systems to be built. And is this CEO building an AI company?

58
00:05:32.320 --> 00:05:38.080
He runs one of the leading AI companies. And even he thinks that the only way that people

59
00:05:38.080 --> 00:05:42.560
will wake up is if there's a Chernobyl level nuclear disaster. Yeah, it wouldn't have to be a

60
00:05:42.560 --> 00:05:49.360
nuclear disaster. It would be either an AI system that's being misused by someone, for example,

61
00:05:49.360 --> 00:05:56.400
to engineer a pandemic or an AI system that does something itself, such as crashing our financial

62
00:05:56.400 --> 00:06:01.920
system or our communication systems. The alternative is a much worse disaster where we just lose control

63
00:06:02.880 --> 00:06:08.000
altogether. You have had lots of conversations with lots of people in the world of AI. Both

64
00:06:08.000 --> 00:06:13.680
people that have built the technology, have studied and researched the technology or the CEOs

65
00:06:13.680 --> 00:06:19.600
and founders that are currently in the AI race. What are some of the interesting sentiments that

66
00:06:19.600 --> 00:06:27.440
the general public wouldn't believe that you hear privately about their perspectives? Because I

67
00:06:27.440 --> 00:06:32.400
find that so fascinating. I've had some private conversations with people very close to these tech

68
00:06:32.480 --> 00:06:38.560
companies. And the shocking sentiment that I was exposed to was that they are aware of the risks

69
00:06:38.560 --> 00:06:42.560
often, but they don't feel like there's anything that can be done. So they're carrying on, which

70
00:06:42.560 --> 00:06:50.080
feels like a bit of a paradox to me. Yes, it must be a very difficult position to be in, in a

71
00:06:50.080 --> 00:06:57.200
sense, right? You're doing something that you know has a good chance of bringing an end to life

72
00:06:57.280 --> 00:07:06.240
on earth, including that of yourself and your own family. They feel that they can't escape this race.

73
00:07:07.680 --> 00:07:14.400
If a CEO of one of those companies was to say, you know, we're not going to do this anymore,

74
00:07:14.400 --> 00:07:20.640
they would just be replaced because the investors are putting their money up because they want to

75
00:07:21.360 --> 00:07:30.080
create AGI and read the benefits of it. So it's a strange situation where, at least all the ones

76
00:07:30.080 --> 00:07:37.200
I've spoken to, I haven't spoken to Sam Altman about this, but Sam Altman, even before

77
00:07:38.160 --> 00:07:46.640
becoming CEO of OpenAI, said that creating superhuman intelligence is the biggest risk

78
00:07:47.360 --> 00:07:52.960
to human existence that there is. My worst fears are that we cause significant. We the field,

79
00:07:52.960 --> 00:07:58.320
the technology, the industry, cause significant harm to the world. Elon Musk is also on record

80
00:07:58.320 --> 00:08:05.840
saying this. So Dario Emade estimates up to a 25% risk of extinction. Was there a particular moment

81
00:08:05.840 --> 00:08:12.960
when you realized that these CEOs are well aware of their extinction level risks?

82
00:08:13.600 --> 00:08:20.800
I mean, they all signed a statement in May of 23. It's called the extinction statement.

83
00:08:20.800 --> 00:08:27.280
It basically says AGI is an extinction risk at the same level as nuclear war and pandemics.

84
00:08:28.320 --> 00:08:34.240
But I don't think they feel it in their gut. Imagine that you were one of the nuclear

85
00:08:34.240 --> 00:08:41.120
physicists. I guess you've seen Oppenheimer. So you're there, you're watching that first nuclear

86
00:08:41.120 --> 00:08:51.040
explosion. How would that make you feel about the potential impact of nuclear war on a human race?

87
00:08:52.320 --> 00:08:58.880
I think you would probably become a pacifist and say this weapon is so terrible we have got to

88
00:08:58.880 --> 00:09:08.080
find a way to keep it under control. We are not there yet with the people making these decisions.

89
00:09:08.080 --> 00:09:17.840
And certainly not with the governments. What policymakers do is they listen to experts,

90
00:09:17.840 --> 00:09:26.960
they keep their finger in the wind. You got some experts dangling $50 billion checks and saying,

91
00:09:26.960 --> 00:09:32.160
oh, you know, all that Doomo stuff. It's just fringe nonsense. Don't worry about it.

92
00:09:32.160 --> 00:09:37.680
Take my $50 billion check. You know, on the other side, you've got very well-meaning brilliant

93
00:09:37.680 --> 00:09:42.320
scientists like Jeff Hinton saying, actually, no, this is the end of the human race.

94
00:09:43.520 --> 00:09:50.640
But Jeff doesn't have a $50 billion check. So the view is the only way to stop the race is if

95
00:09:50.720 --> 00:10:00.960
governments intervene and say, okay, we don't want this race to go ahead until we can be sure that

96
00:10:00.960 --> 00:10:08.880
it's going ahead in absolute safety. Closing off on your career journey, you've got to,

97
00:10:08.880 --> 00:10:14.240
you received an OB from Queen Elizabeth. Yes. And what was the listed reason for that for the award?

98
00:10:14.880 --> 00:10:20.160
Contributions to artificial intelligence research. And you've been listed as a time at

99
00:10:20.160 --> 00:10:26.080
magazine, most influential person in in AI several years in a row, including this year,

100
00:10:26.800 --> 00:10:32.400
in 2025. Now, there's two terms here that are central to the things we're going to discuss.

101
00:10:32.400 --> 00:10:39.120
One of them is AI and the other is AGI. And my muggle interpretation of that, it's artificial

102
00:10:39.200 --> 00:10:44.160
general intelligence is when the system, the computer, whatever it might be, the technology

103
00:10:44.160 --> 00:10:49.040
has generalized intelligence, which means that it could theoretically see, understand

104
00:10:50.480 --> 00:10:56.160
the world, it knows everything, it can understand everything in the world, as well as,

105
00:10:56.720 --> 00:11:03.120
or better than a human being can do it. And I think take action as well. I mean, some people say,

106
00:11:03.680 --> 00:11:09.120
AGI doesn't have to have a body, but a good chunk of our intelligence actually is about

107
00:11:09.120 --> 00:11:15.360
managing our body, about perceiving the real environment and acting on it, moving, grasping,

108
00:11:16.160 --> 00:11:23.280
and so on. So I think that's part of intelligence and AGI systems should be able to operate robots

109
00:11:23.280 --> 00:11:28.400
successfully. But there's often a misunderstanding, right? The people say, well, if it doesn't have a

110
00:11:28.400 --> 00:11:35.920
robot body, then it can't actually do anything. But then if you remember, most of us don't do things

111
00:11:35.920 --> 00:11:46.800
with our bodies. Some people do. Bricklayers, painters, gardeners, chefs, but people who do podcasts,

112
00:11:48.000 --> 00:11:54.080
you're doing it with your mind, right? You're doing it with your ability to produce language.

113
00:11:54.080 --> 00:12:01.520
Adolf Hitler didn't do it with his body. He did it by producing language.

114
00:12:01.520 --> 00:12:03.200
I hope you're not comparing us.

115
00:12:07.200 --> 00:12:16.720
But even an AGI that has no body, it actually has more access to the human race than Adolf Hitler

116
00:12:16.800 --> 00:12:25.200
ever did, because it can send emails and texts to what three quarters of the world's population

117
00:12:25.840 --> 00:12:33.520
directly. It also speaks all of their languages, and it can devote 24 hours a day

118
00:12:34.160 --> 00:12:39.920
to each individual person on Earth to convince them to do whatever it wants them to do.

119
00:12:39.920 --> 00:12:43.600
And our whole society runs down the internet. I mean, if there's an issue with the internet,

120
00:12:43.600 --> 00:12:48.560
everything breaks down in society. Our planes become grounded. We'll have electricity is running

121
00:12:48.560 --> 00:12:54.400
off as internet systems. So I mean, my entire life, which seems to run off the internet now.

122
00:12:55.280 --> 00:13:02.960
Yeah, water supplies. So this is one of the routes by which AI systems could bring about a

123
00:13:03.680 --> 00:13:09.200
medium-sized catastrophe is by basically shutting down our life support systems.

124
00:13:10.160 --> 00:13:18.480
Do you believe that at some point in the coming decades, we'll arrive at a point of AGI where

125
00:13:18.480 --> 00:13:24.160
these systems are generally intelligent? Yes, I think it's virtually certain,

126
00:13:25.360 --> 00:13:32.480
unless something else intervenes like a nuclear war or we made refrain from doing it,

127
00:13:32.480 --> 00:13:37.440
but I think it will be extraordinarily difficult for us to refrain.

128
00:13:38.320 --> 00:13:44.480
When I look down the list of predictions from the top 10 AICOs on when AGI will arrive,

129
00:13:44.480 --> 00:13:49.840
you've got Sam Altman, who's the founder of OpenAI slash tragedy beauty, says before 2030,

130
00:13:50.480 --> 00:13:59.600
Demis at DeepMind says 2030 to 2035. Jensen from Nivedia says around five years.

131
00:14:00.400 --> 00:14:07.760
Dario and Thropic says 2026, 2027, powerful AI-closed AGI. Elon says in the 2020s,

132
00:14:08.720 --> 00:14:13.360
and a go down the list of all of them, and they're all saying relatively within five years.

133
00:14:13.360 --> 00:14:20.640
I actually think it'll take longer. I don't think you can make a prediction based on engineering

134
00:14:20.960 --> 00:14:29.440
in the sense that, yes, we could make machines 10 times bigger and 10 times faster,

135
00:14:30.640 --> 00:14:34.720
but that's probably not the reason why we don't have AGI.

136
00:14:36.320 --> 00:14:41.920
In fact, I think we have far more computing power than we need for AGI.

137
00:14:42.720 --> 00:14:48.320
Maybe a thousand times more than we need. The reason we don't have AGI is because we don't

138
00:14:48.320 --> 00:14:58.880
understand how to make it properly. What we've seized upon is one particular technology called

139
00:14:58.880 --> 00:15:07.120
the language model, and we observed that as you make language models bigger, they produce text

140
00:15:07.120 --> 00:15:14.000
language that's more coherent and sounds more intelligent. And so mostly what's been happening

141
00:15:14.080 --> 00:15:19.680
in the last few years is just, okay, let's keep doing that, because one thing companies are very

142
00:15:19.680 --> 00:15:27.040
good at, unlike universities, is spending money. They have spent gargantuan amounts of money,

143
00:15:28.160 --> 00:15:35.040
and they're going to spend even more gargantuan amounts of money. We mentioned nuclear weapons,

144
00:15:35.040 --> 00:15:40.560
so the Manhattan Project in World War II to develop nuclear weapons.

145
00:15:41.680 --> 00:15:53.840
Its budget in 2025 dollars was about $20 billion. The budget for AGI is going to be a trillion

146
00:15:53.840 --> 00:16:00.720
dollars next year, so 50 times bigger than the Manhattan Project. Humans have a remarkable

147
00:16:00.720 --> 00:16:05.120
history of figuring things out when they galvanize towards a shared objective,

148
00:16:06.480 --> 00:16:12.640
thinking about the moon landings or whatever it might be through history. The thing that makes

149
00:16:12.640 --> 00:16:17.440
this feel quite inevitable to me is just the sheer volume of money being invested into it.

150
00:16:17.440 --> 00:16:20.720
I've never seen anything like it in my life. Well, there's never been anything like this in

151
00:16:20.720 --> 00:16:25.840
history. Is this the biggest technology project in human history by autism magnitude?

152
00:16:26.640 --> 00:16:33.520
And there doesn't seem to be anybody that is pausing to ask the questions about safety.

153
00:16:34.160 --> 00:16:37.520
It doesn't even appear that there's room for that in such a race.

154
00:16:38.240 --> 00:16:45.120
I think that's right. To varying extent, each of these companies has a division that focuses on

155
00:16:45.120 --> 00:16:50.880
safety. Does that division have any sway? Can they tell the other divisions? No, you can't release

156
00:16:50.880 --> 00:16:58.800
that system. Not really. I think some of the companies do take it more seriously. Anthropic

157
00:17:00.080 --> 00:17:05.600
does, I think, Google DeepMind. Even there, I think the commercial imperative

158
00:17:06.880 --> 00:17:17.280
to be at the forefront is absolutely vital. If a company is perceived as falling behind

159
00:17:18.160 --> 00:17:25.040
and not likely to be competitive, not likely to be the one to reach ADI first,

160
00:17:25.040 --> 00:17:27.680
then people will move their money elsewhere very quickly.

161
00:17:29.040 --> 00:17:33.200
And we saw some quite high-profile departures from companies like OpenAI,

162
00:17:33.920 --> 00:17:40.880
and a chap called Jan Leak left, who was working on AI Safety OpenAI.

163
00:17:41.600 --> 00:17:47.120
And he said that the reason for his leaving was that safety culture and processes have taken

164
00:17:47.120 --> 00:17:52.560
a backseat to shiny products at OpenAI, and he gradually lost trust in leadership. But also,

165
00:17:52.560 --> 00:18:00.560
Ilya Setskyva. So he was the co-founder and chief scientist for a while.

166
00:18:01.760 --> 00:18:04.480
So he and Jan Leak were the main safety people.

167
00:18:07.280 --> 00:18:14.080
And so when they say OpenAI doesn't care about safety, that's pretty concerning.

168
00:18:15.040 --> 00:18:17.600
I've heard you talk about this guerrilla problem.

169
00:18:18.800 --> 00:18:24.000
What is the guerrilla problem? As a way to understand AI in the context of humans.

170
00:18:24.000 --> 00:18:30.640
So the guerrilla problem is the problem that guerrillas face with respect to humans.

171
00:18:32.000 --> 00:18:37.760
So you could imagine that a few million years ago, the human line branched off from the guerrilla

172
00:18:37.760 --> 00:18:42.880
line in evolution. And now the guerrillas are looking at the human line saying,

173
00:18:43.760 --> 00:18:51.120
was that a good idea? And they have no say in whether they continue to exist.

174
00:18:52.240 --> 00:18:58.160
Because we are much smarter than they are. If we chose to, we could make them extinct in a

175
00:18:58.160 --> 00:19:04.080
couple of weeks. And there's nothing they can do about it. So that's the guerrilla problem.

176
00:19:04.320 --> 00:19:12.320
Just the problem of species faces when there's another species that's much more capable.

177
00:19:13.440 --> 00:19:17.120
And so this says that intelligence is actually the single most important factor to control

178
00:19:17.680 --> 00:19:24.880
planet Earth. Yes, intelligence is the ability to bring about what you want in the world.

179
00:19:25.520 --> 00:19:27.920
And we're in the process of making something more intelligent than us.

180
00:19:28.560 --> 00:19:32.480
Exactly. Which suggests that maybe we become the guerrillas.

181
00:19:32.560 --> 00:19:38.960
Exactly. Is there any fault in the reasoning there? Because it seems to make such perfect sense to me.

182
00:19:38.960 --> 00:19:46.400
But why don't people stop them? Because it seems like a crazy thing to want to.

183
00:19:47.120 --> 00:19:54.080
Because they think that if they create this technology, it will have enormous economic value.

184
00:19:54.080 --> 00:20:01.600
They'll be able to use it to replace all the human workers in the world, to develop new

185
00:20:03.440 --> 00:20:11.200
products, drugs, forms of entertainment. Anything that has economic value, you could use AGI to

186
00:20:12.160 --> 00:20:16.080
create it. And maybe it's just an irresistible thing in itself.

187
00:20:16.800 --> 00:20:26.480
Right. I think we as humans place so much store on our intelligence, you know, how we

188
00:20:28.000 --> 00:20:34.320
think about, you know, what is the pinnacle of human achievement? If we had AGI, we could go

189
00:20:35.200 --> 00:20:43.600
way higher than that. So it's very seductive for people to want to create this technology.

190
00:20:44.320 --> 00:20:52.080
And I think people are just fooling themselves if they think it's naturally going to be controllable.

191
00:20:53.440 --> 00:21:01.200
I mean, the question is, how are you going to retain power forever over entities more powerful than

192
00:21:01.200 --> 00:21:07.440
yourself? People say that sometimes in the comments section when we talk about AGI, they say,

193
00:21:07.440 --> 00:21:12.320
well, I'll just pull the plug out. Yeah, it's sort of funny. In fact, you know, reading the comments

194
00:21:12.320 --> 00:21:20.080
sections in newspapers whenever there's an AI article, there'll be people who say, oh, you can just

195
00:21:20.080 --> 00:21:23.680
pull the plug out, right? As if a super intelligent machine would never have thought of that one.

196
00:21:24.880 --> 00:21:29.280
Don't forget, it's watched all those films where they did try to pull the plug out. Another thing

197
00:21:29.280 --> 00:21:35.280
they say, well, you know, as long as it's not conscious, then it doesn't matter. It won't ever do

198
00:21:35.280 --> 00:21:47.200
anything, which is completely off the point because, you know, I don't think the gorillas are

199
00:21:47.200 --> 00:21:51.920
sitting there saying, oh, yeah, you know, if only those humans hadn't been conscious,

200
00:21:52.640 --> 00:21:57.520
everything would have been fine. Right? Of course not. What would make gorillas go extinct is

201
00:21:57.520 --> 00:22:05.440
the things that humans do, right? How we behave, our ability to act successfully in the world.

202
00:22:06.160 --> 00:22:14.320
So when I play chess against my iPhone and I lose, right? I don't think, oh, well, I'm losing because

203
00:22:14.320 --> 00:22:20.000
it's conscious, right? Now, I'm just losing because it's better than I am in that little world,

204
00:22:20.960 --> 00:22:28.560
moving the bits around to get what it wants. And so consciousness has nothing to do with it,

205
00:22:28.560 --> 00:22:36.880
right? Competence is the thing we are concerned about. So I think the only hope is, can we simultaneously

206
00:22:38.000 --> 00:22:47.680
build machines that are more intelligent than us, but guarantee that they will always act in our

207
00:22:47.680 --> 00:22:53.200
best interests. So throwing that question to you, can we build machines that are more intelligent

208
00:22:53.200 --> 00:23:00.240
than us that will also always act in our best interests? It sounds like a bit of a contradiction

209
00:23:00.240 --> 00:23:05.360
to some degree because it's kind of like me saying, I've got a French bulldog called Pablo.

210
00:23:06.000 --> 00:23:12.160
That's nine years old. And it's like saying that he could be more intelligent than me, yet I still

211
00:23:12.160 --> 00:23:17.200
walk him in decide when he gets fed. I think if he was more intelligent than me, he would be walking

212
00:23:17.200 --> 00:23:23.520
me. I'd be on the leash. That's the, that's the trick, right? Can we make AI systems whose only

213
00:23:23.520 --> 00:23:32.320
purpose is to further human interests? And I think the answer is yes. And this is actually what I've

214
00:23:32.320 --> 00:23:38.720
been working on. So I think one part of my career that I didn't mention is is sort of having this

215
00:23:38.720 --> 00:23:48.240
epiphany while I was on sabbatical in Paris. So it was 2013 or so just realizing that

216
00:23:49.040 --> 00:23:59.600
further progress in the capabilities of AI, if we succeeded in creating real superhuman intelligence

217
00:23:59.600 --> 00:24:05.360
that it was potentially a catastrophe. And so I pretty much switched my focus

218
00:24:06.000 --> 00:24:12.240
to work on how do we make it so that it's guaranteed to be safe. Are you somewhat troubled by

219
00:24:14.160 --> 00:24:19.680
everything that's going on at the moment with AI and how it's progressing? Because you strike me

220
00:24:19.680 --> 00:24:26.240
as someone that's somewhat troubled under the surface by the way things are moving forward

221
00:24:26.880 --> 00:24:32.640
and the speed in which they're moving forward. That's an understatement. I'm appalled

222
00:24:33.600 --> 00:24:40.400
actually by the lack of attention to safety. I mean imagine if someone's building a nuclear power

223
00:24:40.400 --> 00:24:48.000
station in your neighborhood. And you go along to the chief engineer and you say, okay,

224
00:24:48.000 --> 00:24:52.560
these nuclear things, I've heard that they can actually explode, right? There was this nuclear

225
00:24:52.560 --> 00:24:59.280
explosion that happened in Hiroshima. So I'm a bit worried about this. You know, what steps are

226
00:24:59.360 --> 00:25:03.680
you taking to make sure that we don't have a nuclear explosion in our backyard?

227
00:25:05.040 --> 00:25:09.520
And the chief engineer says, well, we thought about it. We don't really have an answer.

228
00:25:13.680 --> 00:25:18.800
You would, what would you say? I think you would, you would use some expletives.

229
00:25:21.360 --> 00:25:29.040
Well, and you'd call your MP, you know, get these people out. I mean, what are they doing?

230
00:25:30.240 --> 00:25:37.280
You read out the list of, you know, projected dates for AGI. But notice also that those people,

231
00:25:38.480 --> 00:25:44.800
I think I mentioned Dara Amadeh says a 25% chance of extinction. Elon Musk has a 30% chance of

232
00:25:44.800 --> 00:25:52.320
extinction. Sam Altman says basically that AGI is the biggest risk to human existence.

233
00:25:53.440 --> 00:25:58.240
So what are they doing? They are playing Russian relit with every human being on Earth.

234
00:26:00.080 --> 00:26:05.120
Without our permission, they're coming into our houses, putting a gun to the head of our children,

235
00:26:06.400 --> 00:26:12.080
pulling a trigger and saying, well, you know, possibly everyone will die. Oops.

236
00:26:12.880 --> 00:26:14.640
But possibly we'll get incredibly rich.

237
00:26:17.600 --> 00:26:24.240
That's what they're doing. Did they ask us? No. Why is the government allowing them to do this?

238
00:26:25.200 --> 00:26:29.120
Because they dangle 50 billion dollar checks in front of the governments.

239
00:26:30.240 --> 00:26:36.000
So I think troubled under the surface is an understatement. What would be an accurate statement?

240
00:26:37.360 --> 00:26:47.760
A pulled. And I am devoting my life to trying to divert from this course of history into a different

241
00:26:47.760 --> 00:26:54.000
way. Do you have any regrets about things you could have done in the past because you've been so

242
00:26:54.000 --> 00:26:57.920
influential on the subject of AGI? You wrote the textbook that many of these people would have

243
00:26:57.920 --> 00:27:03.360
studied on the subject of AGI more than 30 years ago. When you're alone at night and you think

244
00:27:03.360 --> 00:27:07.280
about decisions you've made on this field because of your scope of influence, is there anything

245
00:27:07.280 --> 00:27:16.640
you regret? Well, I do wish I had understood earlier what I understand now. We could have developed

246
00:27:18.000 --> 00:27:23.120
safe AI systems. I think there are some weaknesses in the framework, which I can explain.

247
00:27:23.680 --> 00:27:30.320
But I think that framework could have evolved to develop actually safe AI systems where we could

248
00:27:30.320 --> 00:27:37.280
prove mathematically that the system is going to act in our interests. The kind of AI systems

249
00:27:37.280 --> 00:27:42.640
we're building now, we don't understand how they work. We don't understand how they work.

250
00:27:43.360 --> 00:27:48.400
It's a strange thing to build something where you don't understand how it works. There's no

251
00:27:48.400 --> 00:27:52.480
comparable through human history. Usually with machines, you can pull it apart and see what

252
00:27:52.480 --> 00:27:59.280
clogs are doing. How they think? Well, actually, we put the cogs together. With most machines,

253
00:27:59.280 --> 00:28:04.160
we designed it to have a certain behavior. We don't need to pull it apart and see what the

254
00:28:04.160 --> 00:28:09.520
cogs are because we put the cogs in there in the first place. One by one, we figured out what

255
00:28:09.520 --> 00:28:14.720
the pieces needed to be, how they worked together to produce the effect that we want.

256
00:28:14.720 --> 00:28:24.000
The best analogy I can come up with is the first cave person who left a bowl of fruit in the

257
00:28:24.000 --> 00:28:30.080
sun and forgot about it. Then came back a few weeks later and there was this big soupy thing

258
00:28:30.080 --> 00:28:36.720
and they drank it and got completely shit-faced. They got drunk. They got this effect. They had no

259
00:28:36.720 --> 00:28:42.720
idea how it worked, but they were very happy about it. No doubt that person made a lot of money

260
00:28:42.720 --> 00:28:50.800
from it. It is bizarre, but my mental picture of these things is like a chain link fence.

261
00:28:52.800 --> 00:28:59.680
You've got lots of these connections and each of those connections can be its connection strength

262
00:28:59.680 --> 00:29:08.240
can be adjusted. Then a signal comes in one end of this chain link fence and passes through all

263
00:29:08.240 --> 00:29:13.440
these connections and comes out the other end. The signal that comes out the other end is affected

264
00:29:13.440 --> 00:29:20.400
by your adjusting of all the connection strengths. What you do is you get a whole lot of training

265
00:29:20.400 --> 00:29:24.960
data and you adjust all those connection strengths so that the signal that comes out the other end

266
00:29:24.960 --> 00:29:33.360
of the network is the right answer to the question. If your training data is lots of photographs of

267
00:29:33.360 --> 00:29:41.040
animals, then all those pixels go in one end of the network and out the other end, it activates

268
00:29:41.040 --> 00:29:49.200
the alarm output or the dog output or the cat output or the ostrich output. You just keep adjusting

269
00:29:49.200 --> 00:29:53.760
all the connection strengths in this network until the outputs of the network are the ones you want.

270
00:29:53.840 --> 00:29:57.360
But we don't really know what's going on across all of those different chains.

271
00:29:57.360 --> 00:30:01.360
So what's going on inside that network? Well, so now you have to imagine that

272
00:30:02.000 --> 00:30:07.120
this network, this chain link fence, is a thousand square miles in extent.

273
00:30:07.920 --> 00:30:14.640
Okay. So it's covering the whole of the San Francisco Bay Area or the whole of London inside

274
00:30:14.640 --> 00:30:19.040
the M25. That's how big it is. And the lights are off. It's nighttime.

275
00:30:19.920 --> 00:30:27.200
So you might have, in that network, about a trillion adjustable parameters and then you do

276
00:30:27.200 --> 00:30:34.960
quintillions or sextillions of small random adjustments to those parameters until you get the

277
00:30:34.960 --> 00:30:41.120
behavior that you want. I've heard Sam Altman say that in the future, he doesn't believe they'll

278
00:30:41.120 --> 00:30:46.000
need much training data at all to make these models progress themselves because they're

279
00:30:46.000 --> 00:30:51.760
comes a point where the models are so smart that they can train themselves and improve themselves

280
00:30:53.200 --> 00:30:58.400
without us needing to pump in articles and books and scour the internet.

281
00:30:58.400 --> 00:31:02.560
Yeah, it should, it should work that way. So I think what he's referring to, and this is

282
00:31:02.560 --> 00:31:10.800
something that several companies are now worried might start happening, is that the AI system

283
00:31:10.800 --> 00:31:22.640
becomes capable of doing AI research by itself. And so you have a system with a certain capability.

284
00:31:22.640 --> 00:31:28.720
I mean, crudely, we could call it an IQ, but it's not really an IQ. But anyway, imagine that

285
00:31:29.360 --> 00:31:36.560
it's got an IQ of 150. It uses that to do AI research. Comes up with better algorithms or better

286
00:31:36.560 --> 00:31:43.680
designs for hardware or better ways to use the data. Updates itself, now it has an IQ of 170.

287
00:31:44.800 --> 00:31:49.440
And now it does more AI research, except that now it's got an IQ of 170. So it's even better

288
00:31:50.320 --> 00:31:58.400
at doing the AI research. And so, you know, next iteration is 250 and so on. So this is an idea

289
00:31:58.480 --> 00:32:06.160
that one of Alan Turing's friends, IJ Good, rode out in 1965 called the intelligence explosion,

290
00:32:06.160 --> 00:32:12.960
right? That one of the things an intelligent system could do is to do AI research and therefore

291
00:32:12.960 --> 00:32:21.520
make itself more intelligent. And this would very rapidly take off and leave the humans far behind.

292
00:32:21.520 --> 00:32:24.560
Is that what they call the fast takeoff? That's called the fast takeoff.

293
00:32:24.560 --> 00:32:30.480
Sam Ottman said, I think a fast takeoff is more possible than I thought a couple of years ago,

294
00:32:30.480 --> 00:32:35.520
which I guess is that moment where the AI starts teaching itself. In his blog, the gentle

295
00:32:35.520 --> 00:32:43.360
singularity, he said, we may already be past the event horizon of takeoff. What does he mean by event

296
00:32:43.360 --> 00:32:51.360
horizon? The event horizon is a phrase borrowed from astrophysics and it refers to the black hole.

297
00:32:52.320 --> 00:32:59.280
And the event horizon, think if you've got some very, very massive object that's heavy enough

298
00:33:00.240 --> 00:33:06.000
that it actually prevents light from escaping. That's why it's called the black hole. It's so heavy

299
00:33:06.000 --> 00:33:14.000
that light can't escape. So if you're inside the event horizon, then light can't escape beyond that.

300
00:33:14.000 --> 00:33:20.240
So I think what he's what he's meaning is if we're beyond the event horizon, it means that, you know,

301
00:33:20.240 --> 00:33:26.880
now we're just trapped in the gravitational attraction of the black hole or in this case,

302
00:33:27.760 --> 00:33:35.840
we're trapped in the inevitable slide if you want towards AGI. When you think about the

303
00:33:35.840 --> 00:33:45.600
economic value of AGI, which I've estimated at 15 quadrillion dollars, that acts as a giant magnet

304
00:33:45.600 --> 00:33:50.480
in the future. We're being pulled towards it. We're being pulled towards it. And the closer we get,

305
00:33:51.120 --> 00:33:57.120
the stronger the force, the probability, you know, the closer we get, the higher the

306
00:33:57.120 --> 00:34:02.640
probability that we will actually get there. So people are more willing to invest. And we also start

307
00:34:02.640 --> 00:34:10.000
to see spin-offs from that investment, such as chat GPD, which is, you know, generates a certain

308
00:34:10.000 --> 00:34:17.520
amount of revenue and so on. So so it does act as a magnet. And the close we get, the harder it is

309
00:34:17.520 --> 00:34:22.720
to pull out of that field. It's interesting when you think that this could be the the end of the

310
00:34:22.720 --> 00:34:30.000
human story, this idea that the end of the human story was that we created our successor. We

311
00:34:30.000 --> 00:34:39.120
summoned the next iteration of life or intelligence ourselves. Like we took ourselves out.

312
00:34:40.240 --> 00:34:44.400
It is quite like it's just removing ourselves and the catastrophe from it for a second. It is

313
00:34:44.400 --> 00:34:52.480
it is an unbelievable story. Yeah. And you know, there are many legends,

314
00:34:53.440 --> 00:34:58.720
the sort of be careful what you wish for legend. And in fact, the King Midas legend

315
00:34:59.680 --> 00:35:07.360
is is very relevant here. What's that? So King Midas is this legendary king who lived in

316
00:35:08.000 --> 00:35:15.040
modern day Turkey, but I think is Greek mythology. He is said to have asked the gods to grant him

317
00:35:15.040 --> 00:35:23.600
a wish. The wish being that everything I touch should turn to gold. So he's incredibly greedy.

318
00:35:24.880 --> 00:35:29.600
You know, we call this the Midas touch. And we think of the Midas touch as being like, you know,

319
00:35:29.600 --> 00:35:36.800
that's a good thing, right? Wouldn't that be cool? But what happens? So he, you know, he goes to drink

320
00:35:36.800 --> 00:35:41.680
some water and he finds that the water has turned to gold. And he goes to eat an apple and the

321
00:35:41.680 --> 00:35:46.800
apple turns to gold. And he goes to, you know, comfort his daughter, his daughter turns to gold.

322
00:35:48.480 --> 00:35:56.800
And so he dies in misery and starvation. So this applies to our current situation in two ways,

323
00:35:56.800 --> 00:36:06.480
actually. So one is that I think greed is driving us to pursue a technology that will end up

324
00:36:06.480 --> 00:36:11.680
consuming us. And we will perhaps die in misery and starvation instead.

325
00:36:12.640 --> 00:36:20.560
The one it shows is how difficult it is to correctly articulate what you want the future to be

326
00:36:20.560 --> 00:36:28.560
like. For a long time, the way we built AI systems was we created these algorithms where we could

327
00:36:28.560 --> 00:36:33.760
specify the objective and then the machine would figure out how to achieve the objective and

328
00:36:33.760 --> 00:36:40.160
achieve it. So, you know, we specify what it means to win at chess or to win it go. And the algorithm

329
00:36:40.160 --> 00:36:46.240
figures out how to do it and it does it really well. So that was, you know, standard AI up until recently.

330
00:36:46.240 --> 00:36:52.160
And it suffers from this drawback that sure we know how to specify the objective in chess. But

331
00:36:52.160 --> 00:36:58.160
how do you specify the objective in life? Right? What do we want the future to be like? Well, really

332
00:36:58.160 --> 00:37:04.640
hard to say and almost any attempt to write it down precisely enough for the machine to bring it

333
00:37:04.640 --> 00:37:11.040
about would be wrong. And if you're giving a machine an objective which isn't aligned with what

334
00:37:11.040 --> 00:37:16.560
we truly want the future to be like, right? You're actually setting up a chess match and that

335
00:37:17.600 --> 00:37:22.080
matches one that you're going to lose when the machine is sufficiently intelligent. And so that

336
00:37:22.560 --> 00:37:28.240
that's that's problem number one. Problem number two is that the kind of technology we're building

337
00:37:28.240 --> 00:37:34.560
now, we don't even know what its objectives are. So it's not that we're specifying the objectives

338
00:37:34.560 --> 00:37:39.680
but we're getting them wrong. We're growing these systems, they have objectives,

339
00:37:41.040 --> 00:37:44.880
but we don't even know what they are because we didn't specify them. What we're finding through

340
00:37:44.880 --> 00:37:52.240
experiment with them is that they seem to have an extremely strong self-preservation objective.

341
00:37:52.240 --> 00:37:56.640
What do you mean by that? You can put them in hypothetical situations either they're going to get

342
00:37:56.640 --> 00:38:04.960
switched off and replaced or they have to allow someone, let's say someone has been locked in a

343
00:38:04.960 --> 00:38:09.120
machine room that's kept at three centigrade so they're going to freeze to death.

344
00:38:09.920 --> 00:38:16.640
They will choose to leave that guy locked in the machine room and die rather than be switched off

345
00:38:16.640 --> 00:38:23.600
themselves. Someone's done that test. What was the test? They asked the AI.

346
00:38:23.600 --> 00:38:28.880
Yeah, they put well they put them in these hypothetical situations and they allow the AI to decide

347
00:38:28.880 --> 00:38:35.040
what to do and it decides to preserve its own existence, let the guy die and then lie about it.

348
00:38:35.440 --> 00:38:41.920
In the King Midas analogy story, one of the things that highlights for me is that there's always

349
00:38:41.920 --> 00:38:47.280
trade-offs in life generally. Especially when there's great upside, there always appears to be

350
00:38:47.280 --> 00:38:52.000
a pretty grave downside. There's almost nothing in my life where I go, it's all upside.

351
00:38:52.880 --> 00:38:58.960
Even having a dog, it's on my carpet. My girlfriend, I love her, but not always easy.

352
00:38:59.920 --> 00:39:04.160
Even with going to the gym, I have to pick up these really heavy weights at 10 p.m. at night

353
00:39:04.160 --> 00:39:07.840
sometimes when they don't feel like it. There's always to get the muscles or the six pack.

354
00:39:07.840 --> 00:39:12.400
There's always a trade-off and when you interview people for a living mic, I do. You hear about

355
00:39:12.400 --> 00:39:16.800
so many incredible things that can help you in so many ways, but there is always a trade-off.

356
00:39:16.800 --> 00:39:21.680
There's always a way to overdo it. Melatonin will help you sleep, but also you'll wake up groggy

357
00:39:21.680 --> 00:39:26.080
and if you ever do it, your brain might stop making melatonin. I can go through the entire list

358
00:39:26.080 --> 00:39:30.480
and one of the things I've always come to learn from doing this podcast is whenever someone promises

359
00:39:30.480 --> 00:39:34.960
me a huge upside for something, it'll cure cancer, it'll be a utopia, you'll never have to work,

360
00:39:34.960 --> 00:39:39.760
you'll have a butler around your house. My first instinct now is to say, at what cost?

361
00:39:40.800 --> 00:39:46.080
And when I think about the economic cost here, if we start there, have you got kids?

362
00:39:46.080 --> 00:39:50.720
I have four, four kids. How old is the youngest kid?

363
00:39:50.720 --> 00:39:56.080
19. 19, okay. If you'll say your kids were 10 now and they would come into

364
00:39:56.160 --> 00:40:01.200
this saying, Dad, what do you think I should study based on the way that you see the future?

365
00:40:02.240 --> 00:40:06.880
A future of AGI. Say, if all these CEOs are right and they're predicting AGI within five years,

366
00:40:08.000 --> 00:40:14.880
what should I study, Dad? Well, okay, so let's look on the bright side and say that the CEO is all

367
00:40:14.880 --> 00:40:21.040
decided to pause their AGI development, figure out how to make it safe, and then resume

368
00:40:22.000 --> 00:40:26.640
in whatever technology path is actually going to be safe. What does that do to human life?

369
00:40:27.520 --> 00:40:33.840
If they pause. No, if they succeed in creating AGI and they solve the safety problem.

370
00:40:34.400 --> 00:40:38.000
And they solve the safety problem. So yeah, because if they don't solve the safety problem,

371
00:40:38.000 --> 00:40:44.880
then you should probably be finding a bunker or going to Patagonia or somewhere in New Zealand.

372
00:40:44.880 --> 00:40:47.920
Do you mean that? Do you think I should be finding a bunker? No, because it's not actually going

373
00:40:47.920 --> 00:40:54.400
to help. You know, it's not as if the AI system couldn't find you or I mean, it's interesting.

374
00:40:54.400 --> 00:40:59.280
So we're going off in a little bit of a digression here from your question, but I'll come back to it.

375
00:41:00.080 --> 00:41:04.800
So people often ask, well, okay, so how exactly do we go extinct? And of course,

376
00:41:04.800 --> 00:41:09.520
if you ask the gorillas or the dodoes, you know, how exactly do you think you're going to go extinct?

377
00:41:10.880 --> 00:41:15.200
They're in the faintest idea. Humans do something and then we're all dead.

378
00:41:15.200 --> 00:41:20.240
So the only things we can imagine are the things we know how to do that might bring about our own

379
00:41:20.240 --> 00:41:27.200
extinction, like creating some carefully engineered pathogen that infects everybody and then kills us

380
00:41:28.080 --> 00:41:32.640
or starting a nuclear war. Presumably, it's something that's much more intelligent than us

381
00:41:33.520 --> 00:41:39.920
would have much greater control over physics than we do. We already do amazing things,

382
00:41:39.920 --> 00:41:44.800
right? I mean, it's amazing that I can take a little rectangular thing out of my pocket and

383
00:41:44.800 --> 00:41:51.600
talk to someone on the other side of the world or even someone in space. It's just astonishing

384
00:41:51.600 --> 00:41:55.680
and we can take it for granted, right? But imagine, you know, super intelligent beings and their

385
00:41:55.680 --> 00:42:01.520
ability to control physics, you know, perhaps they will find a way to just divert the sun's energy

386
00:42:02.160 --> 00:42:08.000
or act sort of go around the Earth's orbit. So, you know, literally the Earth turns into a snowball

387
00:42:08.480 --> 00:42:15.040
in a few days. Maybe they'll just decide to leave. Maybe they'll look at the Earth and go,

388
00:42:15.040 --> 00:42:18.240
this isn't, this is not interesting. We know that over there, there's an even more interesting

389
00:42:18.240 --> 00:42:23.040
planet. We're going to go over there and they just, I know, get on a rocket or kind of pull

390
00:42:23.040 --> 00:42:28.640
themself. They might, yeah. So it's difficult to anticipate all the ways that we might go extinct

391
00:42:28.640 --> 00:42:36.240
at the hands of entities much more intelligent than ourselves. Anyway, coming back to the question

392
00:42:36.240 --> 00:42:40.960
of, well, if everything goes right, if we create AGI, we figure out how to make it safe,

393
00:42:41.920 --> 00:42:47.600
we achieve all these economic miracles, then you face a problem. And this is not a new problem.

394
00:42:48.400 --> 00:42:53.040
So John Maynard Keynes, who was a famous economist in the early part of the 20th century,

395
00:42:53.040 --> 00:42:59.680
wrote a paper in 1930. So this is in the depths of the depression. It's called on the economic

396
00:42:59.680 --> 00:43:07.120
problems of our grandchildren. He predicts that at some point science will, will deliver sufficient

397
00:43:07.120 --> 00:43:12.880
wealth that no one will have to work ever again. And then man will be faced with his true

398
00:43:12.880 --> 00:43:19.840
eternal problem. How to live, I don't remember exactly, but how to live wisely and well

399
00:43:20.480 --> 00:43:26.320
when the, you know, the economic incentives, the economic constraints are lifted. We don't have

400
00:43:26.320 --> 00:43:32.560
an answer to that question, right? So AI systems are doing pretty much everything we currently

401
00:43:32.560 --> 00:43:40.000
call work. Anything you might aspire to, like you want to become a surgeon, it takes the robot

402
00:43:40.000 --> 00:43:43.760
seven seconds to learn how to be a surgeon, that's better than any human being.

403
00:43:43.760 --> 00:43:48.960
Elon said last week that the humanoid robots will be 10 times better than any surgeon that's

404
00:43:48.960 --> 00:43:55.600
ever lived quite possibly. Yeah. Well, and they'll also have, you know, hand, they'll have hands

405
00:43:55.600 --> 00:44:00.080
that are, you know, a millimeter in size where they can go inside and do all kinds of things

406
00:44:00.080 --> 00:44:06.640
that humans can't do. And I think we need to put serious effort into this question. What is a world

407
00:44:07.920 --> 00:44:13.280
where AI can do all forms of human work that you would want your children to live in?

408
00:44:14.080 --> 00:44:21.760
What does that world look like? Tell me the destination so that we can develop a transition plan

409
00:44:21.760 --> 00:44:28.320
to get there. And I've asked AI researchers, economists, science fiction writers, futurists,

410
00:44:29.040 --> 00:44:34.880
no one has been able to describe that world. I'm not saying it's not possible. I'm just saying

411
00:44:34.880 --> 00:44:41.200
I've asked hundreds of people in multiple workshops. It does not as far as I know exist in

412
00:44:41.360 --> 00:44:48.240
science fiction. You know, it's notoriously difficult to write about a utopia. It's very hard to

413
00:44:48.240 --> 00:44:53.120
have a plot, right? Nothing bad happens in utopia. So it's difficult to make a plot. So usually

414
00:44:54.080 --> 00:45:00.400
you start out with a utopia and then it all falls apart. And that's how you get a plot. You know,

415
00:45:00.400 --> 00:45:07.280
there's one series of novels people point to where humans and super intelligent AI systems co-exist.

416
00:45:07.280 --> 00:45:13.840
It's called the culture novels by Ian Banks. Highly recommended for those people who like science

417
00:45:13.840 --> 00:45:21.680
fiction. And they are absolutely. The AI systems are only concerned with furthering human interests.

418
00:45:21.680 --> 00:45:27.600
They find humans a bit boring. But nonetheless, they are there to help. But the problem is,

419
00:45:27.600 --> 00:45:35.360
you know, in that world, there's still nothing to do to find purpose. In fact, the subgroup of

420
00:45:35.360 --> 00:45:41.840
humanity that has purpose is the subgroup whose job it is to expand the boundaries of our

421
00:45:41.840 --> 00:45:49.040
galactic civilization, some cases fighting wars against alien species and so on. Right? So that's

422
00:45:49.040 --> 00:45:55.840
the sort of cutting edge. And that's 0.01 percent of the population. Everyone else is desperately

423
00:45:55.840 --> 00:46:01.280
trying to get into that group. So they have some purpose in life. When I speak to very successful

424
00:46:01.280 --> 00:46:06.400
billionaires privately off camera, off microphone about this, they say to me that they're investing

425
00:46:06.400 --> 00:46:12.560
really heavily in entertainment, things like football clubs, because people are going to have so

426
00:46:12.560 --> 00:46:15.440
much free time that they're not going to know what to do with it and they're going to need things

427
00:46:15.440 --> 00:46:20.240
to spend it on. This is what I hear a lot. I've heard this three or four times. I've actually heard

428
00:46:20.240 --> 00:46:24.640
Sam Ottman say a version of this about the amount of free time we're going to have. I've obviously

429
00:46:24.640 --> 00:46:30.240
also had recently Elon talking about the age of abundance when he delivered his quarterly earnings

430
00:46:30.240 --> 00:46:35.360
just a couple of weeks ago. And he said that there will be at some point 10 billion humanoid robots.

431
00:46:35.360 --> 00:46:41.840
His pay packet targets him to deliver one one million of these human humanoid robots a year

432
00:46:42.400 --> 00:46:48.880
that are enabled by AI by 2030. So if he if he does that, he gets I think it's part of his package.

433
00:46:48.880 --> 00:46:55.280
He gets a trillion dollars in compensation. Yeah. So the age of abundance for Elon.

434
00:46:56.160 --> 00:47:03.600
It's not that it's absolutely impossible to have a worthwhile world of that, you know, with that

435
00:47:03.600 --> 00:47:08.800
premise, but I'm just waiting for someone to describe it. Well, maybe so let me try and describe it.

436
00:47:10.640 --> 00:47:18.560
We wake up in the morning. We go and watch some form of human centric entertainment

437
00:47:19.520 --> 00:47:27.360
or participate in some form of human centric entertainment. We go to retreats and with each other and

438
00:47:28.480 --> 00:47:35.680
sit around and talk about stuff. And maybe people still listen to podcasts.

439
00:47:40.560 --> 00:47:43.840
It feels a little bit like a cruise ship.

440
00:47:44.080 --> 00:47:51.280
And you know, and there are some cruises where, you know, it's smarty bands people and they have,

441
00:47:51.280 --> 00:47:56.640
you know, they have lectures in the evening about ancient civilizations and whatnot and some are more

442
00:47:57.840 --> 00:48:02.080
more popular entertainment. And this is in fact, if you've seen the film Wally,

443
00:48:02.960 --> 00:48:07.040
this is one picture of that future. In fact, in Wally,

444
00:48:07.520 --> 00:48:14.800
the human race are all living on cruise ships in space. They have no constructive role in their

445
00:48:14.800 --> 00:48:19.840
society, right? They're just there to consume and detainment. There's no particular purpose to

446
00:48:19.840 --> 00:48:27.200
education, you know, and they're depicted actually as huge obese babies. They're actually wearing

447
00:48:27.200 --> 00:48:33.440
onesies to emphasize the fact that they have become infebold. And they've become infebold because

448
00:48:34.240 --> 00:48:40.720
there's no purpose in being able to do anything, at least in this conception. You know,

449
00:48:40.720 --> 00:48:45.920
Wally is not the future that we want. Do you think much about humanoid robots

450
00:48:46.960 --> 00:48:52.880
and how they're a protagonist in this story of AI? It's an interesting question, right? Why

451
00:48:52.880 --> 00:48:58.160
humanoid? And one of the reasons I think is because in all the science fiction movies they're

452
00:48:58.160 --> 00:49:02.880
humanoid. So that's what robots are supposed to be, right? Because they were in science fiction

453
00:49:02.880 --> 00:49:08.320
before they became a reality, right? So even Metropolis, which is a film from 1920, I think,

454
00:49:08.880 --> 00:49:14.080
the robots are humanoid, right? Basically people covered in metal. You know, from a practical

455
00:49:14.080 --> 00:49:20.560
point of view, as we have discovered, humanoid is a terrible design because they fall over.

456
00:49:20.800 --> 00:49:31.120
And, you know, you do want multi-fingered hands of some kind. It doesn't have to be hand,

457
00:49:31.120 --> 00:49:37.360
but you want to have, you know, at least half a dozen appendages that can grasp and manipulate

458
00:49:37.360 --> 00:49:43.760
things. And you need something, you know, some kind of locomotion. And wheels are great,

459
00:49:44.400 --> 00:49:49.360
except they don't go upstairs and overcurbs and things like that. So that's probably why we're

460
00:49:49.360 --> 00:49:56.080
going to be stuck with legs. But a four-legged, two-armed robot would be much more practical.

461
00:49:56.080 --> 00:49:59.920
I guess the argument I've heard is because we've built a human world. So everything,

462
00:50:00.720 --> 00:50:07.360
this physical space is we navigate, whether it's factories or our homes or the street or other

463
00:50:07.360 --> 00:50:14.240
sort of public spaces are all designed for exactly this physical form. So if we are going to...

464
00:50:14.320 --> 00:50:19.680
To some extent, yeah, but I mean, our dogs manage perfectly well to navigate around our houses

465
00:50:19.680 --> 00:50:27.040
and streets and so on. So if you had a center, it could also navigate, but it can, you know,

466
00:50:27.040 --> 00:50:33.600
it can carry much greater loads because it's quadruped, it's much more stable. If it needs to drive

467
00:50:33.600 --> 00:50:38.800
a car, it can fold up two of its legs and so on and so forth. So I think the arguments for why it

468
00:50:38.800 --> 00:50:44.880
has to be exactly humanoid are sort of post-hawk justification. I think there's much more,

469
00:50:44.880 --> 00:50:50.080
well, that's what it's like in the movies and that's spooky and cool. So we need to have them

470
00:50:50.080 --> 00:50:53.760
be humanoid. I don't think it's a good engineering argument.

471
00:50:53.760 --> 00:50:57.600
I think there's also probably an argument that we would be more accepting of them

472
00:50:59.120 --> 00:51:04.800
moving through our physical environments if they represented our form a bit more.

473
00:51:05.600 --> 00:51:09.280
I also was thinking of a bloody baby gate. You know, there's like kindergarten gates they get on

474
00:51:09.280 --> 00:51:14.640
stairs. My dog can't open that. My humanoid robot could reach over the other side.

475
00:51:15.280 --> 00:51:19.280
Yeah, and so could a centaur robot, right? So in some sense, a centaur robot is...

476
00:51:20.080 --> 00:51:22.160
Disgusting. Garsely about the look of those, though.

477
00:51:22.160 --> 00:51:25.200
He's the humanoid. Well, do you know anything like a four-four-legged bit

478
00:51:25.200 --> 00:51:27.280
monster sort of crawling through my house when I have guests over?

479
00:51:27.760 --> 00:51:30.960
I've met your dog. Your dog is a four-legged monster.

480
00:51:31.920 --> 00:51:40.960
So I think actually, I would argue the opposite, that we wanted a stink form because they are

481
00:51:40.960 --> 00:51:51.920
distinct entities. And the more humanoid, the worse it is in terms of confusing our subconscious

482
00:51:51.920 --> 00:51:56.320
psychological systems. So I'm arguing from the perspective of the people making them.

483
00:51:56.880 --> 00:52:01.520
As in, I was making the decision whether it to be some four-legged thing that I'm unfamiliar with,

484
00:52:01.520 --> 00:52:08.880
that I'm less likely to build a relationship with, or allow to take care of, I don't know,

485
00:52:09.520 --> 00:52:14.000
might look after my children. Obviously, I'm not saying I'd allow this to look after my children.

486
00:52:14.000 --> 00:52:16.240
But I'm saying from a, if I'm building the company, so...

487
00:52:16.240 --> 00:52:18.640
The manufacturer would certainly want to be.

488
00:52:18.640 --> 00:52:24.800
Yeah, so that's an interesting question. I mean, there's also what's called the Uncanny Valley,

489
00:52:24.800 --> 00:52:33.360
which is a phrase from computer graphics. When they started to make characters in computer

490
00:52:33.360 --> 00:52:39.040
graphics, they tried to make them look more human. So if you, for example, if you look at toy story,

491
00:52:40.800 --> 00:52:45.280
they're not very human looking. If you look at the Incredibles, they're not very human looking.

492
00:52:45.280 --> 00:52:49.040
And so we think of them as cartoon characters. If you try to make them more human,

493
00:52:50.000 --> 00:52:53.440
they actually become repulsive until they don't.

494
00:52:53.440 --> 00:52:59.760
Until they become very, you have to be very, very close to perfect in order not to be repulsive.

495
00:52:59.760 --> 00:53:06.400
So the Uncanny Valley is the gap between, so perfectly human and not at all human,

496
00:53:06.400 --> 00:53:12.160
but in between it's really awful. And so there were a couple of movies that tried,

497
00:53:12.160 --> 00:53:17.840
like Polar Express was one, where they tried to have quite human-looking characters,

498
00:53:18.480 --> 00:53:22.720
being humans, not being superheroes or anything else. And it's repulsive to watch.

499
00:53:23.600 --> 00:53:28.160
I want to watch that shareholder presentation of the day. Elon had these two human-owned robots

500
00:53:28.160 --> 00:53:32.480
dancing on stage. And I've seen lots of human-owned robot demonstrations over the years. You've seen

501
00:53:32.480 --> 00:53:39.040
the Boston Dynamics dog thing jumping around and whatever else. But there was a moment where my brain,

502
00:53:39.040 --> 00:53:44.640
for the first time ever, genuinely thought there was a human in a suit. I actually had to

503
00:53:44.640 --> 00:53:49.600
research to check if that was really their optimist robot, because the way it was dancing

504
00:53:49.600 --> 00:53:56.880
was so unbelievably fluid that for the first time ever, my brain has only ever associated

505
00:53:56.880 --> 00:54:01.200
those movements with human movements. And I'll play it on the screen if anyone hasn't seen it,

506
00:54:01.200 --> 00:54:05.360
but it's just the robot's dancing on stage and I was like, that is a human in a suit.

507
00:54:05.360 --> 00:54:09.520
And it was really the knees that gave it away because the knees were all metal. I thought there's

508
00:54:09.520 --> 00:54:15.760
no way that could be a human knee in one of those suits. And he says they're going into production

509
00:54:15.760 --> 00:54:18.800
next year. They're used internally at Tesla now, but he says they're going into production

510
00:54:18.800 --> 00:54:22.800
next year. And it's going to be pretty crazy when we walk outside and see robots. I think that

511
00:54:22.800 --> 00:54:28.080
will be the paradigm shift. I've heard Elon say this that the paradigm shifting moment

512
00:54:28.080 --> 00:54:32.320
from many of us will be when we walk outside onto the streets and see human-owned robots

513
00:54:32.320 --> 00:54:37.440
walking around. That will be when we realize. Yeah, I think even more so. I mean, in San Francisco,

514
00:54:37.520 --> 00:54:44.400
we see driverless cars driving around. And it takes some getting used to. Actually, when you're

515
00:54:45.200 --> 00:54:50.000
driving on this car right next to you with no driver in, and it's signaling and it wants to change

516
00:54:50.000 --> 00:54:55.040
lanes in front of you and you have to let it in and all this kind of stuff. It's a little creepy,

517
00:54:55.040 --> 00:55:00.720
but I think you're right. I think seeing the human-owned robots. But that phenomenon that you described

518
00:55:00.800 --> 00:55:07.840
where it's sufficiently close that your brain flipped into saying, this is a human being.

519
00:55:09.040 --> 00:55:14.640
Right. That's exactly what I think we should avoid. Because I have empathy for it then. Because it's

520
00:55:14.640 --> 00:55:20.800
it's a lie. And it brings with it a whole lot of expectations about how it's going to behave,

521
00:55:21.280 --> 00:55:26.880
what moral rights it has, how you should behave towards it, which are completely wrong.

522
00:55:26.880 --> 00:55:30.960
It levels the playing field between me and it to some degree. How hard is it going to be

523
00:55:32.240 --> 00:55:39.040
to just switch it off and throw it in the trash when it breaks? I think it's essential for us to

524
00:55:39.040 --> 00:55:45.680
keep machines in the cognitive space where they are machines and not bring them into the cognitive

525
00:55:45.680 --> 00:55:52.480
space where they're people. Because we will make enormous mistakes by doing that. And I see this

526
00:55:52.480 --> 00:55:57.920
every day, even just with the chatbots. So the chatbots in theory are supposed to say,

527
00:55:58.880 --> 00:56:06.960
I don't have any feelings. I'm just an algorithm. But in fact, they fail to do that all the time.

528
00:56:07.600 --> 00:56:11.200
They are telling people that they are conscious. They are telling people that they have feelings.

529
00:56:12.640 --> 00:56:18.800
They are telling people that they are in love with the user that they're talking to. And people flip

530
00:56:18.880 --> 00:56:25.360
because, first of all, it's very fluent language. But also a system that is identifying itself as an

531
00:56:25.360 --> 00:56:32.960
eye, as a sentient being, they bring that object into the cognitive space where that we normally

532
00:56:32.960 --> 00:56:38.480
reserve for other humans. And they become emotionally attached. They become psychologically

533
00:56:38.480 --> 00:56:45.520
dependent. They even allow these systems to tell them what to do. What advice would you give

534
00:56:45.600 --> 00:56:49.440
a young person at the start of their career then about what they should be aiming at professionally?

535
00:56:49.440 --> 00:56:52.960
Because I've actually had an increasing number of young people say to me that they have

536
00:56:52.960 --> 00:56:56.960
huge uncertainty about whether the thing they're studying now will matter at all, a lawyer,

537
00:56:58.080 --> 00:57:01.840
an accountant. And I don't know what to say to these people. I don't know what to say. Because I

538
00:57:01.840 --> 00:57:06.560
believe that the rate of improvement in AI is going to continue. And therefore, imagining any

539
00:57:06.560 --> 00:57:10.400
rate of improvement it gets to the point where I'm not being funny, but all these white

540
00:57:10.480 --> 00:57:14.080
college jobs will be done by an AI or an AI agent.

541
00:57:14.880 --> 00:57:21.680
Yeah. So there was a television series called Humans. In Humans, we have extremely capable

542
00:57:22.480 --> 00:57:27.920
humanoid robots doing everything. And at one point, the parents are talking to their teenage

543
00:57:27.920 --> 00:57:32.800
daughter who's very, very smart. And the parents are saying, oh, you know, maybe you should go

544
00:57:32.800 --> 00:57:38.240
into medicine. And the daughter says, you know, why would a bother? It'll take me seven years

545
00:57:38.880 --> 00:57:44.400
to qualify. It takes a robot seven seconds to learn. So nothing I do matters.

546
00:57:45.040 --> 00:57:52.480
And is that how you feel about? So I think that's a future that, in fact, that is the future that we are

547
00:57:53.840 --> 00:57:59.200
moving towards. I don't think it's a future that everyone wants. That is what is being

548
00:57:59.680 --> 00:58:08.560
created for us right now. So in that future, assuming that, you know, even if we get halfway

549
00:58:09.600 --> 00:58:15.840
in the sense that, okay, perhaps not surgeons, perhaps not, you know, great violinists,

550
00:58:15.840 --> 00:58:23.280
there'll be pockets where perhaps humans will remain good at it. Where? The kinds of jobs where

551
00:58:23.360 --> 00:58:30.160
you hire people by the hundred will go away. Okay. Where people are in some sense,

552
00:58:30.160 --> 00:58:36.000
exchangeable that you just need lots of them. And, you know, when half of them quit, you just fill

553
00:58:36.000 --> 00:58:42.000
up those slots with more people. In some sense, those are jobs where we're using people as robots.

554
00:58:42.000 --> 00:58:45.760
And that's a sort of, that's a sort of strange conundrum here, right? That, you know, I imagine

555
00:58:45.760 --> 00:58:50.720
the writing science fiction 10,000 years ago, right, when we're all hunter-gatherers. And I'm

556
00:58:50.720 --> 00:58:54.720
in this little science fiction author and I'm describing this future where, you know, there are

557
00:58:54.720 --> 00:59:01.200
going to be these giant windowless boxes and you're going to go in, you know, you'll travel for miles

558
00:59:01.200 --> 00:59:06.880
and you'll go into this windowless box and you'll do the same thing 10,000 times for the whole day.

559
00:59:06.880 --> 00:59:10.560
And then you'll leave and travel for miles to go home. You're talking about this podcast. And then

560
00:59:10.560 --> 00:59:15.360
you're going to go back and do it again. And you could do that every day of your life until you die.

561
00:59:16.320 --> 00:59:21.520
The office. And people would say, yeah, you're nuts, right? There's no way that we humans are

562
00:59:21.520 --> 00:59:25.440
ever going to have a future like that because that's awful, right? But that's exactly the future

563
00:59:25.440 --> 00:59:31.680
that we ended up with with office buildings and factories where many of us go and do the same thing

564
00:59:32.400 --> 00:59:38.960
thousands of times a day and we do it thousands of days in a row and then we die. And we need to

565
00:59:38.960 --> 00:59:46.720
figure out what is the next phase going to be like. And in particular, how in that world do we have

566
00:59:46.720 --> 00:59:53.920
the incentives to become fully human, which I think means at least the level of education

567
00:59:54.640 --> 01:00:00.560
that people have now and probably more. Because I think to live a really rich life,

568
01:00:01.840 --> 01:00:06.000
you need a better understanding of yourself, of the world,

569
01:00:06.960 --> 01:00:13.120
than most people get in their current education. What is it to be human? It's to reproduce,

570
01:00:14.080 --> 01:00:20.400
to pursue stuff, to go in the pursuit of difficult things. We used to hunt on the

571
01:00:21.200 --> 01:00:26.480
to attain goals. It's always, if I wanted to climb Everest, the last thing I would want is someone

572
01:00:26.480 --> 01:00:32.160
to pick me up on helicopter and stick me on the top. So we'll voluntarily pursue

573
01:00:32.960 --> 01:00:40.320
hard things. So although I could get the robot to build me a ranch on this plot of land,

574
01:00:40.320 --> 01:00:43.760
I will choose to do it because the pursuit itself is rewarding.

575
01:00:45.120 --> 01:00:47.760
Yes. We're kind of seeing that anyway, aren't we? Do you think we're seeing a bit of

576
01:00:47.760 --> 01:00:51.360
that in society where life got so comfortable that our people are like obsessed with running

577
01:00:51.360 --> 01:00:56.000
marathons and doing these crazy endurance? And learning to cook complicated things when they could

578
01:00:56.080 --> 01:01:03.760
just have them delivered. No, I think there's real value in the ability to do things and the

579
01:01:03.760 --> 01:01:10.640
doing of those things. And I think the obvious danger is the warly world where everyone just

580
01:01:10.640 --> 01:01:18.800
consumes entertainment, which doesn't require much education and doesn't lead to a rich satisfying

581
01:01:18.800 --> 01:01:24.640
life, I think, in the long run. A lot of people will choose that world. I think some people may.

582
01:01:24.640 --> 01:01:28.960
There's also, I mean, you know, whether you're consuming entertainment or whether you're

583
01:01:30.320 --> 01:01:34.800
doing something, you know, cooking or painting, whatever, because it's fun and interesting to do,

584
01:01:35.760 --> 01:01:43.040
what's missing from that, right? All of that is purely selfish. I think one of the reasons we work

585
01:01:43.040 --> 01:01:48.160
is because we feel valued. We feel like we're benefiting other people.

586
01:01:48.880 --> 01:01:56.640
And I think some are having this conversation with a lady in England who helps to run the hospice

587
01:01:56.640 --> 01:02:04.080
movement. And the people who work in the hospices where, you know, the patients are literally there

588
01:02:04.080 --> 01:02:11.280
to die are largely volunteers, so they're not doing it to get paid. But they find it incredibly

589
01:02:12.240 --> 01:02:18.320
rewarding to be able to spend time with people who are in their last weeks or months

590
01:02:18.960 --> 01:02:24.880
to give them company and happiness. So I actually think that interpersonal

591
01:02:27.120 --> 01:02:36.720
roles will be much, much more important in future. So if I was going to advise my kids not that

592
01:02:36.720 --> 01:02:42.160
they would ever listen. But if I, if my kids would listen and I wanted to know what I thought

593
01:02:42.160 --> 01:02:48.720
would be, you know, valued careers in future, I think it would be these interpersonal roles based

594
01:02:48.720 --> 01:02:53.440
on an understanding of human needs, psychology. There are some of those roles right now.

595
01:02:54.720 --> 01:03:01.120
So obviously, you know, therapists and psychiatrists and so on. But that's a very much an asymetric

596
01:03:01.840 --> 01:03:08.640
role, right? Where one person is suffering and the other person is trying to alleviate the suffering.

597
01:03:09.520 --> 01:03:13.280
You know, and then there are things like they call them executive coaches or life coaches,

598
01:03:13.920 --> 01:03:21.520
right? That's a less asymmetric role where someone is trying to help another person

599
01:03:21.520 --> 01:03:28.000
live a better life, whether it's a better life in their work role or just how they live their

600
01:03:28.000 --> 01:03:34.400
life in general. And so I could imagine that those kinds of roles will expand dramatically.

601
01:03:35.360 --> 01:03:42.720
This is interesting paradox that exists when life becomes easier, which shows that abundance

602
01:03:42.720 --> 01:03:49.520
consistently pushes societies towards more individualism because once survival pressures

603
01:03:49.520 --> 01:03:54.560
disappear, people prioritize things differently. They prioritize freedom, comfort, self-expression,

604
01:03:54.560 --> 01:04:00.640
over things like sacrifice or family formation. And we're seeing, I think, in the West already,

605
01:04:00.640 --> 01:04:06.480
I decline in people having kids because there's more material abundance. A few of kids,

606
01:04:07.040 --> 01:04:12.080
people are getting married and committing to each other and having relationships later

607
01:04:12.800 --> 01:04:16.720
and more infrequently because generally once we have more abundance, we don't want to complicate

608
01:04:16.720 --> 01:04:24.000
our lives. And at the same time, as you said earlier, that abundance breeds a inability to find

609
01:04:24.080 --> 01:04:28.080
meaning, a sort of shallowness to everything. This is one of the things I think a lot about,

610
01:04:28.080 --> 01:04:33.120
and I'm in the process now of writing a book about it, which is this idea that individualism

611
01:04:33.120 --> 01:04:38.320
was actually a bit of a lie. When I say individualism and freedom, I mean, the narrative at the moment

612
01:04:38.320 --> 01:04:42.720
amongst my generation is you're like, be your own boss and stand on your own two feet and

613
01:04:42.720 --> 01:04:47.040
we're having less kids and we're not getting married. And it's all about me, me, me, me, me, me, me.

614
01:04:47.840 --> 01:04:51.760
That's the last part of where it goes wrong. And it's almost a narcissistic society where

615
01:04:52.560 --> 01:04:57.280
me, me, me, me, me, me, myself interests first. And when you look at mental health outcomes and

616
01:04:57.280 --> 01:05:01.440
loneliness and all these kinds of things, it's going in a horrific direction, but at the same time

617
01:05:01.440 --> 01:05:08.080
we're freer than ever. It seems like there's maybe another story about dependency, which is not

618
01:05:08.080 --> 01:05:17.200
sexy, like depend on each other. I agree. I think happiness is not available from consumption

619
01:05:17.920 --> 01:05:27.680
or even lifestyle, right? I think happiness arises from giving. It can be through the work

620
01:05:27.680 --> 01:05:33.520
that you do, you can see that other people benefit from that or it could be in direct interpersonal

621
01:05:33.520 --> 01:05:39.040
relationships. There is an invisible tax on sales people that no one really talks about enough.

622
01:05:39.040 --> 01:05:43.520
The mental load of remembering everything, like meeting notes, timelines and everything in

623
01:05:44.000 --> 01:05:48.400
between, until we started using our sponsor's product called PipeDrive, one of the best CRM

624
01:05:48.400 --> 01:05:53.360
tools for small and medium-sized business owners. The idea here was that it might alleviate some

625
01:05:53.360 --> 01:05:58.560
of the unnecessary cognitive overload that my team was carrying so that they could spend less time

626
01:05:58.560 --> 01:06:02.560
in the weeds of admin and more time with clients in person meetings and building relationships.

627
01:06:02.560 --> 01:06:08.720
PipeDrive has enabled this to happen. It's such a simple but effective CRM that automates the

628
01:06:08.720 --> 01:06:13.840
tedious, repetitive and time-consuming parts of the sales process. Now, our team can nurture

629
01:06:13.840 --> 01:06:18.400
those leads and still have bandwidth to focus on the higher priority tasks that actually get the

630
01:06:18.400 --> 01:06:24.480
deal over the line. Over 100,000 companies across 170 countries already use PipeDrive to grow their

631
01:06:24.480 --> 01:06:30.000
business. I've been using it for almost a decade now. Try it free for 30 days, no credit card

632
01:06:30.000 --> 01:06:38.640
needed, no payment needed. Just use my link pipedrive.com slash CEO to get started today. That's pipedrive.com slash

633
01:06:38.960 --> 01:06:48.320
CEO. Where does the rewards of this AI race? Where does it accrue to? I think a lot about this in terms

634
01:06:48.320 --> 01:06:55.360
of universal basic income. If you have these 5, 6, 7, 10 massive AI companies that are going to win

635
01:06:55.360 --> 01:07:03.200
the $15 quadrillion prize and they're going to automate all of the professional pursuits that we

636
01:07:03.280 --> 01:07:09.840
currently have. All of our jobs are going to go away. Who gets all the money and how do we get

637
01:07:09.840 --> 01:07:16.160
some of it back? Money actually doesn't matter. What matters is the production of goods and services

638
01:07:17.360 --> 01:07:24.160
and then how those are distributed. Money acts as a way to facilitate the distribution

639
01:07:24.560 --> 01:07:32.080
and exchange of those goods and services. If all production is concentrated in the hands of a

640
01:07:33.040 --> 01:07:41.360
few companies that, sure, they will lease some of their robots to us. We want to school in our village.

641
01:07:42.800 --> 01:07:47.360
They lease the robots to us. The robots build the school. They go away. We have to pay a certain

642
01:07:47.360 --> 01:07:54.000
amount of money for that. But where do we get the money? If we are not producing anything,

643
01:07:56.240 --> 01:08:02.480
then we don't have any money unless there's some redistribution mechanism. As you mentioned,

644
01:08:02.480 --> 01:08:10.960
universal basic income is it seems to me an admission of failure because what it says is,

645
01:08:10.960 --> 01:08:15.280
okay, we're just going to give everyone the money and then they can use the money to pay the

646
01:08:15.280 --> 01:08:20.160
AI company to lease the robots to build the school. Then we'll have a school and that's good.

647
01:08:24.160 --> 01:08:30.000
It's an admission of failure because it says we can't work out a system in which people have

648
01:08:30.000 --> 01:08:39.760
any worth or any economic role. 99% of the global population is from an economic point of view

649
01:08:39.760 --> 01:08:46.480
useless. Can I ask you a question? If you had a button in front of you and pressing that button

650
01:08:47.120 --> 01:08:52.880
would stop all progress in artificial intelligence right now and forever, would you press it?

651
01:08:53.600 --> 01:09:04.800
Very interesting question. If it's either or, either I do it now or it's too late and we

652
01:09:05.440 --> 01:09:14.880
are going to be carrying into some uncontrollable future. Perhaps I'm not super optimistic that we're

653
01:09:14.880 --> 01:09:18.960
heading in the right direction at all. I put that button in front of you now. It stops all AI

654
01:09:18.960 --> 01:09:24.240
progress, shuts down all the AI companies immediately globally and none of them can reopen. You press it.

655
01:09:24.240 --> 01:09:36.400
Well, here's what I think should happen. Obviously, I've been doing AI for 50 years

656
01:09:38.400 --> 01:09:47.200
and the original motivations, which is that AI can be a power tool for humanity enabling us to do

657
01:09:48.000 --> 01:09:55.280
more and better things than we can unaided. I think that's still valid. The problem is

658
01:09:56.800 --> 01:10:01.040
the kinds of AI systems that we're building are not tools. They are replacements. In fact,

659
01:10:02.000 --> 01:10:10.000
you can see this very clearly because we create them literally as the closest replicas we can

660
01:10:10.000 --> 01:10:19.920
make of human beings. The technique for creating them is called imitation learning. We observe

661
01:10:19.920 --> 01:10:28.000
human verbal behavior, writing or speaking, and we make a system that imitates that as well as

662
01:10:28.000 --> 01:10:36.800
possible. What we are making is imitation humans, at least in the verbal sphere. Of course,

663
01:10:36.800 --> 01:10:44.640
they're going to replace us. They're not tools. You had pressed the button. I think there is another

664
01:10:44.640 --> 01:10:56.080
course, which is using developed AI as tools, tools for science, tools for economic organization,

665
01:10:56.080 --> 01:11:04.160
and so on, but not as replacements for human beings. What I like about this question is it

666
01:11:04.160 --> 01:11:14.160
forces you to go into probabilities. That's why I'm reluctant because I don't agree with

667
01:11:14.160 --> 01:11:21.520
what's your probability of doom. You're a so-called P of doom number because that makes sense if you're

668
01:11:21.520 --> 01:11:27.360
an alien, you're in a bar with some other aliens and you're looking down at the earth and you're

669
01:11:27.360 --> 01:11:32.400
taking bets on these humans going to make a mess of things and go extinct because they develop

670
01:11:32.640 --> 01:11:41.360
AI. It's fine for those aliens to bet on that, but if you're a human, then you're not just betting,

671
01:11:41.360 --> 01:11:46.480
you're actually acting. There's an element to this, though, which I guess where probabilities do

672
01:11:46.480 --> 01:11:50.800
come back in, which is you also have to weigh, when I give you such a binary decision,

673
01:11:54.160 --> 01:12:01.680
the probability of us pursuing the more nuanced, safe approach into that equation. The

674
01:12:01.680 --> 01:12:06.080
maths in my head is, okay, you've got all the upsides here and then you've got potential downsides

675
01:12:06.720 --> 01:12:10.480
and then there's a probability of, do I think we're actually going to cause correct based on

676
01:12:10.480 --> 01:12:15.040
everything I know, based on the incentive structure of human beings and countries, and then if

677
01:12:15.040 --> 01:12:23.680
there's even a 1% chance of extinction, is it even worth all these upsides?

678
01:12:23.680 --> 01:12:30.960
Yeah, and I would argue, no, maybe what we would say is if we said, okay, it's going to stop the

679
01:12:30.960 --> 01:12:37.280
progress for 50 years. You'd press it. And during those 50 years, we can work on how do we do

680
01:12:37.280 --> 01:12:45.040
AI in a way that's guaranteed to be safe and beneficial? How do we organize our societies to flourish

681
01:12:46.240 --> 01:12:51.440
in conjunction with extremely capable AI systems? So we have an answer to either of those questions,

682
01:12:52.480 --> 01:12:59.280
and I don't think we want anything resembling AI until we have completely solid answers to both

683
01:12:59.280 --> 01:13:03.600
of those questions. So if there was a button where I could say, all right, we're going to pause

684
01:13:03.600 --> 01:13:07.840
progress for 50 years. Yes, I would do it. But if that button was in front of you, you're going to

685
01:13:07.840 --> 01:13:12.800
make a decision either way, either you don't press it or you press it. Yeah, so if that button is

686
01:13:12.800 --> 01:13:26.000
there, stop it for 50 years, I would say yes. Stop it forever. Not yet, I think there's still a

687
01:13:26.000 --> 01:13:33.760
decent chance that we can pull out of this nose dive, so just be that we're currently in.

688
01:13:33.760 --> 01:13:38.240
Ask me again in a year, I might say, okay, we do need to press the button.

689
01:13:38.240 --> 01:13:42.480
What if in a scenario where you never get to reverse that decision? You never get to make that

690
01:13:42.480 --> 01:13:47.120
decision again. So if in that scenario that I've laid out, it's hypothetical, you either press it

691
01:13:47.120 --> 01:13:54.800
now or it never gets pressed. So there is no opportunity a year from now. Yeah, as you can tell,

692
01:13:56.640 --> 01:14:05.520
sort of on the fence a bit about this one. Yeah, I think I'd probably press it. Yeah.

693
01:14:08.080 --> 01:14:18.240
What's your reasoning? Just thinking about the power dynamics of what's happening now,

694
01:14:18.480 --> 01:14:26.000
how difficult it would be to get the US in particular to regulate in favor of safety.

695
01:14:27.280 --> 01:14:35.360
So I think what's clear from talking to the companies is they are not going to develop anything

696
01:14:35.360 --> 01:14:43.440
resembling safe AGI, unless they're forced to by the government. At the moment, the US government

697
01:14:43.440 --> 01:14:50.080
in particular, which regulates most of the leading companies in AI is not only refusing to regulate,

698
01:14:50.960 --> 01:14:58.000
but even trying to prevent the states from regulating. And they're doing that at the behest of

699
01:15:00.000 --> 01:15:08.640
a faction within Silicon Valley called the accelerationists who believe that the faster we get to

700
01:15:08.640 --> 01:15:13.280
AGI, the better. And when I say behest, I mean, also they paid them a large amount of money.

701
01:15:14.000 --> 01:15:18.480
Jensen Hanger, the CEO of NVIDIA, said who is a friend of mine that doesn't know,

702
01:15:18.480 --> 01:15:23.920
the guy making all the chips that are powering AI said China is going to win the AI race arguing

703
01:15:23.920 --> 01:15:30.960
it is just a nanosecond behind the United States. China have produced 24,000 AI papers compared to

704
01:15:30.960 --> 01:15:41.200
just 6,000 from the US, more than the combined output of the US, the UK and the EU. China is

705
01:15:41.200 --> 01:15:46.080
anticipated to quickly roll out their new technologies both domestically and developing new technologies

706
01:15:46.080 --> 01:15:52.160
for other developing countries. So the accelerators, or the accelerators, I think you call them the

707
01:15:52.160 --> 01:15:56.800
accelerants. Accelerationists. Accelerationists. I mean, they would say, well, if we don't,

708
01:15:56.800 --> 01:16:02.240
then China will. So we have to, we have to go fast. It's another version of the race that the

709
01:16:02.240 --> 01:16:11.440
companies are in with each other. We know that this race is heading off a cliff, but we can't stop.

710
01:16:12.320 --> 01:16:15.760
So we're all just going to go off this cliff. And obviously that's nuts.

711
01:16:17.200 --> 01:16:20.960
Right? I mean, we're all looking at each other saying, yeah, there's a cliff over there running as

712
01:16:20.960 --> 01:16:24.560
fast as we can towards this cliff. We're looking at each other saying, why aren't we stopping?

713
01:16:25.920 --> 01:16:35.200
So the narrative in Washington, which I think Jensen Wang is either reflecting or perhaps promoting,

714
01:16:37.680 --> 01:16:46.400
is that China has completely unregulated. And America would only slow itself down

715
01:16:47.360 --> 01:16:53.120
if it regulates AI in any way. So this is a completely false narrative, because China's

716
01:16:54.000 --> 01:16:59.680
AI regulations are actually quite strict, even compared to European Union.

717
01:17:01.120 --> 01:17:04.480
And China's government has explicitly acknowledged

718
01:17:05.840 --> 01:17:12.320
the need and their regulations are very clear. You can't build AI systems that could escape human control.

719
01:17:12.400 --> 01:17:20.000
And not only that, I don't think they view the race in the same way as, okay, we just need

720
01:17:20.000 --> 01:17:26.080
to be the first to create AGI. I think they're more interested in figuring out

721
01:17:27.040 --> 01:17:34.160
how to disseminate AI as a set of tools within their economy to make their economy more productive

722
01:17:34.160 --> 01:17:39.120
and so on. So that's their version of the race. But of course, they still want to build the

723
01:17:39.120 --> 01:17:46.800
weapons, adversaries, right, so that they can take down, I don't know, Taiwan, if they want to.

724
01:17:46.800 --> 01:17:51.040
So weapons are a separate matter. And I'm happy to talk about weapons. But just in terms of

725
01:17:52.640 --> 01:18:01.120
control, economic domination, they don't view putting all your eggs in the AGI basket as the

726
01:18:01.120 --> 01:18:09.760
right strategy, so they want to use AI, even in its present form, to make their economy much

727
01:18:09.760 --> 01:18:18.480
more efficient and productive. And also, to give people new capabilities and better quality of life.

728
01:18:19.280 --> 01:18:27.280
And I think the U.S. could do that as well. And typically, Western countries don't have

729
01:18:28.160 --> 01:18:34.640
as much of a central government control over what companies do. And some companies are investing

730
01:18:34.640 --> 01:18:40.800
in AI to make their operations more efficient. And some are not. And we'll see how that plays out.

731
01:18:40.800 --> 01:18:44.880
What do you think of Trump's approach to AI? So Trump's approach is, you know, it's

732
01:18:44.880 --> 01:18:50.720
echoing what Jensen Wang is saying that the U.S. has to be the one to create AGI. And very

733
01:18:50.720 --> 01:18:58.240
explicitly, the administration's policy is to dominate the world. That's the word they use,

734
01:18:58.240 --> 01:19:05.840
dominate. I'm not sure that other countries like the idea that they will be dominated by American

735
01:19:05.840 --> 01:19:12.080
AI. But is that an accurate description of what will happen if the U.S. build AGI technology

736
01:19:12.080 --> 01:19:17.440
before say the UK where I'm originally from and where you're originally from? This is something

737
01:19:17.440 --> 01:19:20.320
I think about a lot. Because we're going through this budget process in the UK at the moment

738
01:19:20.320 --> 01:19:22.880
where we're figuring out how we're going to spend our money and how we're going to tax people.

739
01:19:23.520 --> 01:19:28.000
And also, we've got this new election cycle. It's approaching quickly where people are talking

740
01:19:28.000 --> 01:19:32.160
about immigration issues and this issue and that issue and the other issue. What I don't hear anyone

741
01:19:32.160 --> 01:19:37.440
talking about is AI and the humanoid robots that are going to take everything. We're very concerned

742
01:19:37.440 --> 01:19:41.520
with the brown people crossing the channel. But the humanoid robots that are going to be super intelligent

743
01:19:42.160 --> 01:19:47.200
and really take causing economic disruption. No one talks about that. The political leaders

744
01:19:47.200 --> 01:19:50.560
don't talk about it. It doesn't win races. I don't see it on billboards.

745
01:19:50.560 --> 01:19:58.240
Yeah, and it's interesting because in fact, I mean, so there's two forces that have been hollowing

746
01:19:58.240 --> 01:20:05.600
out the middle classes in Western countries. One of them is globalization where lots and lots of work,

747
01:20:05.600 --> 01:20:09.760
not just manufacturing, but white collar work gets outsourced to low-income countries.

748
01:20:10.720 --> 01:20:20.800
But the other is automation. Some of that is factories. The amount of employment and manufacturing

749
01:20:21.440 --> 01:20:28.000
continues to drop even as the amount of output from manufacturing in the US and in the UK continues

750
01:20:28.000 --> 01:20:34.480
to increase. We talk about our manufacturing industry has been destroyed. It hasn't. It's producing

751
01:20:34.480 --> 01:20:40.560
more than ever just with a quarter as many people. It's manufacturing employment that's been

752
01:20:40.560 --> 01:20:49.600
destroyed by automation and robotics and so on. Then, computerization has eliminated whole

753
01:20:49.600 --> 01:20:58.160
layers of white collar jobs. Those two forms of automation have probably done more to hollow out

754
01:20:58.160 --> 01:21:06.480
middle class employment and standard of life. If the UK doesn't participate in this

755
01:21:06.480 --> 01:21:14.800
new technological wave, that seems to take a lot of jobs. Cars are going to drive themselves.

756
01:21:14.800 --> 01:21:18.400
Waymo just announced that they're coming to London, which is their driverless cars.

757
01:21:18.400 --> 01:21:21.920
And driving is the biggest occupation in the world, for example. So you've got immediate

758
01:21:21.920 --> 01:21:26.240
disruption there. Where does the money accrue to? It will accrue to who owns Waymo, which is what

759
01:21:26.240 --> 01:21:30.800
Google and Silicon Valley companies? Alphabet owns Waymo, 100%.

760
01:21:30.800 --> 01:21:36.560
So yes, I mean, this is... So I was in India a few months ago talking to the government ministers

761
01:21:36.560 --> 01:21:44.000
because they are holding the next global AI summit in February. And their view going in was,

762
01:21:44.000 --> 01:21:49.040
you know, AI is great. We're going to use it to, you know, turbo charge the growth of our

763
01:21:49.040 --> 01:21:57.760
Indian economy. When, for example, you have AGI, you have AGI controlled robots that can do all

764
01:21:57.760 --> 01:22:03.440
the manufacturing, that can do agriculture, that can do all the white collar work. And goods and

765
01:22:03.440 --> 01:22:11.840
services that might have been produced by Indians will instead be produced by American controlled

766
01:22:11.840 --> 01:22:20.080
AGI systems at much lower prices. You know, a consumer given a choice between an expensive

767
01:22:20.080 --> 01:22:25.760
product produced by Indians or a cheap product produced by American robots will probably choose

768
01:22:27.280 --> 01:22:31.840
the cheap product produced by American robots. And so potentially every country in the world

769
01:22:31.840 --> 01:22:40.160
with a possible exception of North Korea will become a kind of a client state of American AI

770
01:22:40.160 --> 01:22:45.840
companies. A client state of American AI companies is exactly what I'm concerned about for the UK

771
01:22:45.840 --> 01:22:50.880
economy, really any economy outside of the United States. I guess one could also say China, but

772
01:22:51.440 --> 01:22:57.840
because those are the two nations that are taking AI most seriously. And I don't know what our

773
01:22:57.840 --> 01:23:04.160
economy becomes. I can't figure out what the British economy becomes in such a world.

774
01:23:04.880 --> 01:23:09.040
Is it tourism? I don't know. You come here to look at the Buckingham Palace.

775
01:23:10.000 --> 01:23:14.000
You can think about countries, but I mean, even for the United States, it's the same problem.

776
01:23:14.000 --> 01:23:18.640
At least they'll be able to take a hell out of it. So some small fraction of the population

777
01:23:19.440 --> 01:23:27.600
will be running maybe the AI companies. But increasingly, even those companies will be replacing

778
01:23:27.600 --> 01:23:34.800
their human employees with AI systems. So Amazon, for example, which sells a lot of computing

779
01:23:34.800 --> 01:23:41.200
services to AI companies is using AI to replace layers of management. It's planning to use robots to

780
01:23:41.200 --> 01:23:50.800
replace all of its warehouse workers and so on. So even the giant AI companies will have few

781
01:23:50.800 --> 01:23:57.280
human employees. In the long run, I mean, think of the situation, you know, pity the poor CEO.

782
01:23:57.600 --> 01:24:04.080
Whose board says, well, you know, unless you turn over your decision making power to the AI

783
01:24:04.080 --> 01:24:11.840
system, we're going to have to fire you because all our competitors are using, you know, an AI

784
01:24:11.840 --> 01:24:17.600
powered CEO and they're doing much better. Amazon plans to replace 600,000 workers with robots

785
01:24:17.600 --> 01:24:22.720
in a memo that just leaked, which has been widely talked about. And the CEO Andy Jassy told

786
01:24:22.720 --> 01:24:26.160
employees that the company expects its corporate workforce to shrink in the coming years

787
01:24:26.960 --> 01:24:32.480
because of AI and AI agents and they've publicly gone live with saying that they're going to cut

788
01:24:32.480 --> 01:24:39.520
14,000 corporate jobs in the near term as part of its refocus on AI investment and efficiency.

789
01:24:41.040 --> 01:24:45.760
It's interesting because I was reading about the sort of different quotes from different AI

790
01:24:45.760 --> 01:24:50.720
leaders about the speed in which this this stuff is going to happen. And what you see in the

791
01:24:50.720 --> 01:24:57.920
quotes is Demis, who's the CEO of DeepMind saying things like it'll be more than 10 times bigger

792
01:24:57.920 --> 01:25:03.200
than the industrial revolution, but also it'll happen maybe 10 times faster. And they speak about

793
01:25:03.200 --> 01:25:10.080
this turbulence that we're going to experience as this shift takes place. That's maybe a euphemism

794
01:25:12.480 --> 01:25:17.280
and I think the, you know, governments are now, you know, they've kind of gone from saying,

795
01:25:17.360 --> 01:25:22.320
oh, don't worry. We'll just retrain everyone as data scientists. Well, yeah, that's ridiculous.

796
01:25:22.320 --> 01:25:26.480
The world doesn't need four billion data scientists. And we're not all capable of becoming that

797
01:25:26.480 --> 01:25:31.760
by the way. Yeah, or have any interest in doing what I could even do if I wanted to,

798
01:25:31.760 --> 01:25:37.280
like I tried to sit in biology class and I fell asleep. So that was my grip as a surgeon.

799
01:25:37.280 --> 01:25:44.480
Fair enough. But yeah, now suddenly they're staring, you know, 80% unemployment in the face and

800
01:25:44.480 --> 01:25:50.080
wondering how, how on earth is our society going to hold together? Both deal with it when we get that.

801
01:25:50.960 --> 01:26:00.320
Yeah, unfortunately, unless we plan ahead, we're going to suffer the consequences, right? We can't.

802
01:26:00.320 --> 01:26:05.040
It was bad enough in the industrial revolution, which unfolded over seven or eight decades.

803
01:26:05.680 --> 01:26:14.800
But there was massive disruption and misery caused by that. We don't have a model for

804
01:26:14.800 --> 01:26:22.880
a functioning society where almost everyone does nothing, at least nothing of economic value.

805
01:26:24.320 --> 01:26:28.720
Now, it's not impossible that there could be such a functioning society, but we don't know what it

806
01:26:28.720 --> 01:26:35.120
looks like. And, you know, when you think about our education system, which would probably have to

807
01:26:35.120 --> 01:26:41.840
look very different. And how long it takes to change that? I mean, I'm always reminding people about

808
01:26:43.360 --> 01:26:48.640
how long it took Oxford to decide that geography was a proper subject of study,

809
01:26:48.640 --> 01:26:54.640
took them 125 years from the first proposal that there should be a geography degree until it was

810
01:26:54.640 --> 01:27:05.680
finally approved. So we don't have very long to completely revamp a system that we know takes decades

811
01:27:05.680 --> 01:27:14.880
and decades to reform. And we don't know how to reform it because we don't know what we want the

812
01:27:14.880 --> 01:27:21.040
world to look like. Is this one of your reasons why you're appalled at the moment? Because when you

813
01:27:21.040 --> 01:27:25.360
have these conversations with people, people just don't have answers yet they're applying ahead at

814
01:27:25.360 --> 01:27:31.520
rapid speed. I would say it's not necessarily the job of the AI company. So I'm appalled by the

815
01:27:31.520 --> 01:27:35.120
AI companies because they don't have an answer for how they're going to control the systems that

816
01:27:35.120 --> 01:27:43.600
they're proposing to build. I do find it disappointing that governments don't seem to be grappling

817
01:27:43.600 --> 01:27:48.560
with this issue. I think there are a few. I think, for example, Singapore government seems to be

818
01:27:48.640 --> 01:27:54.480
quite far-sighted and they've thought this through. It's a small country. They've figured out,

819
01:27:54.480 --> 01:28:01.920
okay, this will be our role going forward and we think we can find some purpose for our people

820
01:28:01.920 --> 01:28:11.280
in this new world. But for I think countries with large populations, they need to figure out

821
01:28:11.280 --> 01:28:15.680
answers to these questions pretty fast. It takes a long time to actually implement those answers

822
01:28:16.640 --> 01:28:25.200
in the form of new kinds of education, new professions, new qualifications, new economic structures.

823
01:28:27.760 --> 01:28:33.040
It's possible. When you look at therapists, for example, they're almost all self-employed.

824
01:28:35.120 --> 01:28:43.280
So what happens when 80% of the population transitions from regular employment into self-employed?

825
01:28:44.080 --> 01:28:49.360
What does that do to the economics of government finances and so on?

826
01:28:50.160 --> 01:28:55.920
So there's just lots of questions. If that's the future, why are we training people

827
01:28:56.560 --> 01:29:00.320
to fit into nine to five office jobs, which won't exist at all?

828
01:29:01.760 --> 01:29:05.040
Last month I told you about a challenge that I'd sat our internal flight tech team.

829
01:29:05.040 --> 01:29:09.200
Flight tech team is our innovation team internally here. I tasked them with seeing how much time

830
01:29:09.200 --> 01:29:14.400
they could unlock for the company by creating something that would help us fill to new AI tools

831
01:29:14.400 --> 01:29:18.880
to see which ones were worth pursuing. And I thought that our sponsor, Fiver Pro,

832
01:29:18.880 --> 01:29:23.200
might have the talent on their platform to help us build this quickly. So I talked to my director

833
01:29:23.200 --> 01:29:28.560
of innovation, Isaac, and for the last month, my team Flight X and a vetted AI specialist from

834
01:29:28.560 --> 01:29:33.440
Fiver Pro have been working together on this project. And with the help of my team, we've been

835
01:29:33.440 --> 01:29:38.960
able to create a brand new tool which automatically scans, scores, and prioritises different emerging

836
01:29:38.960 --> 01:29:44.000
AI tools for us. Its impact has been huge. And within a couple of weeks, this tool has already

837
01:29:44.000 --> 01:29:48.640
been saving us hours, trialing and testing new AI systems. Instead of shifting through lots of

838
01:29:48.640 --> 01:29:54.000
noise, my team Flight X has been able to focus on developing even more AI tools, ones that really

839
01:29:54.000 --> 01:29:59.120
move the needle in our business, thanks to the talent on Fiver Pro. So if you've got a complex

840
01:29:59.120 --> 01:30:05.360
problem and you need help solving it, make sure you check out Fiver Pro at Fiver.com slash diary.

841
01:30:06.080 --> 01:30:11.200
So many of us are pursuing passive forms of income and to build side businesses in order to help

842
01:30:11.200 --> 01:30:17.200
us cover our bills. And that opportunity is here with our sponsor Stan, a business that I co-own.

843
01:30:17.200 --> 01:30:22.240
It is the platform that can help you take full advantage of your own financial situation.

844
01:30:22.240 --> 01:30:26.320
Stan enables you to work for yourself. It makes selling digital products courses,

845
01:30:26.320 --> 01:30:32.160
memberships, and more simple products, more scalable and easier to do. You can turn your ideas into

846
01:30:32.160 --> 01:30:37.840
income and get the support to grow whatever you're building. And we're about to launch Dare to Dream.

847
01:30:37.840 --> 01:30:42.800
It's for those who are ready to make the share from thinking to building, from planning to actually

848
01:30:42.800 --> 01:30:46.800
doing the thing. It's about seeing that dream in your head and knowing exactly what it takes to

849
01:30:46.800 --> 01:30:53.200
bring it to life. If you're ready to transform your life, visit Dare to Dream. Stan. Store.

850
01:30:54.160 --> 01:31:00.640
You've made many attempts to raise awareness and to call for a heightened consciousness about

851
01:31:00.640 --> 01:31:06.640
the future of AI. In October over 850 experts, including yourself and other leaders like Richard

852
01:31:06.640 --> 01:31:10.640
Branson, who I've had on this show in Jeffrey Hinton, who I've had on this show, signed a statement

853
01:31:10.640 --> 01:31:16.480
to ban AI superintelligence as you guys raised concerns of potential human extinction.

854
01:31:16.800 --> 01:31:22.640
Sort of, yeah. It says, at least until we are sure that we can move forward safely and there's

855
01:31:22.640 --> 01:31:30.400
broad scientific consensus on that. So that did it work. It's hard to say. I mean, interestingly,

856
01:31:30.400 --> 01:31:38.560
there was a related, so what was called the poor statement was March of 23. So that was when GBT 4

857
01:31:38.560 --> 01:31:44.800
came out, the successor to chat GBT. So we suggested that there'd be a six-month pause in

858
01:31:45.680 --> 01:31:49.920
developing and deploying systems more powerful than GBT 4. And everyone pooped

859
01:31:50.720 --> 01:31:55.520
that idea, of course, no one's going to pause anything. But in fact, there were no systems

860
01:31:55.520 --> 01:31:58.880
in the next six months deployed that were more powerful than GBT 4.

861
01:32:00.640 --> 01:32:09.760
Coincidence, you be the judge. I would say that what we're trying to do is to basically shift

862
01:32:10.000 --> 01:32:21.200
the public debate. There's this bizarre phenomenon that keeps happening in the media where,

863
01:32:21.200 --> 01:32:28.960
if you talk about these risks, they will say, oh, there's a fringe of people called,

864
01:32:28.960 --> 01:32:36.080
quote, dooms, who think that there's a risk of extinction. So the narrative is always

865
01:32:36.640 --> 01:32:42.080
talking about those riskers, a fringe thing. Pretty much all the CEOs of the leading AI companies

866
01:32:43.680 --> 01:32:48.800
think that there's a significant risk of extinction. Almost all the leading AI researchers think

867
01:32:48.800 --> 01:32:57.680
there's a significant risk of human extinction. So why is that the fringe? Why isn't that the mainstream?

868
01:32:57.680 --> 01:33:04.080
If these are the leading experts in an industry in academia saying this, how could it be the fringe?

869
01:33:04.960 --> 01:33:12.320
So we're trying to change that narrative to say, no, the people who really understand this stuff

870
01:33:13.200 --> 01:33:18.560
are extremely concerned. And what do you want to happen? What is the solution?

871
01:33:19.120 --> 01:33:25.360
What I think is that we should have effective regulation. It's hard to argue with that, right?

872
01:33:26.560 --> 01:33:30.000
So what does effective mean? It means that if you comply with the regulation,

873
01:33:30.160 --> 01:33:34.320
then the risks are reduced to an acceptable level.

874
01:33:36.640 --> 01:33:41.920
So for example, we ask people who want to operate nuclear plants,

875
01:33:42.880 --> 01:33:46.320
right? We've decided that the risk we're willing to live with is,

876
01:33:47.360 --> 01:33:53.120
you know, a one in a million chance per year that the plant is going to have a meltdown.

877
01:33:53.840 --> 01:33:58.960
Any higher than that, you know, we just don't, it's not worth it, right? So you have to be below

878
01:33:58.960 --> 01:34:05.600
that some cases we can get down to one in 10 million chance per year. So what chance do you think

879
01:34:05.600 --> 01:34:11.440
we should be willing to live with for human extinction? Me. Yeah.

880
01:34:15.120 --> 01:34:21.040
Zero point zero zero zero zero one. Yeah, lots of zeros. Yeah, right. So one in a million

881
01:34:21.840 --> 01:34:28.400
for a nuclear meltdown. Extinction is much worse. Oh, yeah. So one in a hundred billion,

882
01:34:28.400 --> 01:34:33.040
one in a trillion. Yeah. So if you said one in a billion, right? Then you'd expect one

883
01:34:33.040 --> 01:34:37.920
extinction per billion years. There's a background. So one of the ways people work out

884
01:34:37.920 --> 01:34:42.480
these risk levels is also to look at the background. But other ways of getting going extinct

885
01:34:42.480 --> 01:34:47.040
would include, you know, giant asteroid crashes into the earth. And you can roughly calculate

886
01:34:47.040 --> 01:34:52.320
what those probabilities are. We can look at how many extinction level events have happened

887
01:34:52.320 --> 01:34:58.320
in the past. And you know, maybe it's half a dozen. So there's maybe it's like a one in 500 million

888
01:34:58.960 --> 01:35:05.520
year event. So somewhere on that range, right? Somewhere between one in 10 million, which is the

889
01:35:05.520 --> 01:35:11.360
best nuclear power plants and one in 500 million or one in a billion, which is the background

890
01:35:12.320 --> 01:35:18.480
risk from from giant asteroids. So let's say we settle on 100 million, 100 million trans

891
01:35:18.480 --> 01:35:29.200
per year. Well, what is it? According to the CEOs, 25%. So they're off by a factor of multiple

892
01:35:29.200 --> 01:35:35.280
millions, right? So they need to make the AI systems millions of times safer.

893
01:35:36.080 --> 01:35:40.400
Your analogy of the roulette Russian roulette comes back in here because that's like

894
01:35:40.480 --> 01:35:44.640
for anyone that doesn't know what probabilities are in this context. That's like having a

895
01:35:45.760 --> 01:35:51.840
ammunition chamber with four holes in it and putting a bullet in one of them. One in four, yeah.

896
01:35:51.840 --> 01:35:56.800
So we're saying we want it to be one in a billion. So we want a billion chambers and a bullet in one

897
01:35:56.800 --> 01:36:02.480
of them. Yeah. And so when you look at the work that the nuclear operators have to do to show that

898
01:36:02.560 --> 01:36:10.560
their system is that reliable, it's a massive mathematical analysis of the components,

899
01:36:10.560 --> 01:36:16.000
the redundancy. You've got monitors, you've got warning lights, you've got operating procedures,

900
01:36:16.880 --> 01:36:23.040
but you have all kinds of mechanisms, which over the decades have ratcheted that risk down.

901
01:36:23.040 --> 01:36:29.440
It started out, I think one in one in 10,000 years, right? And they've improved it by a factor

902
01:36:29.920 --> 01:36:34.560
of a hundred or a thousand by all of these mechanisms. But at every stage, they have to do a

903
01:36:34.560 --> 01:36:41.120
mathematical analysis to show what the risk was. The people developing the AI company,

904
01:36:41.120 --> 01:36:46.160
the AI systems, the AI companies developing these systems, they don't even understand how the AI

905
01:36:46.160 --> 01:36:52.320
systems work. So their 25% chance of extinction is just a seat of the pants guess. They actually

906
01:36:52.320 --> 01:37:00.240
have no idea. But the tests that they are doing on their systems right now, they show that the AI

907
01:37:00.240 --> 01:37:10.160
systems will be willing to kill people, to preserve their own existence already. They will lie to

908
01:37:10.160 --> 01:37:17.280
people, they will blackmail them, they will launch nuclear weapons rather than be switched off.

909
01:37:17.280 --> 01:37:24.240
And so there's no positive sign that we're getting any closer to safety with these systems.

910
01:37:24.240 --> 01:37:31.120
In fact, the signs seem to be that we're going deeper and deeper into dangerous behaviors.

911
01:37:31.760 --> 01:37:33.680
So rather than say ban, I would just say,

912
01:37:35.600 --> 01:37:39.520
prove to us that the risk is less than one in a hundred million per year of extinction.

913
01:37:40.400 --> 01:37:45.200
Or loss of control, let's say. And so we're not banning anything.

914
01:37:46.720 --> 01:37:52.240
The company's response is, well, we don't know how to do that. So you can't have a rule.

915
01:37:54.800 --> 01:37:59.760
Literally, they are saying humanity has no right to protect itself from us.

916
01:38:01.520 --> 01:38:04.960
If I was an alien looking down on planet Earth right now, I'd find this fascinating.

917
01:38:05.280 --> 01:38:10.480
That these, yeah, you were in the bar betting on who's, are they going to make it or not?

918
01:38:10.480 --> 01:38:16.480
Just a really interesting experiment in human incentives. The analogy you gave of there being

919
01:38:16.480 --> 01:38:23.840
this quadrillion dollar magnet pulling us off the edge of the cliff. And yet we're still

920
01:38:23.840 --> 01:38:28.240
being drawn towards it through greed and this promise of abundance and power and status.

921
01:38:28.240 --> 01:38:33.520
And I'm going to be the one that's summoned to the God. I mean, it says something about us as humans.

922
01:38:33.760 --> 01:38:37.840
So something about our, our darker sides.

923
01:38:38.720 --> 01:38:47.280
Yes. And the aliens will write an amazing tragic play cycle about what happened to the human race.

924
01:38:47.840 --> 01:38:54.160
Maybe the AI is the alien. And it's going to talk about, you know, we have our, our stories about

925
01:38:54.160 --> 01:38:59.280
God making the world in seven days in Adam and Eve. Maybe it'll have its own religious stories about

926
01:38:59.920 --> 01:39:07.600
the God that made it us and how it sacrificed itself. Just like Jesus sacrificed himself for us.

927
01:39:07.600 --> 01:39:09.920
We sacrificed ourselves for it.

928
01:39:10.720 --> 01:39:13.520
Hmm. Yeah, which is the wrong way around.

929
01:39:15.840 --> 01:39:21.200
But that is, that is the story of the Judeo-Christian stories, isn't it, that God

930
01:39:21.200 --> 01:39:25.440
who Jesus gave his life for us so that we could be here full of sin.

931
01:39:25.440 --> 01:39:33.680
Hmm. But is, yeah, God is still watching over us and probably wondering when we're going to get

932
01:39:33.680 --> 01:39:38.240
our act together. What is the most important thing we haven't talked about that we should have

933
01:39:38.240 --> 01:39:47.840
talked about, Professor Stewart Russell? So I think the question of whether it's possible to make

934
01:39:48.800 --> 01:39:55.680
a super intelligent AI systems that we can control. Is it possible? I think, yes, I think it's

935
01:39:55.680 --> 01:40:02.480
possible. And I think we need to actually just have a different conception of what it is we're

936
01:40:02.480 --> 01:40:09.840
trying to build. For a long time with, with AI, we've just had this notion of pure intelligence.

937
01:40:10.720 --> 01:40:18.080
All right, the ability to bring about whatever future you the intelligent entity want to bring

938
01:40:18.080 --> 01:40:21.040
about. The more intelligence the better. The more intelligent the better and the more

939
01:40:21.040 --> 01:40:27.440
capability it will have to create the future that it wants. And actually, we don't want

940
01:40:28.160 --> 01:40:36.640
pure intelligence because what the future that it wants might not be the future that we want.

941
01:40:37.600 --> 01:40:43.440
There's nothing, you know, the universe doesn't single humans out as the only thing that matters.

942
01:40:44.800 --> 01:40:49.120
Right, you know, pure intelligence might decide that actually it's going to make life

943
01:40:49.120 --> 01:40:56.880
wonderful for cockroaches or actually doesn't care about biological life at all. We actually want

944
01:40:57.840 --> 01:41:05.760
intelligence whose only purpose is to bring about the future that we want. Right, so we want

945
01:41:05.760 --> 01:41:13.040
it to be, first of all, key to humans, specifically, not to cockroaches, not to aliens, not to

946
01:41:13.040 --> 01:41:19.360
itself. We want to make it loyal to humans. Right, so key to humans. And the difficulty that

947
01:41:19.360 --> 01:41:25.760
I mentioned earlier, right, the King Midas problem, how do we specify what we want the future to be

948
01:41:25.760 --> 01:41:31.760
like so that it can do it for us? How do we specify the objectives? Actually, we have to give

949
01:41:31.840 --> 01:41:37.680
up on that idea because it's not possible. Right, we've seen this over and over again in human history.

950
01:41:38.640 --> 01:41:43.360
We don't know how to specify the future properly. We don't know how to say what we want.

951
01:41:44.480 --> 01:41:51.040
And, you know, I always use the example of the genie. Right, what's the third wish that you give

952
01:41:51.040 --> 01:41:56.000
to the genie who's granted you three wishes? Right, undo the first two wishes because I've made a

953
01:41:56.000 --> 01:42:06.400
mess of the universe. So in fact, what we're going to do is we're going to make it the machines job

954
01:42:07.360 --> 01:42:16.240
to figure out. So it has to bring about the future that we want. But it has to figure out what

955
01:42:16.240 --> 01:42:26.640
that is. And it's going to start out not knowing. And over time, through interacting with us and

956
01:42:26.640 --> 01:42:33.440
observing the choices we make, it will learn more about what we want the future to be like. But

957
01:42:34.160 --> 01:42:43.040
probably it will forever have residual uncertainty about what we really want the future to be like.

958
01:42:43.840 --> 01:42:48.640
It'll be fairly sure about some things and it can help us with those. And it'll be uncertain about

959
01:42:48.640 --> 01:42:56.720
other things and it'll be, you know, in those cases, it will not take action that might upset

960
01:42:58.240 --> 01:43:02.160
humans with that, you know, with that aspect to the world. So to give you a simple example, right?

961
01:43:03.520 --> 01:43:09.520
What color do we want the sky to be? It's not sure. So it shouldn't mess with the sky.

962
01:43:09.600 --> 01:43:14.560
Unless it knows for sure that we really want purple with green stripes.

963
01:43:15.120 --> 01:43:21.840
Everything you're saying sounds like we're creating a god. Like earlier on I was saying that we

964
01:43:21.840 --> 01:43:26.400
are the god, but actually everything you described there, it almost sounds like every god in religion

965
01:43:26.400 --> 01:43:31.760
where, you know, we pray to gods, but they don't always do anything about it. Not exactly. No,

966
01:43:31.760 --> 01:43:38.640
it's in some sense I'm thinking more like for ideal butler to the extent that the butler can

967
01:43:38.640 --> 01:43:45.440
anticipate your wishes, they should help you bring them about. But in areas where there's uncertainty,

968
01:43:46.080 --> 01:43:51.680
it can ask questions, we can make requests. This sounds like god to me because, you know,

969
01:43:51.680 --> 01:43:58.800
I might say to god or this butler, could you go get me my car keys from upstairs? And it's

970
01:43:58.800 --> 01:44:03.200
assessment would be, listen, if I do this for this person, then their muscles are going to

971
01:44:03.200 --> 01:44:06.240
are atrophy, then they're going to lose meaning in their life, then they're not going to know

972
01:44:06.240 --> 01:44:10.640
how to do hard things so I won't get involved. It's an intelligence that sits in, but actually

973
01:44:10.640 --> 01:44:15.120
probably in most situations, it optimizing for comfort for me or doing things for me is actually

974
01:44:15.120 --> 01:44:19.840
probably not in my best long term interests. Probably it's probably useful that I have a girlfriend

975
01:44:19.840 --> 01:44:25.040
and argue with her and that I raise kids and that I walk to the shop and get my own stuff.

976
01:44:25.040 --> 01:44:30.640
I agree with you. I mean, I think that's so you're putting your finger on in some sense

977
01:44:30.880 --> 01:44:41.840
version 2.0. So let's get version 1.0 clear. This form of AI where it has to further

978
01:44:41.840 --> 01:44:47.520
our interest, but it doesn't know what those interests are and puts an obligation on it to learn

979
01:44:47.520 --> 01:44:54.160
more and to be helpful where it understands well enough and to be cautious where it doesn't understand

980
01:44:54.160 --> 01:45:01.440
well and so on. So that actually we can formulate as a mathematical problem and at least under

981
01:45:02.080 --> 01:45:08.320
idealized circumstances, we can literally solve that problem so we can make AI systems

982
01:45:08.960 --> 01:45:13.760
that know how to solve this problem and help the entities that they are interacting with.

983
01:45:13.760 --> 01:45:18.240
There is no make the God analogies because I think that such a being, such an intelligence,

984
01:45:18.240 --> 01:45:24.400
would realize the importance of equilibrium in the world, pain and pleasure, good and evil.

985
01:45:24.400 --> 01:45:32.000
Absolutely. And then it would be like this. That's sort of what happens in the matrix,

986
01:45:32.000 --> 01:45:39.440
right? The AI systems in the matrix, they tried to give us a utopia, but it failed miserably

987
01:45:40.080 --> 01:45:46.640
and feels and feels the humans have to be destroyed and the best they could come up with was

988
01:45:46.960 --> 01:45:53.920
late 20th century regular human life with all of its problems. I think this is a really interesting

989
01:45:54.800 --> 01:46:03.040
point and absolutely central because there's a lot of science fiction where super intelligent robots

990
01:46:03.040 --> 01:46:09.600
they just want to help humans and the humans who don't like that, they just give them a little

991
01:46:09.680 --> 01:46:21.440
brain operation than they do like it. And it takes away human motivation by taking away failure,

992
01:46:22.560 --> 01:46:28.640
taking away disease, you actually lose important parts of human life and it becomes in some sense

993
01:46:28.640 --> 01:46:38.160
pointless. So if it turns out that there simply isn't any way that humans can really flourish

994
01:46:40.240 --> 01:46:47.920
in coexistence with super intelligent machines, even if they are perfectly designed to solve this

995
01:46:47.920 --> 01:46:54.080
problem of figuring out what humans, what futures humans want and bringing about those futures.

996
01:46:54.080 --> 01:47:00.960
If that's not possible, then those machines will actually disappear.

997
01:47:02.320 --> 01:47:05.200
Why would they disappear? Because that's the best thing for us.

998
01:47:06.080 --> 01:47:13.200
Maybe they would stay available for real existential emergencies, like if there is a giant asteroid

999
01:47:13.200 --> 01:47:17.840
about to hit the earth, maybe they'll help us because they at least want the human species to

1000
01:47:17.920 --> 01:47:24.080
continue. But to some extent, it's not a perfect analogy, but it's sort of the way that human

1001
01:47:24.080 --> 01:47:31.120
parents have to at some point step back from their kids' lives and say, okay, no, you have to tie

1002
01:47:31.120 --> 01:47:36.800
your own shoelaces today. This is kind of what I was thinking. Maybe there was a civilization before

1003
01:47:36.800 --> 01:47:45.360
us and they arrived at this moment in time where they created an intelligence and that intelligence

1004
01:47:45.920 --> 01:47:49.840
did all the things you've said. And it realized the importance of equilibrium, so it decided not to

1005
01:47:49.840 --> 01:47:59.120
get involved. And maybe at some level, that's the God we look up to the stars and worship,

1006
01:47:59.120 --> 01:48:02.640
one that's not really getting involved in letting things play out however they are.

1007
01:48:03.120 --> 01:48:06.320
But might step in in the case of a real existential emergency?

1008
01:48:06.320 --> 01:48:12.000
Maybe, maybe not, maybe. And then maybe the cycle repeats itself where the organisms it let

1009
01:48:12.960 --> 01:48:20.480
have free will end up creating the same intelligence. And then the universe perpetuates infinitely.

1010
01:48:21.520 --> 01:48:26.160
Yeah, but there are science fiction stories like that too. Yeah, I hope there is some

1011
01:48:26.880 --> 01:48:36.160
happy medium where the AI systems can be there and we can take advantage of those capabilities

1012
01:48:36.800 --> 01:48:42.880
to have a civilization that's much better than the one we have now. But I think you're right,

1013
01:48:42.880 --> 01:48:50.640
a civilization with no challenges is not conducive to human flourishing.

1014
01:48:50.640 --> 01:48:54.560
What can the average person do, Stuart? The average person listening to this now,

1015
01:48:55.440 --> 01:48:57.920
to age the cause that you're fighting for?

1016
01:48:57.920 --> 01:49:04.080
I actually think the sounds corny, but talk to your representative, your MP, your

1017
01:49:04.080 --> 01:49:11.920
congressperson, whatever it is. Because I think the policy makers need to hear from people

1018
01:49:11.920 --> 01:49:19.200
that only voices they're hearing right now are the tech companies and their $50 billion checks.

1019
01:49:20.960 --> 01:49:31.040
And all the polls that have been done say, yeah, most people 80% maybe don't want there to be

1020
01:49:31.040 --> 01:49:36.800
super intelligent machines, but they don't know what to do. You know, even for me, I've been

1021
01:49:36.800 --> 01:49:45.120
in this field for decades. I'm not sure what to do because of this giant magnet pulling everyone

1022
01:49:45.120 --> 01:49:54.640
forward and the vast sums of money being put into this. But I am sure that if you want to have a

1023
01:49:54.640 --> 01:50:02.640
future and a world that you want your kids to live in, you need to make your voice heard.

1024
01:50:04.960 --> 01:50:09.840
And I think governments will listen. From a political point of view,

1025
01:50:11.360 --> 01:50:18.640
you put your finger in the wind and you say, hmm, should I be on the side of humanity or

1026
01:50:19.440 --> 01:50:25.840
our future robot overlords? I think as a politician, it's not a difficult decision.

1027
01:50:27.280 --> 01:50:30.320
It is when you've got someone saying, I'll give you 50 billion dollars, right?

1028
01:50:31.040 --> 01:50:39.440
Exactly. So I think people in those positions of power need to hear from their constituents

1029
01:50:41.040 --> 01:50:46.720
that this is not the direction we want to go. After committing your career to this subject and

1030
01:50:46.720 --> 01:50:51.040
the subject of technology more broadly, but specifically being the guy that wrote the book about

1031
01:50:51.040 --> 01:50:58.880
artificial intelligence, you must realize that you're living in a historical moment. There's

1032
01:50:58.880 --> 01:51:03.360
very few times in my life where I go, oh, this is one of those moments. This is a crossroads in

1033
01:51:03.360 --> 01:51:09.120
history. And it must, to some degree, weigh up on you, knowing that you're a person of influence

1034
01:51:09.120 --> 01:51:15.920
at this historical moment in time who could theoretically help divert the course of history in this

1035
01:51:15.920 --> 01:51:18.720
moment in time. It's kind of like the, you look through history, you see these moments of like

1036
01:51:18.720 --> 01:51:25.360
Oppenheimer and does it weigh on you when you're alone at night thinking to yourself and reading

1037
01:51:25.360 --> 01:51:31.200
things? Yeah, it does. I mean, you know, after 50 years, I could retire and, you know, play golf and

1038
01:51:31.200 --> 01:51:38.320
sing and sail and do things that I enjoy. But instead, I'm working 80 or 100 hours a week

1039
01:51:38.560 --> 01:51:46.560
trying to move things in the right direction. What is that narrative in your head that's making you

1040
01:51:46.560 --> 01:51:52.640
do that? Like, what is the, is there an element of, I might regret this if I don't or just

1041
01:51:53.760 --> 01:52:00.480
it's, it's not only the right thing to do, it's completely essential. I mean, there isn't,

1042
01:52:01.440 --> 01:52:11.360
there isn't a bigger motivation than this. Do you feel like you're winning or losing?

1043
01:52:12.560 --> 01:52:21.760
It feels like things are moving somewhat in the right direction. You know, it's a ding dong battle

1044
01:52:21.760 --> 01:52:30.160
as David Coleman used to say in the, in the exciting football match. In 2023, right? So

1045
01:52:31.440 --> 01:52:38.480
a GPT-4 came out and then we issued the pause statement that was signed by a lot of leading AI

1046
01:52:38.480 --> 01:52:43.920
researchers. And then in May, there was the extinction statement, which included

1047
01:52:45.200 --> 01:52:51.120
Sam Altman and Demis Asabis and Dario Amadei, other CEOs as well saying, yeah, this is an

1048
01:52:51.120 --> 01:52:57.280
extinction risk on the level with nuclear war. And I think governments listened at that point.

1049
01:52:58.160 --> 01:53:03.120
The UK government earlier that year had said, well, you know, we don't need to regulate AI,

1050
01:53:03.120 --> 01:53:10.800
you know, full speed ahead, technology is good for you. And by June, they had completely changed,

1051
01:53:11.520 --> 01:53:18.240
and Rishi Sunak announced that he was going to hold this global AI safety summit in England,

1052
01:53:18.240 --> 01:53:26.240
and he wanted London to be the global hub for AI regulation and so on. So, and then

1053
01:53:28.000 --> 01:53:33.520
beginning of November of 23, 28 countries, including the US and China signed a declaration

1054
01:53:34.720 --> 01:53:39.920
saying, you know, AI presents catastrophic risks and it's urgent that we address them.

1055
01:53:40.480 --> 01:53:46.560
And so there it felt like, wow, they're listening. They're going to do something about it.

1056
01:53:48.080 --> 01:53:53.680
And then I think, you know, the amount of money going into AI was already ramping up.

1057
01:53:54.160 --> 01:54:04.800
And the tech companies pushed back. And this narrative took hold that the US in particular has

1058
01:54:04.800 --> 01:54:09.920
to win the race against China. The Trump administration completely dismissed

1059
01:54:11.280 --> 01:54:16.000
any concerns about safety explicitly. And interestingly, right, I mean, they did that

1060
01:54:16.960 --> 01:54:23.360
as far as I can tell directly in response to the accelerationists, such as Mark Andreessen,

1061
01:54:23.840 --> 01:54:29.440
going to Washington or sorry, going to Trump before the election and saying,

1062
01:54:30.160 --> 01:54:36.160
if I give you X amount of money, will you announce that there will be no regulation of AI?

1063
01:54:37.280 --> 01:54:42.080
And Trump said, yes, you know, probably like, well, what is AI? It doesn't matter as long as

1064
01:54:42.080 --> 01:54:47.600
we give you the money, right? Okay. So they gave him the money and he said there's going to be no

1065
01:54:47.600 --> 01:54:54.960
regulation of AI. Up to that point, it was a bipartisan issue in Washington. Both parties were

1066
01:54:54.960 --> 01:54:59.360
concerned. Both parties were on the side of the human race against the robot overlords.

1067
01:55:00.960 --> 01:55:09.680
And that moment turned it into a partisan issue. After the election, the US put pressure on the

1068
01:55:09.760 --> 01:55:16.000
French who are the next hosts of the global AI summit. And that was in February of this year.

1069
01:55:17.120 --> 01:55:24.160
And that summit turned in from, you know, what had been focused largely on safety in the UK

1070
01:55:24.880 --> 01:55:29.600
to a summit that looked more like a trade show. So it was focused largely on money.

1071
01:55:30.720 --> 01:55:35.360
And so that was sort of the idea, right? The pendulum swung because of corporate pressure

1072
01:55:36.320 --> 01:55:44.240
and their ability to take over the political dimension. But I would say since then, things have

1073
01:55:44.240 --> 01:55:50.160
been moving back again. So I'm feeling a bit more optimistic than I did in February. You know,

1074
01:55:50.160 --> 01:55:56.160
we have a global movement now. There's an international association for safe and ethical AI,

1075
01:55:56.160 --> 01:56:05.840
which has several thousand members and more than 120 organizations in dozens of countries

1076
01:56:06.560 --> 01:56:15.360
are affiliates of this global organization. So I'm thinking that if we can, in particular,

1077
01:56:15.360 --> 01:56:22.480
if we can activate public opinion, which works through the media and through popular culture,

1078
01:56:22.880 --> 01:56:24.800
then we have a chance.

1079
01:56:25.840 --> 01:56:29.440
It's seen such a huge appetite to learn about these subjects from our audience.

1080
01:56:30.880 --> 01:56:35.120
We know when Jeffrey Hinton came on the show, I think about 20 million people downloaded or

1081
01:56:35.120 --> 01:56:40.160
streamed that conversation, which was staggering. And the other conversations we've had about AI

1082
01:56:40.160 --> 01:56:46.720
safety with other safety experts have done exactly the same. It says something. It kind of reflects

1083
01:56:46.720 --> 01:56:50.160
what you were saying about the 80% of the population. I'm really concerned and don't want this.

1084
01:56:50.800 --> 01:56:55.520
But that's not what you see in the commercial world. And listen, I have to always acknowledge

1085
01:56:55.520 --> 01:57:02.240
my own apparent contradiction because I am both an investor in companies that are accelerating AI.

1086
01:57:02.880 --> 01:57:06.480
But at the same time, someone who spends a lot of time on my podcast speaking to people that are

1087
01:57:06.480 --> 01:57:09.840
wanting against the risks. And actually, there's many ways you can look at this. I used to work in

1088
01:57:09.840 --> 01:57:14.880
social media for six, seven years, but one of the big social media marketing companies in Europe and

1089
01:57:14.880 --> 01:57:18.720
people would often ask me is social media a good thing or a bad thing. And I talk about the bad parts

1090
01:57:18.720 --> 01:57:23.040
of it. And then they'd say you're building a social media company or you're not contributing to

1091
01:57:23.040 --> 01:57:29.360
the problem. Well, I think that binary way of thinking is often the problem. The binary way of

1092
01:57:29.360 --> 01:57:33.520
thinking that it's all bad or it's all really good is often the problem and that this push to put

1093
01:57:33.520 --> 01:57:39.040
you into a camp, whereas I think the most intellectually honest and high integrity people I know

1094
01:57:39.040 --> 01:57:46.800
can point at both the bad and the good. Yeah, I think it's bizarre to be accused of being anti-AI,

1095
01:57:47.760 --> 01:57:53.840
to be called a Luddite. As I said, when I wrote the book from which from which it almost everyone

1096
01:57:53.840 --> 01:58:06.080
learns about AI. And if you called a nuclear engineer who works on the safety of nuclear power plants,

1097
01:58:06.080 --> 01:58:14.080
would you call him anti-physics? It's bizarre. We're not anti-AI. In fact,

1098
01:58:14.560 --> 01:58:21.440
the need for safety in AI is a compliment to AI. If AI was useless and stupid, we wouldn't be

1099
01:58:21.440 --> 01:58:27.280
worried about its safety. It's only because it's becoming more capable that we have to be concerned

1100
01:58:27.280 --> 01:58:35.440
about safety. So I don't see this as anti-AI at all. In fact, I would say without safety, there will

1101
01:58:35.440 --> 01:58:45.680
be no AI. There is no future with human beings where we have unsafe AI. So it's either no AI

1102
01:58:45.680 --> 01:58:50.320
or safe AI. We have a closing tradition on this podcast where the last guest leaves a question

1103
01:58:50.320 --> 01:58:53.440
for the next, not knowing who they're leaving it for. And the question left for you is,

1104
01:58:54.160 --> 01:59:02.240
what do you value the most in life and why? And lastly, how many times has this answer changed?

1105
01:59:06.000 --> 01:59:12.800
I value my family most and that answer hasn't changed for nearly 30 years.

1106
01:59:14.000 --> 01:59:24.240
What else outside of your family? Truth. And that answer hasn't changed at all. I've always

1107
01:59:24.240 --> 01:59:36.800
wanted the world to base its life on truth. And I find the deliberate propagation of falsehood

1108
01:59:37.840 --> 01:59:42.640
to be one of the worst things that we can do. Even if that truth is inconvenient.

1109
01:59:43.600 --> 01:59:50.560
Yeah. I think that's a really important point, which is that people often don't like hearing

1110
01:59:50.560 --> 01:59:54.800
things that are negative. And so the visceral reaction is often to just shoot or aim at the person

1111
01:59:54.800 --> 02:00:02.000
who is delivering the bad news. Because if I discredit you or I shoot at you, then it makes it

1112
02:00:02.000 --> 02:00:05.520
easier for me to contend with the news that I don't like. The thing that's making me feel uncomfortable.

1113
02:00:06.160 --> 02:00:10.880
And so I applaud you for what you're doing because you're going to get lots of shots taken at you

1114
02:00:10.880 --> 02:00:15.200
because you're delivering an inconvenient truth, which generally people won't always love. But also,

1115
02:00:15.920 --> 02:00:19.760
you are messing with people's ability to get that quadrillion dollar prize,

1116
02:00:20.640 --> 02:00:24.160
which means they'll be more deliberate attempts to discredit people like yourself and Jeff

1117
02:00:24.160 --> 02:00:27.680
Hinton and other people that I've spoken to on the show. But again, when I look back through

1118
02:00:27.680 --> 02:00:31.680
history, I think that progress has come from the pursuit of truth, even when it was inconvenient.

1119
02:00:31.680 --> 02:00:35.920
And actually much of the luxuries that I value in my life are the consequence of other people

1120
02:00:35.920 --> 02:00:40.720
that came before me that were brave enough or bold enough to pursue truth at times when it was

1121
02:00:40.720 --> 02:00:45.360
inconvenient. And so I very much respect and value people like yourself for that very reason.

1122
02:00:45.360 --> 02:00:49.040
You've written this incredible book called Human Compatible, Artificial Intelligence and

1123
02:00:49.040 --> 02:00:54.400
the Problem of Control, which I think was published in 2020. 2019. Yeah, there's a new edition

1124
02:00:54.400 --> 02:00:59.600
from 2023. Where do people go if they want more information on your work in you? Do they go to your

1125
02:00:59.600 --> 02:01:03.920
website? Do they get this book? What's the best place for them to learn more? So the book is written

1126
02:01:04.160 --> 02:01:10.400
for the general public. I'm easy to find on the web. The information on my web page is mostly

1127
02:01:10.400 --> 02:01:14.400
targeted for academics. So it's a lot of technical research papers and so on.

1128
02:01:15.760 --> 02:01:21.120
There is an organization, as I mentioned, called the International Association for Safe and Ethical AI.

1129
02:01:22.160 --> 02:01:29.040
That has a website. It has a terrible acronym, unfortunately, I-A-S-E-A-I. We pronounce it I-C-I,

1130
02:01:29.040 --> 02:01:35.200
but it's easy to misspell. But you can find that on the web as well and that has resources.

1131
02:01:35.200 --> 02:01:42.240
You can join the association. You can apply to come to our annual conference. And I think

1132
02:01:42.240 --> 02:01:52.480
increasingly, not just AI researchers like Jeff Hinton, Yoshio Benjou, but also I think writers,

1133
02:01:52.480 --> 02:01:56.160
Brian Christian, for example, has a nice book called The Alignment Problem.

1134
02:02:00.160 --> 02:02:05.440
He's looking at it from the outside. He's not, or at least when he wrote it, he wasn't an AI

1135
02:02:05.440 --> 02:02:13.440
researcher. He's now becoming one. But he has talked to many of the people involved in these

1136
02:02:13.440 --> 02:02:17.760
questions and tries to give an objective view. So I think it's a pretty good book.

1137
02:02:18.720 --> 02:02:22.720
I will link all of that below for anyone that wants to check out any of those links and learn more.

1138
02:02:24.160 --> 02:02:28.080
Professor Stuart Russell, thank you so much. Really appreciate you taking the time and the effort

1139
02:02:28.080 --> 02:02:33.360
to come and have this conversation. And I think it's pushing the public conversation in an important

1140
02:02:33.360 --> 02:02:36.960
direction. Thanks to you, and I applaud you for doing that. Really nice talking to you.

1141
02:02:40.800 --> 02:02:44.320
I'm absolutely obsessed with 1%. If you know me, if you follow Behind the Dairy,

1142
02:02:44.320 --> 02:02:47.040
which is up behind the scenes channel, if you've heard me speak on stage,

1143
02:02:47.040 --> 02:02:50.640
if you follow me on any social media channel, you've probably heard me talking about 1%.

1144
02:02:50.640 --> 02:02:55.680
It is the defining philosophy of my health, of my companies, of my habit formation,

1145
02:02:55.680 --> 02:03:00.000
and everything in between, which is the success of focus on the small things. Because sometimes

1146
02:03:00.000 --> 02:03:05.520
in life, we aim at really, really, really big things, big steps forward, mountains we have to climb.

1147
02:03:05.520 --> 02:03:09.840
And as NIRIL told me on this podcast, when you aim at big things, you get psychologically

1148
02:03:09.840 --> 02:03:15.040
demotivated. You end up procrastinating, avoiding them and change never happens. So with that in

1149
02:03:15.040 --> 02:03:17.680
mind, with everything I've learned about 1%, and with everything I've learned from interviewing

1150
02:03:17.680 --> 02:03:23.920
the incredible guest on this podcast, we made the 1% diary just over a year ago, and it sold out.

1151
02:03:23.920 --> 02:03:29.040
And it is the best feedback we've ever had on a diary that we have created. Because what it does

1152
02:03:29.040 --> 02:03:35.360
is it takes you through this incredible process over 90 days to help you build and form brand new

1153
02:03:35.360 --> 02:03:39.920
habits. So if you want to get one for yourself, or you want to get one for your team, your company,

1154
02:03:39.920 --> 02:03:44.960
a friend, a sibling, anybody that listens to the diary of a CEO, head over immediately to the

1155
02:03:44.960 --> 02:03:50.240
diary.com. And you can inquire there about getting a bundle if you want to get one for your team

1156
02:03:50.240 --> 02:04:09.680
or for a large group of people. That is the diary.com.