WEBVTT

1
00:00:00.111 --> 00:00:13.621
Hello internet, this is your captain speaking Lester Nare joined as always by my co-host and our resident PhD newly nicknamed some physics nerd Krishna Chowdhury Courtesy of young Jamie.

2
00:00:13.981 --> 00:00:14.781
Yes, how's it going?

3
00:00:15.222 --> 00:00:17.183
Um, it's been an interesting week.

4
00:00:17.423 --> 00:00:19.405
It's been it's been pretty incredible, right?

5
00:00:19.725 --> 00:00:24.068
I I got a DM And it was a screenshot of your face.

6
00:00:24.668 --> 00:00:29.091
Yeah, and Joe Rogan's face Yeah, and and I thought it was AI

7
00:00:30.486 --> 00:00:31.786
I thought it was just some deep fake.

8
00:00:32.507 --> 00:00:34.347
And it took us a minute to find it.

9
00:00:34.507 --> 00:00:36.468
So kudos first and foremost.

10
00:00:36.748 --> 00:00:37.888
We're back for season two.

11
00:00:38.208 --> 00:00:39.149
We're super excited.

12
00:00:39.889 --> 00:00:52.493
This week has been a little bit crazy because young Jamie, the producer over at the Joe Rogan Experience, decided to reference our Nuno Lurero Christmas special episode in the middle of a UFO segment, no less.

13
00:00:52.513 --> 00:00:52.773
Yeah.

14
00:00:53.273 --> 00:00:56.034
And that was very unexpected.

15
00:00:56.194 --> 00:00:57.496
Yeah, it was pretty incredible.

16
00:00:58.056 --> 00:01:00.499
And he called us some physics nerds.

17
00:01:00.579 --> 00:01:00.800
Yeah.

18
00:01:01.440 --> 00:01:02.081
Point of order.

19
00:01:03.143 --> 00:01:04.124
I am no physics nerd.

20
00:01:04.664 --> 00:01:06.086
And it was a compliment to me.

21
00:01:06.186 --> 00:01:06.847
It was a compliment.

22
00:01:06.907 --> 00:01:07.768
It was a compliment.

23
00:01:08.189 --> 00:01:08.890
It was a compliment.

24
00:01:09.190 --> 00:01:10.672
And you're an honorary physics nerd.

25
00:01:10.692 --> 00:01:12.754
I feel like you've learned a lot.

26
00:01:13.875 --> 00:01:15.937
I mean, I've learned a lot.

27
00:01:16.037 --> 00:01:21.321
So if I can quote Navier Stokes' equation off rip, you know, it's not too bad.

28
00:01:21.461 --> 00:01:22.042
Exactly.

29
00:01:22.963 --> 00:01:23.643
We're going to dive in.

30
00:01:23.663 --> 00:01:25.004
We have a great episode today.

31
00:01:25.024 --> 00:01:30.208
We have a new segment that we're going to try out for in between one of our stories.

32
00:01:30.929 --> 00:01:33.611
And we're going to talk about a variety of cool topics.

33
00:01:33.631 --> 00:01:36.273
We're going to start off with Roman Concrete.

34
00:01:37.665 --> 00:01:40.526
And then pivot over to our favorite institution, Princeton.

35
00:01:40.586 --> 00:01:41.426
Go Tigers!

36
00:01:41.846 --> 00:01:44.727
For a new cognitive Lego.

37
00:01:45.087 --> 00:01:45.387
Yes.

38
00:01:45.767 --> 00:01:47.187
Which I love Legos.

39
00:01:47.207 --> 00:01:49.228
We can see from the Lego that's behind us here.

40
00:01:49.348 --> 00:01:54.689
Yes, but this is the Legos that make up our thoughts and our abstract ideas in our head.

41
00:01:55.169 --> 00:01:57.630
I think it's a very cool story, and it's very cool research.

42
00:01:57.810 --> 00:01:59.130
Minority Report Inbound.

43
00:01:59.150 --> 00:01:59.270
Yes.

44
00:01:59.890 --> 00:02:02.372
There was another deep seek moment.

45
00:02:02.452 --> 00:02:03.453
Took over the internet.

46
00:02:03.473 --> 00:02:05.154
They did a January's back to back.

47
00:02:05.434 --> 00:02:07.816
I think our one was January last year.

48
00:02:07.836 --> 00:02:08.016
Yep.

49
00:02:08.436 --> 00:02:13.380
And now some new, something about constrained manifolds.

50
00:02:13.420 --> 00:02:13.821
Yeah.

51
00:02:14.161 --> 00:02:15.262
Manifold constraints.

52
00:02:15.602 --> 00:02:16.362
Very complicated.

53
00:02:16.703 --> 00:02:21.106
And we will end our story with our former classmates, favorite character.

54
00:02:21.804 --> 00:02:23.286
Finance, bro-ology.

55
00:02:23.867 --> 00:02:25.168
It's our first economic story.

56
00:02:25.288 --> 00:02:26.069
Yeah.

57
00:02:26.170 --> 00:02:27.111
That'll be interesting.

58
00:02:27.692 --> 00:02:33.599
And as always, we're going to get into the weeds of the research papers, the methods, the experimental design.

59
00:02:34.000 --> 00:02:38.285
You might learn something today because this is from First Principle.

60
00:02:54.075 --> 00:02:57.598
Okay, our first story is about Roman concrete.

61
00:02:59.040 --> 00:02:59.880
Have you ever been to Rome?

62
00:03:00.601 --> 00:03:01.021
I have not.

63
00:03:01.081 --> 00:03:06.066
I was just in Italy, but I did not get the chance to partake in the life of the Roman.

64
00:03:06.647 --> 00:03:07.727
Yeah, Rome is...

65
00:03:08.680 --> 00:03:27.906
honestly, one of my favorite cities that I've ever been to because the amount of history that's there and the continuous civilization, you know, it's, it's been around for a millennia now and you can see buildings that are literally 2000 years old that I can scarce believe they built 2000 years ago.

66
00:03:28.326 --> 00:03:30.166
It's actually like pretty incredible.

67
00:03:30.386 --> 00:03:33.927
And all of that is because of something called Roman concrete.

68
00:03:34.488 --> 00:03:34.748
Okay.

69
00:03:35.388 --> 00:03:35.528
Um,

70
00:03:36.168 --> 00:03:46.568
There's been a breakthrough because a recent excavation in Pompeii has actually led us to figure out how they made that Roman concrete.

71
00:03:47.309 --> 00:03:48.772
And it's been elusive for...

72
00:03:49.709 --> 00:04:00.052
to even material scientists in the modern day, how the Romans made that and how they could build such buildings that last 2,000 years and are still used today.

73
00:04:00.433 --> 00:04:00.673
Okay?

74
00:04:00.953 --> 00:04:02.293
It's pretty incredible.

75
00:04:02.613 --> 00:04:04.034
I think I heard about that on one of the tours.

76
00:04:04.074 --> 00:04:06.014
Oh, the concrete is da-da-da.

77
00:04:06.054 --> 00:04:06.775
Yeah, yeah.

78
00:04:07.215 --> 00:04:09.936
I mean, it's really quite something if you think about it.

79
00:04:10.016 --> 00:04:14.217
So there was this thing called the Roman Architectural Revolution, okay?

80
00:04:14.317 --> 00:04:17.238
It's this widespread use of Roman architecture, right?

81
00:04:17.789 --> 00:04:20.311
Previously unused forms of architecture.

82
00:04:20.611 --> 00:04:23.634
They used to have arches, vaults, domes.

83
00:04:23.934 --> 00:04:27.157
Before the Romans, the Greeks never made this kind of stuff.

84
00:04:27.397 --> 00:04:29.819
But because of concrete, we can actually do it.

85
00:04:29.879 --> 00:04:32.021
This is the Pont du Gard in France.

86
00:04:32.321 --> 00:04:37.326
What we just saw was the semicircular arches that were in the theater of Marcellus.

87
00:04:37.366 --> 00:04:40.328
This is all by Augustus, the Emperor Augustus.

88
00:04:40.769 --> 00:04:43.371
Yeah, he's the guy who took over right after Caesar got stabbed.

89
00:04:43.471 --> 00:04:43.671
Right.

90
00:04:43.871 --> 00:04:44.091
Right.

91
00:04:44.892 --> 00:04:45.353
Two Brutus?

92
00:04:46.494 --> 00:04:53.403
Yeah, A2 Brutus was Caesar, and then Augustus said, you know, friends, Romans, countrymen, lend me your ears.

93
00:04:54.004 --> 00:04:54.685
But Caesar...

94
00:04:55.486 --> 00:04:58.170
But Brutus is an honorable man, right?

95
00:04:58.850 --> 00:04:59.511
Great, great.

96
00:04:59.772 --> 00:05:01.053
One of my favorite Shakespeare plays.

97
00:05:01.534 --> 00:05:01.734
So...

98
00:05:02.315 --> 00:05:05.617
They're building all this crazy, crazy stuff, right?

99
00:05:05.937 --> 00:05:08.339
40 AD, this is the time of Jesus.

100
00:05:08.759 --> 00:05:14.483
The crown jewel is really the Pantheon, which was built by Emperor Hadrian in 126 AD.

101
00:05:14.843 --> 00:05:21.307
That building is still intact, and people can go inside, and there's no damage to it.

102
00:05:21.787 --> 00:05:22.608
It's pretty insane.

103
00:05:22.688 --> 00:05:31.694
2,000 years after it's built, the Pantheon's dome is still the largest unreinforced concrete dome in the world.

104
00:05:32.573 --> 00:05:33.434
That's today?

105
00:05:33.554 --> 00:05:33.995
Today.

106
00:05:34.736 --> 00:05:37.959
That's actually pretty incredible, especially given the scale of architecture projects that are happening.

107
00:05:38.099 --> 00:05:38.680
Yeah, yeah, yeah.

108
00:05:38.720 --> 00:05:39.121
Yeah.

109
00:05:39.281 --> 00:05:40.162
It's pretty insane.

110
00:05:40.222 --> 00:05:44.447
I think the Superdome was, like, coming close, but then, like, Hurricane Katrina happened.

111
00:05:45.148 --> 00:05:47.150
But this is a single dome.

112
00:05:47.170 --> 00:05:48.952
It's a concrete dome with a hole on the top.

113
00:05:49.933 --> 00:05:50.133
Right.

114
00:05:50.214 --> 00:05:51.075
It's massive.

115
00:05:51.155 --> 00:05:51.375
Yeah.

116
00:05:51.636 --> 00:05:55.141
And it's just still around entirely made out of concrete.

117
00:05:55.461 --> 00:06:00.288
We have our next photo where it shows you can see it from the Roman skyline.

118
00:06:00.429 --> 00:06:01.390
That's the concrete dome.

119
00:06:01.811 --> 00:06:02.211
Right.

120
00:06:02.271 --> 00:06:04.435
And it's still it's that's 2000 years.

121
00:06:04.455 --> 00:06:05.937
That's so crazy.

122
00:06:06.197 --> 00:06:10.260
Yeah, that dome is 2,000 years old and it's still alive to this day.

123
00:06:10.600 --> 00:06:15.543
And all of this has to do with the fact that the Romans invented Roman concrete.

124
00:06:16.064 --> 00:06:16.844
Okay?

125
00:06:17.665 --> 00:06:20.366
And we still can't figure out how they did it.

126
00:06:20.647 --> 00:06:20.887
Okay.

127
00:06:21.147 --> 00:06:22.308
But we're getting close.

128
00:06:22.628 --> 00:06:25.570
To me, that's—how do we not know?

129
00:06:25.870 --> 00:06:26.090
Yeah.

130
00:06:26.331 --> 00:06:29.895
I mean, we're going to find out why it's so difficult, right?

131
00:06:30.055 --> 00:06:41.569
In 2017, there was a paper in the American Mineralologist from the University of Utah, and they actually showed that when this concrete got in touch with marine ecosystems, so with saltwater,

132
00:06:42.678 --> 00:06:45.379
it would actually strengthen itself.

133
00:06:45.759 --> 00:06:49.299
It was like a smart material where it was like repairing itself.

134
00:06:49.479 --> 00:06:56.841
If there was a crack in the concrete, unlike modern concrete, where the thing just like, you know, you got to decommission it, you got to tear it down, build a new bridge.

135
00:06:57.221 --> 00:06:59.741
This stuff was actually self-reinforcing.

136
00:07:00.281 --> 00:07:00.522
Okay.

137
00:07:00.842 --> 00:07:01.902
So that was in 2017.

138
00:07:01.962 --> 00:07:06.423
Already we've got like a smart material that's better than modern day concrete.

139
00:07:06.763 --> 00:07:06.983
Okay.

140
00:07:07.183 --> 00:07:07.883
So the question is,

141
00:07:09.050 --> 00:07:10.051
How do you do that?

142
00:07:10.131 --> 00:07:10.632
Yeah, yeah.

143
00:07:10.652 --> 00:07:13.516
Because we see it in superhero movies when they get the thingy.

144
00:07:13.676 --> 00:07:17.261
Or the Foundation, the Apple TV show.

145
00:07:17.922 --> 00:07:23.409
One of the, spoiler alert, one of the characters is this sentient robot that's lived on forever.

146
00:07:23.870 --> 00:07:26.173
And has a self-healing kind of.

147
00:07:26.333 --> 00:07:29.836
Yeah, these Romans are doing it 2,000 years ago.

148
00:07:29.896 --> 00:07:31.097
So it's like, what's going on?

149
00:07:31.838 --> 00:07:32.979
So we want to figure out how it is.

150
00:07:33.239 --> 00:07:35.801
The first clue comes from a guy named Vitruvius.

151
00:07:36.202 --> 00:07:40.746
He actually wrote the first architecture book in the world back in 30 BC.

152
00:07:40.766 --> 00:07:42.567
It's called De Architectura.

153
00:07:42.587 --> 00:07:42.707
Yeah.

154
00:07:43.808 --> 00:07:47.091
And he described a way of mixing the concrete.

155
00:07:47.591 --> 00:07:51.875
So what you do is you take something called quick lime, which is just calcium oxide.

156
00:07:52.195 --> 00:07:55.758
That's something that you get when you take limestone, you heat it up.

157
00:07:56.118 --> 00:07:57.539
Limestone is calcium carbonate.

158
00:07:57.859 --> 00:08:00.501
That's like a sedimentary rock that you get from, you know,

159
00:08:01.782 --> 00:08:06.347
Millions of years of sedimentation from all of these ancient sea creatures with like seashells.

160
00:08:06.487 --> 00:08:06.647
Okay.

161
00:08:07.048 --> 00:08:09.490
You pack up a bunch of seashells, you get limestone.

162
00:08:09.730 --> 00:08:13.734
You heat it up, you get quick lime, which is CAO, calcium oxide.

163
00:08:13.854 --> 00:08:14.095
Okay.

164
00:08:14.835 --> 00:08:15.096
Okay.

165
00:08:15.596 --> 00:08:21.362
And when you combine that in excess water over months, you get some stable form of lime putty.

166
00:08:21.682 --> 00:08:24.405
And that's the process that he described earlier.

167
00:08:25.198 --> 00:08:27.921
in his book, De Architectura.

168
00:08:28.141 --> 00:08:28.421
Okay.

169
00:08:28.481 --> 00:08:28.701
Yeah.

170
00:08:28.781 --> 00:08:28.981
Got it.

171
00:08:29.002 --> 00:08:29.182
Okay.

172
00:08:29.722 --> 00:08:41.673
So the defects that they found, whenever you look at this Roman concrete in like super detail, you'll find these millimeter grain to centimeter grain defects of lime.

173
00:08:42.053 --> 00:08:42.894
It's like full lime.

174
00:08:42.914 --> 00:08:44.415
So it's like not fully mixed in.

175
00:08:44.615 --> 00:08:44.835
Okay.

176
00:08:45.036 --> 00:08:45.256
Okay.

177
00:08:45.916 --> 00:08:46.076
And...

178
00:08:47.198 --> 00:08:52.465
It was just thought that all of this lime that's not fully mixed in is kind of a defect.

179
00:08:52.985 --> 00:08:54.027
Okay?

180
00:08:54.347 --> 00:08:57.851
Then in 2023, a paper from MIT comes out.

181
00:08:58.072 --> 00:08:58.272
Okay?

182
00:08:58.292 --> 00:08:59.573
This is from the Masich Lab.

183
00:09:00.515 --> 00:09:03.498
And this describes a new form of mixing.

184
00:09:03.578 --> 00:09:04.780
This is hot mixing.

185
00:09:05.140 --> 00:09:06.141
So in the previous one...

186
00:09:06.762 --> 00:09:09.783
In Vitruvius, in his document, that's 2,000 years old.

187
00:09:10.203 --> 00:09:12.003
I mean, the guy was there, presumably.

188
00:09:12.043 --> 00:09:15.224
So he's describing how he saw the concrete being made.

189
00:09:15.484 --> 00:09:16.625
He said it's cold mixing.

190
00:09:16.865 --> 00:09:18.365
This calcium oxide.

191
00:09:18.645 --> 00:09:23.847
Yeah, which is you get the calcium oxide, you mix it with water, and then you mix the other stuff.

192
00:09:24.387 --> 00:09:24.627
Okay?

193
00:09:25.247 --> 00:09:30.992
What the MIT guys are saying is, no, actually what we do is we get the lime class.

194
00:09:31.092 --> 00:09:31.772
We get that lime.

195
00:09:31.792 --> 00:09:33.073
We don't mix it with water first.

196
00:09:33.133 --> 00:09:39.118
We mix it with volcanic ash and all of these other ingredients, and then we put in the concrete.

197
00:09:39.378 --> 00:09:41.680
And that order is very important.

198
00:09:41.700 --> 00:09:42.440
Order of operations.

199
00:09:42.581 --> 00:09:47.024
Yes, because I told you in chemistry, chemistry, this is basically materials chemistry, right?

200
00:09:47.064 --> 00:09:47.845
Yeah.

201
00:09:47.905 --> 00:09:48.545
It's like cooking.

202
00:09:48.665 --> 00:09:50.807
The order of operations is very, very important.

203
00:09:51.007 --> 00:09:51.127
Yes.

204
00:09:51.147 --> 00:09:51.307
Okay.

205
00:09:51.467 --> 00:09:51.648
Yes.

206
00:09:51.748 --> 00:09:52.008
So –

207
00:09:53.453 --> 00:10:02.496
What the MIT 2023 paper says is that we're going to do hot mixing, which is mix the lime with the volcanic ash, mix it together, and then put in the water.

208
00:10:02.516 --> 00:10:02.816
Okay?

209
00:10:02.836 --> 00:10:03.036
Okay.

210
00:10:03.516 --> 00:10:11.739
And then these chunks of lime, they're going to be the reason why the Roman concrete actually can self-heal.

211
00:10:11.959 --> 00:10:12.239
Okay.

212
00:10:12.419 --> 00:10:12.659
Okay?

213
00:10:13.100 --> 00:10:14.040
It's by design.

214
00:10:14.180 --> 00:10:14.460
Okay.

215
00:10:14.520 --> 00:10:15.300
It's not a defect.

216
00:10:15.420 --> 00:10:15.680
Okay.

217
00:10:15.961 --> 00:10:16.221
Okay?

218
00:10:16.361 --> 00:10:16.621
Okay.

219
00:10:17.701 --> 00:10:18.922
So the central problem.

220
00:10:19.322 --> 00:10:19.502
Right.

221
00:10:19.642 --> 00:10:20.082
Who's right?

222
00:10:20.717 --> 00:10:20.917
Yeah.

223
00:10:21.457 --> 00:10:21.657
Right?

224
00:10:21.817 --> 00:10:21.998
Right.

225
00:10:22.038 --> 00:10:22.938
Hot or cold.

226
00:10:23.038 --> 00:10:24.119
Like, who is right?

227
00:10:24.239 --> 00:10:26.760
Who's the guy who actually can do it?

228
00:10:28.420 --> 00:10:30.921
Like, is it the hot mixing or is it the cold mixing?

229
00:10:31.662 --> 00:10:31.862
Right?

230
00:10:32.562 --> 00:10:33.182
Can I take a guess?

231
00:10:33.583 --> 00:10:33.783
Yeah.

232
00:10:35.984 --> 00:10:36.424
Actually, no.

233
00:10:36.464 --> 00:10:36.744
Go ahead.

234
00:10:37.444 --> 00:10:38.205
It's the MIT guy.

235
00:10:39.225 --> 00:10:39.725
It turns out.

236
00:10:39.845 --> 00:10:42.346
It turns out the original author 2,000 years ago.

237
00:10:42.366 --> 00:10:44.187
Right.

238
00:10:47.433 --> 00:10:51.416
I think got it wrong, but there might be a good reason for it, and we'll get to that at the end.

239
00:10:51.476 --> 00:10:51.716
Okay.

240
00:10:51.736 --> 00:10:51.956
Okay?

241
00:10:52.136 --> 00:10:52.397
Okay.

242
00:10:52.517 --> 00:11:06.007
So the paper comes out last month in Nature Communications, photo number 12, and this is the paper, an unfinished Pompeii construction site.

243
00:11:06.987 --> 00:11:07.107
Okay?

244
00:11:07.128 --> 00:11:07.308
Okay.

245
00:11:07.508 --> 00:11:07.728
Okay.

246
00:11:09.134 --> 00:11:09.695
That's the key.

247
00:11:10.035 --> 00:11:10.375
Okay.

248
00:11:10.435 --> 00:11:11.896
Pompeii is the key.

249
00:11:12.076 --> 00:11:12.317
Okay.

250
00:11:12.457 --> 00:11:12.697
Okay?

251
00:11:13.177 --> 00:11:17.181
I went to Pompeii, I think it was two years ago.

252
00:11:17.701 --> 00:11:19.543
Actually, I did go to Pompeii.

253
00:11:19.583 --> 00:11:20.624
Oh, you did go to Pompeii.

254
00:11:20.704 --> 00:11:21.504
Isn't it incredible?

255
00:11:21.564 --> 00:11:21.765
Yeah.

256
00:11:21.865 --> 00:11:28.810
Pompeii is an intact Roman city that has been buried...

257
00:11:30.237 --> 00:11:32.037
Under a volcanic eruption.

258
00:11:32.197 --> 00:11:35.358
Mount Vesuvius erupted in 79 AD.

259
00:11:36.458 --> 00:11:40.699
And it buried an entire volcanic city in moments.

260
00:11:41.859 --> 00:11:46.000
And just like, you know, within a few hours.

261
00:11:46.580 --> 00:11:54.182
And when you say city, it's, you know, particularly for the time, it is so unbelievably large.

262
00:11:54.262 --> 00:11:54.502
Yeah.

263
00:11:55.082 --> 00:11:57.643
And organized and choreographed.

264
00:11:57.703 --> 00:11:57.903
Yeah.

265
00:11:57.943 --> 00:11:58.643
It's like hard...

266
00:11:59.897 --> 00:12:06.523
It was a really surreal and bizarre feeling being like, oh, that street goes all the way over there.

267
00:12:06.623 --> 00:12:07.124
Yeah.

268
00:12:07.144 --> 00:12:08.485
And this one goes all the way over here.

269
00:12:08.605 --> 00:12:09.586
And it's a grid system.

270
00:12:09.726 --> 00:12:10.467
It's really impressive.

271
00:12:10.507 --> 00:12:10.707
Yeah.

272
00:12:10.767 --> 00:12:12.208
And there's a shopping district.

273
00:12:12.228 --> 00:12:12.428
Yes.

274
00:12:12.669 --> 00:12:15.331
And there's like intact restaurants.

275
00:12:15.371 --> 00:12:15.852
Yes.

276
00:12:15.912 --> 00:12:20.035
And you can tell what the function was because it looks just like...

277
00:12:20.985 --> 00:12:21.325
Today.

278
00:12:21.445 --> 00:12:21.906
Yeah, yeah.

279
00:12:21.926 --> 00:12:22.647
There's a mall.

280
00:12:22.947 --> 00:12:23.107
Yeah.

281
00:12:23.487 --> 00:12:26.090
You know, it's insane when you think about it, right?

282
00:12:26.170 --> 00:12:26.410
It is.

283
00:12:26.710 --> 00:12:41.384
And the advantage of studying a site like Pompeii is that what you've got is a city that was 2,000 years old, one of the beach cities of the Roman Empire, and it's frozen in time.

284
00:12:42.165 --> 00:12:45.969
And just like every other city, there's going to be construction sites.

285
00:12:45.989 --> 00:12:46.169
Right.

286
00:12:48.123 --> 00:12:48.403
Okay.

287
00:12:48.624 --> 00:12:49.445
Okay.

288
00:12:49.665 --> 00:12:50.346
You see what's happening.

289
00:12:50.366 --> 00:12:56.134
So if there's a construction site, you can go and look and see how they were mixing the concrete.

290
00:12:56.174 --> 00:12:56.395
Yes.

291
00:12:56.815 --> 00:12:57.476
That's the key.

292
00:12:57.536 --> 00:12:57.776
Yes.

293
00:12:58.177 --> 00:12:59.619
We can see how the sausage was made.

294
00:12:59.779 --> 00:13:00.280
Exactly.

295
00:13:00.660 --> 00:13:04.426
Because much like in all time periods in human history, we're always building something.

296
00:13:04.446 --> 00:13:04.646
Yeah.

297
00:13:04.666 --> 00:13:04.786
Yeah.

298
00:13:05.727 --> 00:13:06.188
Exactly.

299
00:13:06.288 --> 00:13:11.232
And, you know, like they're renovating a building or they're building a new building or something like that.

300
00:13:11.273 --> 00:13:15.156
So they found a construction site and that's the focus of this paper.

301
00:13:15.256 --> 00:13:15.837
OK, got it.

302
00:13:15.877 --> 00:13:21.542
OK, they found an active construction site and what they found was the dry pre-mixed piles.

303
00:13:22.163 --> 00:13:29.190
So not lime that's been sitting in water and cold mixing, but actually lime that's been put in.

304
00:13:29.870 --> 00:13:31.231
And mixed with volcanic ash.

305
00:13:31.551 --> 00:13:31.811
Okay.

306
00:13:31.831 --> 00:13:33.672
And then it's waiting to be mixed with water.

307
00:13:33.792 --> 00:13:34.312
Ah, yeah, okay.

308
00:13:34.372 --> 00:13:34.592
Okay.

309
00:13:34.612 --> 00:13:35.272
Got it, got it, got it.

310
00:13:35.572 --> 00:13:35.812
Got it.

311
00:13:35.893 --> 00:13:40.014
And the reason why this matters chemically, right?

312
00:13:40.414 --> 00:13:44.696
We want to talk about, like, what's the difference between the hot mixing and the cold mixing?

313
00:13:44.756 --> 00:13:48.978
Yeah, like, you know, and I can kind of maybe into it a little bit, but it seems to be really fundamental.

314
00:13:49.158 --> 00:13:49.678
Yes, it is.

315
00:13:49.778 --> 00:13:50.759
It is very fundamental.

316
00:13:51.039 --> 00:13:56.061
So the idea is whenever water reacts with calcium oxide...

317
00:13:56.941 --> 00:13:58.902
It gives off a lot of heat.

318
00:13:59.442 --> 00:13:59.703
Okay.

319
00:13:59.983 --> 00:14:00.243
Okay?

320
00:14:01.063 --> 00:14:02.264
That's the hot mixing part.

321
00:14:02.424 --> 00:14:02.664
Okay.

322
00:14:02.824 --> 00:14:03.044
Okay?

323
00:14:03.465 --> 00:14:15.311
Now, if you do it the way Vitruvius said all those years ago in that book, then you take the lime, you mix it with the water, and because it takes such a long time, the heat is just going to dissipate into the water.

324
00:14:15.411 --> 00:14:16.692
And so that's the cold mixing.

325
00:14:16.992 --> 00:14:17.112
Okay.

326
00:14:17.272 --> 00:14:20.054
If you do it when it's hot, right?

327
00:14:20.074 --> 00:14:21.935
If you do it when it's hot, then...

328
00:14:23.953 --> 00:14:28.796
you've got a mixture of the lime and the volcanic ash that you're mixing together.

329
00:14:29.937 --> 00:14:32.138
That's going to give off a lot of heat.

330
00:14:32.439 --> 00:14:35.661
It's going to reach temperatures nearing 200 degrees Celsius.

331
00:14:36.461 --> 00:14:36.681
Okay?

332
00:14:37.162 --> 00:14:45.907
And in that chemical reaction, what's going to end up happening is it's going to sort of plaster everything together, and that's actually going to preserve chunks of lime

333
00:14:47.227 --> 00:14:51.031
Before they fully dissolve into the rest of the mixture.

334
00:14:51.111 --> 00:14:51.452
I see.

335
00:14:51.732 --> 00:14:52.013
I see.

336
00:14:52.033 --> 00:15:01.423
So there's sort of this timing function where there's a window where it can solidify prior to distribution.

337
00:15:01.603 --> 00:15:01.884
Yes.

338
00:15:02.324 --> 00:15:04.046
But only at certain temperatures.

339
00:15:04.447 --> 00:15:04.647
Yes.

340
00:15:04.867 --> 00:15:05.167
Got it.

341
00:15:05.327 --> 00:15:05.848
Exactly.

342
00:15:06.028 --> 00:15:06.468
Yeah.

343
00:15:06.588 --> 00:15:15.553
And because it's happening so fast, the chunks of lime don't have enough time to really diffuse and become part of a homogenous mixture.

344
00:15:15.573 --> 00:15:15.793
I see.

345
00:15:15.893 --> 00:15:20.795
And so having these grains of lime in your concrete is actually a good idea.

346
00:15:21.436 --> 00:15:21.876
I see.

347
00:15:22.196 --> 00:15:22.496
Okay?

348
00:15:22.616 --> 00:15:22.916
Okay.

349
00:15:23.697 --> 00:15:25.178
And the reason for that— I was going to say—

350
00:15:26.960 --> 00:15:30.645
And the reason for that, we're going to sort of get into a little bit later, right?

351
00:15:30.805 --> 00:15:31.066
Okay.

352
00:15:31.126 --> 00:15:35.152
That's actually key to the self-reporting.

353
00:15:36.448 --> 00:15:57.603
healing process this kind of random distribution of this lime inside the mixture that solidifies in this case at high temperature prior to it all kind of getting distributed like into a muck allow has some derivative effects that create uh favorable outcomes exactly from a construction perspective

354
00:15:57.703 --> 00:15:59.763
Yeah, from a construction perspective, right?

355
00:16:00.164 --> 00:16:03.684
So they found these dry mix piles and they're like, okay, this is great.

356
00:16:03.704 --> 00:16:09.125
That means that we were kind of right in that there's lime mixed with volcanic ash and it's dry.

357
00:16:09.465 --> 00:16:12.806
It's not in the form of the concrete, okay?

358
00:16:12.926 --> 00:16:17.427
Let's look at some of the smoking gun microstructure that they found, okay?

359
00:16:17.707 --> 00:16:21.548
When they looked at the unfinished walls and then they looked at the raw material—

360
00:16:22.621 --> 00:16:37.737
What you see is this really brittle microstructure where you're getting chunks of lime, you're getting microscopic bits of volcanic ash, and the two are exactly the same, meaning the raw material and the unfinished walls are...

361
00:16:38.666 --> 00:16:40.267
The walls that have just been created.

362
00:16:40.287 --> 00:16:40.547
Right.

363
00:16:40.948 --> 00:16:41.148
Right?

364
00:16:41.308 --> 00:16:43.629
Unfinished means there's no plaster, which means it was just done.

365
00:16:43.749 --> 00:16:43.950
Yes.

366
00:16:44.610 --> 00:16:47.512
Those have the same material properties.

367
00:16:47.872 --> 00:16:53.436
So this is, again, helping to verify the hypothesis that it is unfinished.

368
00:16:53.496 --> 00:16:53.716
Yes.

369
00:16:53.856 --> 00:16:57.719
Because when you look at it, when it's really small, the structure of it matches.

370
00:16:58.099 --> 00:17:01.601
Because if it was finished, that microstructure would look different.

371
00:17:01.701 --> 00:17:01.922
Yeah.

372
00:17:02.102 --> 00:17:02.322
Well...

373
00:17:03.023 --> 00:17:03.504
No, hold on.

374
00:17:03.564 --> 00:17:04.706
Okay.

375
00:17:05.087 --> 00:17:08.935
The fact that it's an unfinished wall means that it's very recently been made.

376
00:17:09.716 --> 00:17:09.957
Gotcha.

377
00:17:10.017 --> 00:17:10.258
Okay?

378
00:17:10.278 --> 00:17:10.899
Yes.

379
00:17:11.079 --> 00:17:13.684
And so what that means is that that construction...

380
00:17:15.180 --> 00:17:16.501
Happened from this pile.

381
00:17:17.342 --> 00:17:17.882
I see what you're saying.

382
00:17:17.902 --> 00:17:18.623
You see what I'm saying?

383
00:17:18.683 --> 00:17:23.407
It didn't have like... Because over time... Yeah, because finished means that they put a plaster on it.

384
00:17:23.467 --> 00:17:25.329
And then like now the wall is smooth.

385
00:17:25.549 --> 00:17:28.872
Like if you've been inside the Pompeii buildings, there's like paintings and stuff like that.

386
00:17:29.132 --> 00:17:30.914
This is a new wall, right?

387
00:17:30.974 --> 00:17:32.475
And the raw material is right there.

388
00:17:32.875 --> 00:17:35.417
So you can connect those two and be like this...

389
00:17:36.118 --> 00:17:37.279
Was going to go become that.

390
00:17:37.359 --> 00:17:38.019
Yeah, I gotcha.

391
00:17:38.159 --> 00:17:38.619
You know what I mean?

392
00:17:38.639 --> 00:17:38.779
Yes.

393
00:17:39.240 --> 00:17:40.380
And it's right next to each other.

394
00:17:40.420 --> 00:17:40.580
Yes.

395
00:17:40.861 --> 00:17:45.063
And because the microstructure is exactly the same, you can say that actually, yeah, they did dry mix.

396
00:17:45.163 --> 00:17:46.724
Yeah.

397
00:17:47.184 --> 00:17:49.385
And then do the hot mixing and then make that wall.

398
00:17:49.966 --> 00:17:50.126
Right?

399
00:17:50.246 --> 00:17:50.946
That's the idea.

400
00:17:51.466 --> 00:17:53.888
Got it.

401
00:17:53.948 --> 00:17:54.168
Okay.

402
00:17:54.308 --> 00:18:01.352
The other really cool technique that I thought they used was something called stable isotope mass spectroscopy.

403
00:18:01.572 --> 00:18:01.812
Okay?

404
00:18:02.132 --> 00:18:02.692
Here's the idea.

405
00:18:03.013 --> 00:18:03.273
So...

406
00:18:04.104 --> 00:18:06.586
You know, in nature, you've got isotopes.

407
00:18:07.467 --> 00:18:11.691
Isotopes are elements that have different numbers of neutrons versus protons.

408
00:18:12.491 --> 00:18:15.154
And so they're going to be heavier.

409
00:18:15.454 --> 00:18:20.198
Like, for example, oxygen 18 versus oxygen 16.

410
00:18:21.299 --> 00:18:23.842
Same chemical properties, but it's just heavier.

411
00:18:23.862 --> 00:18:24.322
Okay.

412
00:18:24.522 --> 00:18:29.547
Which means if something gets hot, right, the heavier oxygen is going to move slower.

413
00:18:29.567 --> 00:18:29.967
Okay.

414
00:18:30.584 --> 00:18:38.631
And so in any reaction, there's going to be more lighter oxygen that's going to do the reaction versus the heavier stuff.

415
00:18:38.671 --> 00:18:38.892
Right.

416
00:18:39.112 --> 00:18:40.293
Because it's moving faster.

417
00:18:40.333 --> 00:18:41.934
The lighter oxygen is moving faster.

418
00:18:42.275 --> 00:18:53.565
So you can actually calculate, you can do an experiment, and you can figure out from mass spectroscopy how much of each isotope is in the substance that you're looking at.

419
00:18:53.965 --> 00:18:54.225
Got it.

420
00:18:54.546 --> 00:18:55.106
Does that make sense?

421
00:18:55.126 --> 00:18:55.426
It does.

422
00:18:55.566 --> 00:18:55.847
It does.

423
00:18:56.087 --> 00:18:56.267
Right.

424
00:18:56.567 --> 00:18:57.468
And so that's what they did.

425
00:18:57.568 --> 00:19:03.052
What they looked at was how much of carbon 13 there is and how much oxygen 18 there is.

426
00:19:04.033 --> 00:19:16.043
And what they found was that the slope of these lines, if you look at photo 16, right, it's characteristic of that water limited high temperature fractionation.

427
00:19:16.443 --> 00:19:18.004
Oh, I see what you're saying here.

428
00:19:18.064 --> 00:19:21.927
So the gray triangles are what they found.

429
00:19:22.388 --> 00:19:22.568
Mm-hmm.

430
00:19:23.696 --> 00:19:25.037
At the site in Pompeii.

431
00:19:25.837 --> 00:19:30.841
And what you're looking at, the red box, is where most of those gray triangles are.

432
00:19:31.181 --> 00:19:33.602
And that's exactly what would happen if you pre-mixed.

433
00:19:34.303 --> 00:19:35.643
And then you put water.

434
00:19:35.664 --> 00:19:36.504
That's that quick lime.

435
00:19:36.604 --> 00:19:37.865
Yeah, that's that quick lime.

436
00:19:37.985 --> 00:19:38.445
Exactly.

437
00:19:38.865 --> 00:19:41.987
And so the slaked lime samples, that's the cold mixing.

438
00:19:42.328 --> 00:19:44.609
That's the stuff that you want to use for finishing.

439
00:19:45.089 --> 00:19:50.092
Because it's smooth, and you can plaster it onto the wall, and then you can put mosaics and light paintings.

440
00:19:50.112 --> 00:19:50.933
That's one of the things stuff.

441
00:19:51.253 --> 00:20:13.444
and the ornate stuff but if you want the structure you want you want the concrete that actually self-heals you actually want to do it with the hot stuff this is so like and so it's like there there was also this understanding of the dual use at these difference at these different that's what's insane that that's really that's really cool and and here's here's actually like what's what's sort of happening with the the quicklime

442
00:20:13.764 --> 00:20:14.024
Okay.

443
00:20:14.325 --> 00:20:18.108
And I haven't gotten into how exactly it would work, right?

444
00:20:18.448 --> 00:20:22.772
Like, why would these chunks of lime heal concrete?

445
00:20:23.093 --> 00:20:23.293
Mm-hmm.

446
00:20:23.733 --> 00:20:23.953
Right?

447
00:20:24.173 --> 00:20:24.314
Mm-hmm.

448
00:20:24.374 --> 00:20:25.054
Here's what's happening.

449
00:20:25.675 --> 00:20:28.117
So, suppose you've got concrete and you've got a crack.

450
00:20:28.858 --> 00:20:29.098
Okay.

451
00:20:29.238 --> 00:20:31.100
Now, you've got a crack in the concrete.

452
00:20:31.120 --> 00:20:31.400
Okay.

453
00:20:32.227 --> 00:20:35.170
There's going to be water that goes in into this concrete.

454
00:20:35.290 --> 00:20:40.114
If it's normal concrete, that water is going to push out more and push out more and degrade the whole thing.

455
00:20:40.134 --> 00:20:43.498
But if you have chunks of lime, calcium oxide, right?

456
00:20:43.518 --> 00:20:43.678
Yes.

457
00:20:43.798 --> 00:20:49.783
That's going to dissolve into the concrete and fill, sorry, dissolve with the water and fill the crack.

458
00:20:50.680 --> 00:20:55.762
And become a kind of putty that like seals that crack over time.

459
00:20:55.922 --> 00:20:57.463
It's like an extended release.

460
00:20:57.863 --> 00:20:58.403
Exactly.

461
00:20:58.783 --> 00:21:00.144
So you have these like pellets.

462
00:21:00.244 --> 00:21:03.645
You have these pellets of like glue that get activated.

463
00:21:03.865 --> 00:21:04.045
Right.

464
00:21:04.245 --> 00:21:04.765
Only when.

465
00:21:04.865 --> 00:21:08.447
Only when there's a crack and the water goes in and then it seals that crack.

466
00:21:08.647 --> 00:21:09.407
That's so.

467
00:21:11.208 --> 00:21:13.089
What's so amazing about that is.

468
00:21:15.243 --> 00:21:17.404
They clearly had intuited that.

469
00:21:17.444 --> 00:21:18.105
It's insane.

470
00:21:18.805 --> 00:21:32.617
And what could be perceived as imperfection initially actually had these unintended consequences that actually made the underlying structure more robust.

471
00:21:33.057 --> 00:21:38.762
And just like the open-mindedness to think about stuff and try things to get to that level.

472
00:21:39.507 --> 00:21:41.888
Because in this day and age, we'd be like, oh, no, it's not.

473
00:21:41.928 --> 00:21:42.508
The mixture is.

474
00:21:42.809 --> 00:21:44.129
Yeah.

475
00:21:44.209 --> 00:21:46.110
And that's what we assumed for the longest time.

476
00:21:46.150 --> 00:21:49.192
We assumed that, like, this Roman mixture was just, like, not good.

477
00:21:49.212 --> 00:21:49.472
Good.

478
00:21:49.872 --> 00:21:50.152
You know?

479
00:21:50.692 --> 00:21:54.694
But, no, it might have been by design that they did this.

480
00:21:55.195 --> 00:21:58.536
And it's, dude, it's so insane.

481
00:21:58.576 --> 00:22:00.177
And this actually makes sense, right?

482
00:22:00.197 --> 00:22:01.638
Because Roman-inspired concrete...

483
00:22:02.338 --> 00:22:10.803
This could be a huge thing for us environmentally because Roman-inspired concrete, this is the Pantheon again, it's lasted for thousands of years.

484
00:22:11.703 --> 00:22:23.410
So even though in the process of creating this concrete, it might use more CO2 than, let's say, the more environmentally friendly concrete, the biggest carbon footprint...

485
00:22:24.300 --> 00:22:33.243
When building something or the biggest carbon footprint of a building is the process of building it, not the materials themselves.

486
00:22:33.983 --> 00:22:40.945
If you build it once and it lasts for 2000 years, that building is the greenest building on the planet.

487
00:22:41.345 --> 00:22:50.288
I would argue that the Pantheon, which is still in use, is the most environmentally friendly building because it was built once and then it's been used for 2000 years.

488
00:22:50.328 --> 00:22:50.928
Right, right.

489
00:22:51.228 --> 00:22:51.448
Right.

490
00:22:51.568 --> 00:22:51.948
Yeah.

491
00:22:52.128 --> 00:22:52.388
Yeah.

492
00:22:52.789 --> 00:22:58.830
It, it, it's, it's really hard to, because also the scale of it.

493
00:22:59.311 --> 00:22:59.571
Right.

494
00:23:00.691 --> 00:23:06.013
And I mean, it rained for three days here and everything's falling apart.

495
00:23:06.153 --> 00:23:06.673
Oh my gosh.

496
00:23:06.953 --> 00:23:07.173
Yeah.

497
00:23:07.253 --> 00:23:07.934
Yeah, yeah, yeah.

498
00:23:07.954 --> 00:23:08.895
You know what I mean?

499
00:23:08.915 --> 00:23:09.435
Yeah, it was.

500
00:23:10.196 --> 00:23:11.938
And there's potholes everywhere.

501
00:23:12.098 --> 00:23:13.639
It's just the worst, right?

502
00:23:13.980 --> 00:23:18.144
So it would be really cool if we could actually start replicating this ancient technology.

503
00:23:18.244 --> 00:23:18.844
Right.

504
00:23:18.944 --> 00:23:20.025
And then start using it.

505
00:23:20.386 --> 00:23:28.614
And so Admir Masic, who's the lead author of this study, he actually does have a startup called DMAT.

506
00:23:29.660 --> 00:23:35.586
that is trying to make sustainable and durable concrete solutions inspired by ancient Roman concrete.

507
00:23:35.886 --> 00:23:36.527
That's fantastic.

508
00:23:36.627 --> 00:23:44.814
Okay, so he's using this knowledge that he's gaining from all the research that he's doing on ancient Rome to commercialize eco-concrete.

509
00:23:45.734 --> 00:23:50.096
And they've actually got pilot projects in the US and in Europe right now.

510
00:23:50.336 --> 00:23:51.576
Shout out, Admir.

511
00:23:52.037 --> 00:23:52.477
Well done.

512
00:23:52.717 --> 00:23:53.397
Yeah.

513
00:23:53.757 --> 00:23:54.158
Well done.

514
00:23:54.178 --> 00:23:56.398
When everyone thought it was nothing to see here.

515
00:23:56.539 --> 00:23:57.059
Yeah, yeah.

516
00:23:57.139 --> 00:23:58.559
And he kept at it, right?

517
00:23:58.839 --> 00:24:00.740
And he proved Vitruvius wrong.

518
00:24:00.760 --> 00:24:03.762
Now, there might be a good reason that Vitruvius was wrong, okay?

519
00:24:04.062 --> 00:24:10.384
And it's because Vitruvius actually wrote his architecture textbook in like 30 BC.

520
00:24:12.384 --> 00:24:18.426
But the architectural revolution that we talk about, the Pantheon, Pont du Gard, and all that, that happened about 100 years after.

521
00:24:18.886 --> 00:24:22.888
So it could be that in those 100 years, 100 years is a long time, right?

522
00:24:23.008 --> 00:24:24.088
Even in ancient Rome.

523
00:24:24.148 --> 00:24:30.730
So they could have actually figured out this new technique of hot mixing after he wrote that textbook.

524
00:24:30.810 --> 00:24:31.010
Fair.

525
00:24:31.351 --> 00:24:33.371
And, you know, given how fragile...

526
00:24:35.552 --> 00:24:59.045
any textbooks are maybe somebody else wrote another textbook but it's just been lost to the agents didn't make it right so that's probably one of the reasons the other reason the other reason could be that it was just like translated wrong from like ancient yeah you know roman whatever latin i guess they were using yeah um so there's a lot of reasons but this is a pretty good like nail in the coffin and in the

527
00:25:01.088 --> 00:25:27.029
cold mixing vitruvian theory theory you know yeah right right and like we're pretty sure we're right that it is hot mixing and and the the the group at mit has actually replicated that the the architecture right and they've actually started a whole company with this new type of concrete so it's clearly working i was gonna say the way the way you know something is working is when the uh the capital markets yeah get involved yeah that's when you know it's uh

528
00:25:27.805 --> 00:25:28.747
It's transcended.

529
00:25:29.388 --> 00:25:30.630
That's fascinating.

530
00:25:30.650 --> 00:25:34.236
I remember walking around in Pompeii and just being like, this is crazy.

531
00:25:34.296 --> 00:25:35.258
Yeah, dude.

532
00:25:35.358 --> 00:25:37.261
I was so impressed with Pompeii.

533
00:25:37.462 --> 00:25:39.565
And this makes sense.

534
00:25:40.886 --> 00:25:46.111
I know there's always a lot of memes about, you know, the story about the Romans made concrete that we can't figure out.

535
00:25:46.512 --> 00:25:52.758
And one of the things about this story that's perseverance is really such a fundamental aspect of like scientific research.

536
00:25:53.159 --> 00:25:59.125
Because one, like you're constantly battling just to maintain like to maintain funding, to continue to do the search.

537
00:25:59.665 --> 00:26:08.294
And like people give up after a while if you can't meet some kind of quarterly annual whatever metric that is independent of just the curiosity of it.

538
00:26:08.435 --> 00:26:09.075
Yeah.

539
00:26:09.616 --> 00:26:10.597
So kudos to the team.

540
00:26:10.897 --> 00:26:14.141
It's always hard for us to give big ups to MIT.

541
00:26:14.181 --> 00:26:17.825
But I think U.S. college rankings had them at number one and us at number three.

542
00:26:17.945 --> 00:26:19.006
I think this is fake news.

543
00:26:19.246 --> 00:26:21.327
That's totally fake news.

544
00:26:21.407 --> 00:26:22.328
This is nonsense.

545
00:26:22.428 --> 00:26:22.688
Yeah.

546
00:26:22.788 --> 00:26:25.970
This is absolute nonsense.

547
00:26:26.050 --> 00:26:27.291
Tiger Nation number one.

548
00:26:27.311 --> 00:26:32.314
We're going to get to Tiger Nation, speaking of which, in our second story.

549
00:26:32.354 --> 00:26:47.704
But before we do so, we want to do our new segment, The Rundown, where we're just going to hit on a couple of other recent breaking research stories that have come out this week that we're not going to go deep into but are still fascinating nonetheless.

550
00:26:48.644 --> 00:26:57.871
And we are going to start with the moon rehearsal, which happened on December 20th in preparation for...

551
00:26:57.891 --> 00:27:00.633
This is the sort of Artemis II final dress rehearsal.

552
00:27:01.673 --> 00:27:09.299
NASA successfully completed a full launch day simulation for the first crewed lunar mission in half a century.

553
00:27:09.599 --> 00:27:11.561
So if you don't think we went to the moon before...

554
00:27:12.568 --> 00:27:14.369
We at least might go to the moon now.

555
00:27:14.429 --> 00:27:14.870
Let's see.

556
00:27:15.910 --> 00:27:18.692
This is a collaboration between both NASA and the Canadian Space Agency.

557
00:27:20.013 --> 00:27:25.557
We have astronauts Christina Koch, Victor Glover, Jeremy Hansen, and Reed Weissman.

558
00:27:25.998 --> 00:27:29.560
It's going to be a 10-day mission, which is super exciting.

559
00:27:29.860 --> 00:27:32.742
And they're taking off, I believe it is February 5th.

560
00:27:32.802 --> 00:27:33.463
So it's coming soon.

561
00:27:33.723 --> 00:27:34.664
Yeah, it's going to be great.

562
00:27:34.704 --> 00:27:36.185
It's basically like Apollo 8.

563
00:27:36.305 --> 00:27:39.007
They're not landing on the moon, but...

564
00:27:40.784 --> 00:27:46.549
They're not landing on the moon, but they're actually going to be going around the moon and then coming right back.

565
00:27:47.030 --> 00:27:54.897
And one of the coolest things about this is it's going to be the farthest that humans have ever been from planet Earth.

566
00:27:55.577 --> 00:27:55.918
Really?

567
00:27:55.958 --> 00:27:56.238
Yes.

568
00:27:56.598 --> 00:28:00.462
Because even in the Apollo missions, they went around the moon, but...

569
00:28:01.943 --> 00:28:18.552
The guy who stayed in the lunar capsule, or sorry, in the command module, rather than the two guys who went down in the lunar module, the guy who stayed in the command module, Michael Collins in Apollo 11, for example, was pretty close to the surface of the moon on the other side.

570
00:28:18.572 --> 00:28:19.912
He was on the far side on his own.

571
00:28:19.932 --> 00:28:20.613
Yeah.

572
00:28:21.985 --> 00:28:27.046
With these guys, these guys are actually overshooting, and they're going to go really far out around the back.

573
00:28:27.066 --> 00:28:28.047
They'll catch a wider orbit.

574
00:28:28.067 --> 00:28:28.747
Yeah, yeah.

575
00:28:28.887 --> 00:28:32.768
So they're going to be the farthest humans ever from Earth.

576
00:28:33.128 --> 00:28:36.409
And it's going to be—I'm really excited to see the photos that they get.

577
00:28:36.489 --> 00:28:43.591
It would be cool to see, like, are they going to be so far that they can get the moon and the Earth in the same shot?

578
00:28:43.991 --> 00:28:45.951
It's the pale blue dot V2.

579
00:28:46.151 --> 00:28:46.872
That would be insane.

580
00:28:46.892 --> 00:28:47.992
That would be really, really cool.

581
00:28:48.132 --> 00:28:55.657
Because we don't really have a photo like that of our family, like Earth and the moon together like that.

582
00:28:56.137 --> 00:28:56.657
Actually, we do.

583
00:28:56.697 --> 00:28:57.217
That's not true.

584
00:28:57.257 --> 00:29:04.502
We do have that from there's this Japanese satellite that's quite far away, right in between the sun and the Earth.

585
00:29:04.522 --> 00:29:04.822
Okay.

586
00:29:05.442 --> 00:29:08.984
And every once in a while when the moon goes in front of the Earth, it gets a photo.

587
00:29:09.005 --> 00:29:09.485
It doesn't count.

588
00:29:09.505 --> 00:29:10.265
There's no people in there.

589
00:29:10.365 --> 00:29:11.086
Yeah, exactly.

590
00:29:11.106 --> 00:29:14.088
This would be like a photo taken by a human being.

591
00:29:14.108 --> 00:29:14.268
It's us.

592
00:29:14.288 --> 00:29:14.788
It's us.

593
00:29:14.848 --> 00:29:15.689
That would be really cool.

594
00:29:15.749 --> 00:29:17.972
Not the fruit of our loins, which is technology.

595
00:29:18.012 --> 00:29:18.252
Yeah.

596
00:29:19.193 --> 00:29:19.393
Yeah.

597
00:29:19.914 --> 00:29:21.696
So we'll be on the lookout for Artemis 2.

598
00:29:21.756 --> 00:29:24.539
We actually have Artemis behind us.

599
00:29:25.780 --> 00:29:26.882
Over one of our shoulders.

600
00:29:26.962 --> 00:29:28.023
Oh, over the other shoulder.

601
00:29:28.724 --> 00:29:30.385
Oh, it's this camera now.

602
00:29:30.506 --> 00:29:32.648
We're still getting used to the three cameras set up here.

603
00:29:32.668 --> 00:29:33.209
Yeah.

604
00:29:33.789 --> 00:29:37.512
But it took me not a reasonable amount of time to build.

605
00:29:38.253 --> 00:29:39.694
So that's our story number one in the rundown.

606
00:29:39.734 --> 00:29:44.098
Number two is the cellular cleanup crew.

607
00:29:44.658 --> 00:29:47.240
This came out January 7th.

608
00:29:47.781 --> 00:29:50.023
It's about iDigs, iDags,

609
00:29:50.683 --> 00:29:55.867
which are deleting disease-causing proteins in nature chemistry.

610
00:29:56.267 --> 00:30:05.173
So scientists have found a way to supercharge our cells, natural recycling systems, to completely destroy cancer-shielding enzymes.

611
00:30:05.213 --> 00:30:14.819
And we've talked about T-Rags and the whole immune system and the immune response in our past episodes in Season 1.

612
00:30:15.039 --> 00:30:16.741
Some really big deep guys that are great on this.

613
00:30:16.761 --> 00:30:18.362
That's what kind of caught my eye on this one.

614
00:30:18.402 --> 00:30:18.662
Yeah.

615
00:30:18.942 --> 00:30:22.447
So we have Max Planck Institute involved in this, led by George Winter.

616
00:30:22.988 --> 00:30:28.536
And the idea here is that most drugs today act as inhibitors that just like block, right?

617
00:30:28.677 --> 00:30:29.358
Yeah.

618
00:30:29.438 --> 00:30:30.379
They block the site.

619
00:30:30.740 --> 00:30:32.983
They don't really, you know, activate.

620
00:30:33.083 --> 00:30:33.264
But –

621
00:30:35.572 --> 00:30:41.736
This is now recognizing and destroying the targets entirely.

622
00:30:41.896 --> 00:30:45.357
Yeah, and it's using the body's own machinery.

623
00:30:45.438 --> 00:30:45.598
Right.

624
00:30:45.658 --> 00:30:46.818
That's, I think, what's key.

625
00:30:46.878 --> 00:30:47.719
Right.

626
00:30:47.799 --> 00:30:59.865
It's like it's basically priming these proteins and not actually, like, taking care of the problem themselves, but, like, tagging them for the body's own trash collectors, these E3 ligases, to do it themselves.

627
00:30:59.905 --> 00:31:02.126
And it's always good to just, like, have the body do it.

628
00:31:02.326 --> 00:31:02.486
Right.

629
00:31:02.646 --> 00:31:02.887
Because...

630
00:31:03.947 --> 00:31:05.588
The body has a whole bunch of dynamics.

631
00:31:05.628 --> 00:31:06.449
Let's talk about biology.

632
00:31:06.509 --> 00:31:07.890
There's all these variables.

633
00:31:07.970 --> 00:31:08.230
Yeah.

634
00:31:08.450 --> 00:31:09.471
There's a lot of variables.

635
00:31:09.731 --> 00:31:10.071
Yeah.

636
00:31:10.512 --> 00:31:13.634
If you can help the body rather than just do the task yourself, it's always better.

637
00:31:14.134 --> 00:31:14.314
Right?

638
00:31:14.514 --> 00:31:27.103
So the analogy was if standard drugs are like putting a lock on a door to keep a villain inside, IDEX are calling in a demolition crew to remove the entire room, making sure the threat is gone for good.

639
00:31:27.263 --> 00:31:27.423
Yep.

640
00:31:27.644 --> 00:31:29.625
Again, this is all happening like this is this week.

641
00:31:29.645 --> 00:31:30.325
This is this week.

642
00:31:32.326 --> 00:31:34.567
Third rundown, the chat CBT moment.

643
00:31:35.267 --> 00:31:44.549
But for sleep, Jan 6, Sleep FM, this new thing out of Stanford Medicine that is diagnosing 130 diseases while you sleep.

644
00:31:45.869 --> 00:31:49.310
It's hard to avoid AI in stories all the time because it kind of just is what it is.

645
00:31:49.710 --> 00:31:51.571
But this story is...

646
00:31:52.653 --> 00:31:53.213
Fascinating.

647
00:31:53.313 --> 00:32:03.038
Sleep FM was trained on 600,000 hours of physiological data to be able to spot subtle mismatches between your brain, your heart, and your breathing signals.

648
00:32:03.118 --> 00:32:03.378
Yeah.

649
00:32:03.538 --> 00:32:07.299
And from that, it can then make predictions about a variety of diseases.

650
00:32:07.420 --> 00:32:07.600
Yeah.

651
00:32:07.680 --> 00:32:12.642
And what really got to me was it only needs to analyze one night of sleep data from you.

652
00:32:13.659 --> 00:32:15.160
To figure something out, right?

653
00:32:15.260 --> 00:32:16.080
It's trained.

654
00:32:16.260 --> 00:32:22.843
So it's a pre-trained AI, kind of like the pre-trained transformer models that we use in LLMs and things like that.

655
00:32:23.224 --> 00:32:32.088
So this is an AI that's been pre-trained with, as you said, 600,000 hours of physiological data, but then all it needs to classify you is just one night.

656
00:32:33.028 --> 00:32:36.770
And then it'll tell you one of the 130 diseases that you could have.

657
00:32:37.631 --> 00:32:38.511
It's pretty insane.

658
00:32:39.452 --> 00:32:49.137
Being able to read biological data through like these like sort of derivative signals in such a short, limited amount of like you need such a limited amount.

659
00:32:49.157 --> 00:32:49.697
Yeah, yeah, yeah.

660
00:32:49.717 --> 00:32:56.621
You could just like go and go, you know, to a facility that will monitor all of these different things while you sleep for one night.

661
00:32:56.741 --> 00:32:57.361
And then that's it.

662
00:32:57.441 --> 00:32:57.921
And you got it.

663
00:32:58.081 --> 00:32:58.282
Yeah.

664
00:32:58.542 --> 00:32:58.722
Yeah.

665
00:32:59.403 --> 00:33:01.212
unbelievable access to information.

666
00:33:01.252 --> 00:33:02.277
What will we do about it?

667
00:33:02.357 --> 00:33:03.181
Tune in next week.

668
00:33:04.155 --> 00:33:08.456
Story number four is now an optics and nanotech story.

669
00:33:08.476 --> 00:33:11.616
This was today it actually came out.

670
00:33:11.636 --> 00:33:16.377
The Silica Breakthrough, high-tech lenses for common glass.

671
00:33:16.417 --> 00:33:18.898
This is out of Harvard, Federico Capasso.

672
00:33:19.198 --> 00:33:24.459
So traditionally, flat metal lenses require this exotic and expensive materials to bend light.

673
00:33:24.739 --> 00:33:28.280
You know, we know as you get smaller in scale, things become more expensive.

674
00:33:28.300 --> 00:33:30.080
We see this with chips and all these other things.

675
00:33:30.720 --> 00:33:44.908
And so they've sort of perfected this single clean path of light without having to use, like, they've perfected this ability to do what used to take massive scale and massive cost with basically off-the-shelf glass.

676
00:33:44.968 --> 00:33:45.889
Yeah, yeah.

677
00:33:46.029 --> 00:33:46.809
It's pretty insane.

678
00:33:47.190 --> 00:33:49.011
I mean, it's going to be way cheaper, right?

679
00:33:49.071 --> 00:33:50.972
Massive scale, lower cost.

680
00:33:52.361 --> 00:33:56.422
And we use Glass in a whole number of applications.

681
00:33:56.442 --> 00:34:12.946
And one of the key things for me that I really liked about this story was that you can use all of the engineering that we've developed to make chips and silicon semiconductor industry stuff we can now use to make this Glass, right?

682
00:34:12.966 --> 00:34:19.908
That's one of the reasons why it's so cheap is because we can leverage all of this infrastructure that we've already built for all of our computing.

683
00:34:19.928 --> 00:34:20.408
Yeah.

684
00:34:20.548 --> 00:34:22.149
And now we're making glass out of silica.

685
00:34:22.409 --> 00:34:28.653
It's always great when we don't have to rebuild our industrial base in order to be able to enable and access new technology.

686
00:34:29.133 --> 00:34:35.957
It's like discovering you can build a high-performance jet engine with the standard parts from a local hardware store, which is fascinating.

687
00:34:36.017 --> 00:34:44.482
Our last quick rundown story of the day before we get back into our main stories is this observance.

688
00:34:44.482 --> 00:34:50.324
Ability to watch plants breathing really, really, really well.

689
00:34:50.785 --> 00:34:53.165
This is in Environmental Science, January 7th.

690
00:34:53.446 --> 00:34:57.307
Stomata Insight, a real-time window into crop survival.

691
00:34:57.327 --> 00:35:02.049
So the idea behind this is people are trying to improve how we do certain aspects of agriculture.

692
00:35:02.349 --> 00:35:09.252
The better we can monitor and see stuff, the better we can do things to sort of impact outcomes from a crop perspective.

693
00:35:09.272 --> 00:35:09.592
Yeah.

694
00:35:09.652 --> 00:35:15.250
Yeah, it's like a methods paper that is like, you know, now that we have a way to measure

695
00:35:16.280 --> 00:35:22.485
the stomata opening and closing on the bottom of a leaf, we can figure out what are the genes that actually control these things.

696
00:35:22.965 --> 00:35:32.913
And then, well, can I, you know, CRISPR these genes to then make a drought resistant crop that doesn't open when it's, you know, too dry, things like that.

697
00:35:33.253 --> 00:35:44.382
This is a new imaging system that lets science watch the microscopic mouths of plants as they breed in high definition out of University of Illinois Urbana Champaign, Joseph D. Crawford and Andrew D.B.

698
00:35:44.442 --> 00:35:44.843
Leakey.

699
00:35:45.705 --> 00:35:47.246
Um, very cool.

700
00:35:47.346 --> 00:35:47.886
I love plants.

701
00:35:47.906 --> 00:35:51.768
I love the idea of like, you know, having plants around is always super fun and fantastic.

702
00:35:52.348 --> 00:35:53.869
And that's just a little brief rundown.

703
00:35:53.889 --> 00:35:59.371
We can't always cover every story, but we're trying to illustrate there's so much happening.

704
00:35:59.391 --> 00:36:03.293
There's so much going on that does actually translate into real world.

705
00:36:03.353 --> 00:36:04.653
It's not all just abstract.

706
00:36:04.913 --> 00:36:05.194
Yeah.

707
00:36:05.214 --> 00:36:07.254
It's not mathematicians kind of going back and forth.

708
00:36:07.435 --> 00:36:07.895
Exactly.

709
00:36:07.915 --> 00:36:08.715
About God knows what.

710
00:36:09.175 --> 00:36:15.378
But now my favorite, another story about the good old Princeton Tigers.

711
00:36:15.619 --> 00:36:16.920
Yes, the best.

712
00:36:17.481 --> 00:36:25.888
This is actually from the Princeton Neuroscience Institute, which is the building where I did my senior thesis back in the day.

713
00:36:25.908 --> 00:36:34.296
I was a part of the physics department, but my senior thesis, I was really interested in biological physics.

714
00:36:34.776 --> 00:36:41.462
And so there was a professor there who was interested in physics and how to sort of use physical physics.

715
00:36:42.203 --> 00:37:09.630
intuition to talk about the brain i was particularly focusing on the retina this is talking about much higher order abstraction um task demand like thought and and really abstract things but there is a very nice thread of physics and mathematics in this story and it is going to get kind of technical but we're in for the ride right like that's that's why you guys are here yes

716
00:37:11.208 --> 00:37:12.669
It's really cool.

717
00:37:13.610 --> 00:37:13.870
Okay.

718
00:37:14.411 --> 00:37:18.854
The core problem is something called compositionality.

719
00:37:19.355 --> 00:37:19.615
Okay.

720
00:37:20.436 --> 00:37:25.460
It's a key in biological intelligence and it's something that AI is actually not very good at.

721
00:37:25.900 --> 00:37:26.160
Okay.

722
00:37:26.481 --> 00:37:33.967
And something that the authors used was, you know, if you already know how to bake bread, you can use this ability to bake a cake.

723
00:37:35.533 --> 00:37:35.773
Why?

724
00:37:35.873 --> 00:37:46.180
Because you can actually take parts of your learning when you baked bread, you know, how to mix stuff, how to turn on an oven, how an oven works.

725
00:37:46.900 --> 00:37:53.444
And you can take those models together and then figure out, OK, with a cake, it's like not that much different.

726
00:37:53.484 --> 00:37:53.724
Fine.

727
00:37:53.744 --> 00:37:55.125
There's like a little bit of frosting.

728
00:37:55.445 --> 00:37:56.746
There's like all this like other stuff.

729
00:37:57.126 --> 00:38:02.229
But you can take the Legos, the cognitive Legos that you've built when baking bread.

730
00:38:02.330 --> 00:38:02.450
Yeah.

731
00:38:03.170 --> 00:38:10.260
And then use that with some new stuff to figure out the ability to bake a cake from scratch.

732
00:38:10.881 --> 00:38:12.643
Okay?

733
00:38:12.663 --> 00:38:13.965
AI is not very good at this.

734
00:38:14.205 --> 00:38:14.386
Okay.

735
00:38:14.686 --> 00:38:14.966
Okay?

736
00:38:15.828 --> 00:38:16.068
Like...

737
00:38:17.524 --> 00:38:23.846
AI is like in terms of this generalized intelligence that we kind of are after.

738
00:38:24.006 --> 00:38:24.666
Whatever that means.

739
00:38:24.726 --> 00:38:24.886
Right?

740
00:38:24.926 --> 00:38:25.646
Whatever that means.

741
00:38:25.726 --> 00:38:26.587
It has no clear definition.

742
00:38:26.607 --> 00:38:27.347
But I know I'm being facetious.

743
00:38:27.367 --> 00:38:27.947
You know what I mean?

744
00:38:27.967 --> 00:38:28.647
I'm being facetious.

745
00:38:28.787 --> 00:38:32.528
It's like it's good at certain tasks because you've trained it on certain tasks.

746
00:38:32.888 --> 00:38:34.649
It's seen the data for those certain tasks.

747
00:38:35.049 --> 00:38:35.229
Right?

748
00:38:35.529 --> 00:38:42.071
But as soon as you sort of bring it out of its comfort zone, the performance decreases by a lot.

749
00:38:42.691 --> 00:39:07.195
okay and for a really long time it's been kind of a dilemma this question how can a neural network right that's defined by neurons connected to each other so you've got these synaptic connections you want it to be stable enough so that it doesn't forget old skills right right but you also want it to be plastic enough you want it to be

750
00:39:09.296 --> 00:39:29.752
on the move enough that it can actually rapidly assemble new skills okay so it's this dilemma between stability and plasticity plasticity meaning the ability to change and we as as humans have really mastered that tightrope we really have right we have than every any other thing yeah i mean really see

751
00:39:29.952 --> 00:39:33.013
Yeah, I mean, we have millions of years of evolution to thank for that.

752
00:39:33.353 --> 00:39:39.194
But in terms of AI, whenever you have AI, there's this problem called catastrophic forgetting.

753
00:39:39.534 --> 00:39:39.794
Okay.

754
00:39:40.134 --> 00:39:42.835
And that's the idea when you learn a new task.

755
00:39:43.195 --> 00:39:47.836
And when you're learning a new task, it totally obliterates your ability to do some old tasks.

756
00:39:48.136 --> 00:39:51.036
So in this case, you've got a dog-cat classifier.

757
00:39:51.397 --> 00:39:55.397
And then you train it to, you know, figure out how birds look.

758
00:39:55.777 --> 00:39:58.358
And then all of a sudden, it can't figure out what a dog and a cat look.

759
00:39:59.537 --> 00:40:03.483
This is an exaggeration because nowadays we have learning models that can do much better.

760
00:40:03.703 --> 00:40:07.969
But at the end of the day, this idea of catastrophic forgetting is still very much a problem.

761
00:40:08.129 --> 00:40:08.450
Right.

762
00:40:08.510 --> 00:40:13.096
If you want to expand to just generalized ability to solve tasks.

763
00:40:13.677 --> 00:40:13.817
Okay?

764
00:40:13.837 --> 00:40:13.977
Mm-hmm.

765
00:40:14.994 --> 00:40:18.878
Now, in cognitive science, we have another problem, which is called the binding problem.

766
00:40:19.398 --> 00:40:31.108
And that's the idea of how does the brain dynamically bind different attributes to the same thing without requiring a single neuron for all of the specific stuff?

767
00:40:31.128 --> 00:40:31.829
And here's what I mean.

768
00:40:32.209 --> 00:40:35.572
Over there, you've got a person who's looking at a rolling ball.

769
00:40:35.612 --> 00:40:38.275
The ball is red, and it's moving to the right, right?

770
00:40:38.955 --> 00:40:41.717
Now, there's three attributes here that I can immediately see.

771
00:40:41.797 --> 00:40:42.537
There's motion.

772
00:40:42.637 --> 00:40:43.498
It's moving to the right.

773
00:40:43.558 --> 00:40:44.258
There's color.

774
00:40:44.398 --> 00:40:44.918
It's red.

775
00:40:45.238 --> 00:40:45.919
And there's form.

776
00:40:45.999 --> 00:40:47.760
It's a ball, and it's rolling, right?

777
00:40:48.480 --> 00:40:57.365
Now, the brain doesn't have a single neuron for red rolling ball to the right.

778
00:40:58.880 --> 00:41:00.461
Okay, because that would be insane.

779
00:41:00.742 --> 00:41:00.922
Right.

780
00:41:01.122 --> 00:41:06.226
Okay, I don't have like a single neuron for every little object and every little idea.

781
00:41:06.867 --> 00:41:16.996
What the brain does is somehow combine all of these ideas and bind them together into a single idea of a rolling ball that's moving to the right.

782
00:41:17.016 --> 00:41:17.376
Okay.

783
00:41:17.496 --> 00:41:19.136
Okay, and that's called the binding problem.

784
00:41:19.437 --> 00:41:19.717
Okay.

785
00:41:19.837 --> 00:41:20.497
Okay?

786
00:41:20.657 --> 00:41:27.679
It's a problem in neuroscience, and we don't quite know mechanistically inside the brain how these neurons are actually doing it.

787
00:41:28.079 --> 00:41:29.159
Okay?

788
00:41:29.259 --> 00:41:30.299
You can talk about it.

789
00:41:30.359 --> 00:41:30.539
Right.

790
00:41:30.559 --> 00:41:33.200
Like, oh, there's an idea of a red.

791
00:41:33.540 --> 00:41:36.301
But at the end of the day, it's like, what is the neuron doing?

792
00:41:36.441 --> 00:41:36.601
Right.

793
00:41:36.621 --> 00:41:38.401
Because these are just cells in my brain.

794
00:41:39.762 --> 00:41:44.683
And it is incredible because it's a highly efficient neuron.

795
00:41:44.863 --> 00:41:49.167
highly sustainable architectural system.

796
00:41:49.187 --> 00:41:49.447
Yes.

797
00:41:50.088 --> 00:41:50.288
Yeah.

798
00:41:50.388 --> 00:41:51.849
And like, that can learn anything.

799
00:41:51.969 --> 00:41:53.451
It would be good to know how it works.

800
00:41:53.511 --> 00:41:53.791
Yes.

801
00:41:54.111 --> 00:41:54.892
For a whole number of reasons.

802
00:41:54.912 --> 00:41:55.712
Yes, exactly.

803
00:41:55.793 --> 00:41:59.115
And that's what the key discovery of this paper that came out of nature is.

804
00:41:59.215 --> 00:41:59.496
Okay.

805
00:42:00.176 --> 00:42:01.517
What they figured out is the,

806
00:42:02.418 --> 00:42:27.440
neural basis at a systems neuroscience level of these cognitive legos okay and they used something called low dimensional shared neural subspaces we're going to get into what that is but there's a lot of math involved and some very cool physics intuition which is why i really love this paper because it reminds me of like sort of why i got into like physics and neuroscience and the beauty of that in the first place so

807
00:42:28.354 --> 00:42:34.777
What we're really doing is we're solving the problem of this binding problem using very cool mathematics.

808
00:42:35.637 --> 00:42:38.759
And it might actually give us insights into how to train AI better.

809
00:42:39.259 --> 00:42:48.623
Which makes sense because ultimately if we understand how to solve the binding problem, that's immediately going to be architecture you want to implement into any kind of artificial system that you want to replicate human intelligence.

810
00:42:48.643 --> 00:42:49.424
Yes, exactly.

811
00:42:49.664 --> 00:42:51.245
And AI is, at the end of the day...

812
00:42:52.252 --> 00:42:56.574
For a long time, it's been inspired by human cognition.

813
00:42:57.334 --> 00:43:00.775
So there's a reason why they're called neural networks, right?

814
00:43:01.495 --> 00:43:01.715
Okay.

815
00:43:02.116 --> 00:43:05.457
So let's talk about some mathematics now.

816
00:43:06.377 --> 00:43:06.617
Okay?

817
00:43:06.917 --> 00:43:07.297
Here we go.

818
00:43:08.538 --> 00:43:08.718
So...

819
00:43:11.051 --> 00:43:12.393
Imagine you've got a bunch of neurons.

820
00:43:13.014 --> 00:43:13.195
Okay.

821
00:43:13.295 --> 00:43:15.939
We've got about, I think, 80 billion neurons.

822
00:43:16.480 --> 00:43:17.982
We're not going to talk about that many neurons.

823
00:43:18.523 --> 00:43:22.890
But what you can think about when we think about what is a brain state –

824
00:43:23.583 --> 00:43:23.883
Okay?

825
00:43:23.943 --> 00:43:24.563
Okay.

826
00:43:24.603 --> 00:43:26.383
Let's first talk about what is a brain state.

827
00:43:26.504 --> 00:43:35.545
A brain state would be a description of all of your neurons and which ones are active and which ones are not active.

828
00:43:35.945 --> 00:43:36.145
Okay.

829
00:43:36.165 --> 00:43:36.725
For example.

830
00:43:37.245 --> 00:43:37.465
Right?

831
00:43:37.645 --> 00:43:38.526
Like a zero one.

832
00:43:38.566 --> 00:43:39.306
Like a map of a zero one state.

833
00:43:39.326 --> 00:43:39.446
Yeah.

834
00:43:39.466 --> 00:43:40.746
That's one way of thinking about it.

835
00:43:40.766 --> 00:43:41.186
Just a simple one.

836
00:43:41.206 --> 00:43:44.607
Another way would be like what's the firing rate of each of these neurons.

837
00:43:44.887 --> 00:43:45.107
Right?

838
00:43:45.527 --> 00:43:46.707
And is it?

839
00:43:47.107 --> 00:43:47.307
Yeah.

840
00:43:47.327 --> 00:43:49.908
But at the end of the day, it's something like is this neuron active or not?

841
00:43:50.208 --> 00:43:52.288
And then how is it connected to all the other stuff?

842
00:43:52.308 --> 00:43:52.508
Okay.

843
00:43:52.628 --> 00:44:01.170
But the state of the brain at any given point, like right now when I'm doing this podcast, there's certain neurons that are turning on and off, right?

844
00:44:01.570 --> 00:44:09.491
And my brain is moving from one state to the other as I speak, as I go through what I'm trying to talk about and things like that, okay?

845
00:44:10.212 --> 00:44:15.393
And given we've got so many neurons, we run into this curse of dimensionality.

846
00:44:16.353 --> 00:44:21.214
And that means that if you've got a lot of neurons...

847
00:44:22.358 --> 00:44:25.440
Your brain state lives in a high dimensional space.

848
00:44:26.140 --> 00:44:26.861
Here's what that means.

849
00:44:26.881 --> 00:44:27.421
I was going to say, yeah.

850
00:44:27.441 --> 00:44:27.621
Okay.

851
00:44:28.022 --> 00:44:30.803
So suppose I only have three neurons.

852
00:44:31.904 --> 00:44:32.184
Okay.

853
00:44:32.424 --> 00:44:34.226
Imagine a brain with only three neurons.

854
00:44:34.846 --> 00:44:42.351
That brain is going to live in a 3D space because the X axis is going to tell you how neuron one is behaving.

855
00:44:42.791 --> 00:44:47.134
The Y axis is going to tell you how neuron two and the Z neuron three.

856
00:44:47.194 --> 00:44:48.515
So where I am here is.

857
00:44:49.398 --> 00:44:53.963
at any given moment, is going to tell me what this three-neuron brain is doing.

858
00:44:54.984 --> 00:44:56.486
Now imagine a thousand neurons.

859
00:44:56.947 --> 00:44:59.790
I've got a thousand different axes, right?

860
00:45:00.291 --> 00:45:06.658
And each state in this thousand-dimensional space is a brain state.

861
00:45:07.718 --> 00:45:07.958
Okay?

862
00:45:08.379 --> 00:45:17.045
The curse of dimensionality is that if there's unstructured data all over the place, then the amount of data that is required to figure out where I am is going to be, like, ridiculous.

863
00:45:17.065 --> 00:45:17.785
Yeah, yeah, yeah.

864
00:45:17.805 --> 00:45:18.005
Okay?

865
00:45:18.325 --> 00:45:20.107
So clearly the brain doesn't do that.

866
00:45:20.447 --> 00:45:23.489
And we know that it doesn't do that because— That would be, like, more of a brute force approach.

867
00:45:23.609 --> 00:45:29.053
Yeah, it would be, like, just a really terrible way to do any kind of learning, anything like that.

868
00:45:29.113 --> 00:45:29.533
Highly inefficient.

869
00:45:29.573 --> 00:45:30.374
Highly inefficient.

870
00:45:30.434 --> 00:45:31.635
It would take, like, longer than the—

871
00:45:32.784 --> 00:45:37.653
Lifespan of the organism to actually like get any learning done, get any cognition done.

872
00:45:37.914 --> 00:45:38.134
Okay.

873
00:45:38.455 --> 00:45:42.322
So instead, what the brain does is correlate these neurons.

874
00:45:43.747 --> 00:45:43.987
Okay?

875
00:45:44.247 --> 00:45:45.868
Because neurons are connected to each other.

876
00:45:46.189 --> 00:45:51.292
So if one of them fires, then it's going to influence its neighbors to fire, and so on and so forth.

877
00:45:51.612 --> 00:45:58.697
So the neural activity is actually constrained in what's called a low-dimensional manifold, a manifold you can think of as a surface.

878
00:45:59.237 --> 00:46:04.161
And here you can think, let's go back to our analogy of the three-neuron brain.

879
00:46:04.581 --> 00:46:04.841
Okay?

880
00:46:05.081 --> 00:46:07.723
You've got a three-neuron brain, neuron one, neuron two, neuron three.

881
00:46:07.743 --> 00:46:11.526
But let's say neuron three is actually correlated to how neuron one and neuron two do.

882
00:46:12.206 --> 00:46:12.486
Okay?

883
00:46:14.648 --> 00:46:26.183
My dimensionality is actually not three dimensions, because if I know what neuron one and neuron two are doing, I immediately know what neuron three is doing, because these two, let's say, are input to them.

884
00:46:26.764 --> 00:46:30.970
And so neuron three's activity is on a plane.

885
00:46:32.993 --> 00:46:36.855
And so instead of three dimensions, I'm really only talking about two dimensions.

886
00:46:37.135 --> 00:46:43.137
And my brain states are confined to this manifold in a higher dimensional space.

887
00:46:43.177 --> 00:46:46.479
The manifold being this lower dimensional 2D structure.

888
00:46:46.719 --> 00:46:54.302
The idea is because of the correlation that you know exists, you can abstract the complexity to...

889
00:46:55.222 --> 00:47:15.129
a simpler structure because if ball goes up like if neuron one goes if you know when neuron one goes up neuron two goes right you you don't you can now use yeah yeah i i know that information right yeah yeah and that becomes sort of like i can rule out all this other stuff that otherwise would have been in the probabilities okay yeah exactly yeah you see

890
00:47:15.149 --> 00:47:15.649
See what I'm saying?

891
00:47:15.990 --> 00:47:22.396
So now you're constraining your brain states to a low dimensional manifold in this very high dimensional space.

892
00:47:22.416 --> 00:47:32.425
So even though you have a thousand neurons that you're dealing with, maybe the brain actually only lives on a much lower dimensional surface in this thousand dimensional space.

893
00:47:32.785 --> 00:47:37.309
Just like the three brain, the three neuron brain was only living in a 2D surface.

894
00:47:37.710 --> 00:47:40.813
You could even imagine a 1D surface where it's like a line.

895
00:47:40.833 --> 00:47:41.053
Yes.

896
00:47:41.333 --> 00:47:41.513
Right.

897
00:47:41.753 --> 00:47:41.893
Yes.

898
00:47:41.953 --> 00:47:45.956
And so all three of them are connected in some way and there's no, there's no independence.

899
00:47:46.016 --> 00:47:50.619
It's just like, all you have to do is figure out where you are on the line rather than where you are in 3d space.

900
00:47:51.039 --> 00:47:51.259
Okay.

901
00:47:51.760 --> 00:47:51.940
So,

902
00:47:52.978 --> 00:48:00.283
The central insight here is that if we have two different cognitive variables, remember, we're talking about the binding problem, right?

903
00:48:00.303 --> 00:48:01.244
That's what we're trying to solve.

904
00:48:01.524 --> 00:48:03.906
We had the red and we had the ball and we had the motion.

905
00:48:03.946 --> 00:48:05.667
These are three different things.

906
00:48:06.327 --> 00:48:08.289
That we don't store in individual neurons.

907
00:48:08.389 --> 00:48:09.209
Exactly, right?

908
00:48:09.249 --> 00:48:13.992
So we've got to figure out a way for the brain as a state to store this information.

909
00:48:14.713 --> 00:48:14.953
Okay?

910
00:48:15.534 --> 00:48:19.937
So the way we can do this is if we have a giant...

911
00:48:20.752 --> 00:48:24.714
high-dimensional space, but we can constrain ourselves to manifolds.

912
00:48:25.235 --> 00:48:31.258
What if there's two different manifolds for the two different variables that we're trying to store?

913
00:48:31.939 --> 00:48:32.539
Okay.

914
00:48:32.779 --> 00:48:33.040
Okay?

915
00:48:33.060 --> 00:48:33.240
Yes.

916
00:48:33.580 --> 00:48:34.300
That's the idea.

917
00:48:34.360 --> 00:48:34.660
Okay.

918
00:48:34.681 --> 00:48:43.306
The key idea is you can have in your, let's go back to our three-neuron brain, you can have orthogonal planes.

919
00:48:43.346 --> 00:48:45.507
You can have planes that are perpendicular to each other.

920
00:48:45.887 --> 00:48:47.808
And I've got a photo to show this.

921
00:48:50.918 --> 00:48:55.006
which is one plane, and then I can have a shape plane, let's say.

922
00:48:55.527 --> 00:48:58.712
And that's the second plane, or a motion plane.

923
00:48:58.753 --> 00:49:01.438
And that's the second plane.

924
00:49:01.458 --> 00:49:02.760
So now, here's what we would do.

925
00:49:03.648 --> 00:49:05.829
Your brain is somewhere in 3D, right?

926
00:49:05.910 --> 00:49:07.070
In this 3D space.

927
00:49:07.451 --> 00:49:17.297
But what a downstream circuit can do is look at all of these neurons and then figure out what is the shadow of my brain state on one plane?

928
00:49:17.597 --> 00:49:18.698
That's going to tell me the color.

929
00:49:19.438 --> 00:49:21.740
The shadow of my brain state on the other plane?

930
00:49:21.880 --> 00:49:23.921
That's going to tell me the shape.

931
00:49:24.642 --> 00:49:25.042
You see?

932
00:49:25.302 --> 00:49:29.025
So I can keep track of a bunch of different information.

933
00:49:29.045 --> 00:49:29.285
Yeah.

934
00:49:30.422 --> 00:49:33.344
By constraining myself along these manifolds.

935
00:49:33.364 --> 00:49:34.885
The intersections of different manifolds.

936
00:49:36.406 --> 00:49:37.126
That's the idea.

937
00:49:37.166 --> 00:49:42.549
That's – because the idea is that like for lack of a better way to put it, it's – I'm still grokking.

938
00:49:42.889 --> 00:49:51.374
But like the – you have a high – a very high dimensional space of potential – potentialities.

939
00:49:51.415 --> 00:49:51.535
Yeah.

940
00:49:51.575 --> 00:49:51.675
Yeah.

941
00:49:52.616 --> 00:50:00.643
But you have a very acute space, whether it's on a single manifold or at the intersection of the manifolds, which is actually what you care about.

942
00:50:00.723 --> 00:50:12.252
And that surface area, the amount of information or surface area is fundamentally smaller and shorter to traverse than navigating through the entirety of that larger multidimensional space.

943
00:50:12.272 --> 00:50:12.633
Yeah, yeah.

944
00:50:12.653 --> 00:50:16.836
It's kind of like you can't fully have unstructured space.

945
00:50:17.316 --> 00:50:44.804
just like brain space right right like your brain states have to have to adhere conform to some conform to some order in order to actually like learn and like like get ideas and like keep track of ideas right so what you could do is you could have a brain state that's in this high dimensional space and then you can have another brain circuit that's somewhere else yes that is looking at this brain state and then what it does is calculate okay where along this plane am i yeah yeah

946
00:50:45.004 --> 00:50:49.166
And then another, the color circuit is going to be like, actually, I care about this other plane.

947
00:50:49.186 --> 00:50:51.327
So where along this other plane am I?

948
00:50:51.387 --> 00:50:52.108
Right?

949
00:50:52.328 --> 00:50:55.530
And then the binding problem is kind of solved.

950
00:50:55.850 --> 00:50:56.090
Right.

951
00:50:56.390 --> 00:50:56.590
Right.

952
00:50:56.930 --> 00:50:57.110
Right.

953
00:50:57.271 --> 00:51:05.535
Because you don't, you now have an ability to store the color.

954
00:51:08.013 --> 00:51:12.016
the shape, and the motion along these men.

955
00:51:12.036 --> 00:51:13.077
With the same number of neurons.

956
00:51:13.097 --> 00:51:14.238
With the same number of neurons.

957
00:51:14.318 --> 00:51:18.342
I don't need individual neurons for red, for blue, for this, that, right?

958
00:51:18.442 --> 00:51:25.988
And I don't need an individual neuron for red ball, red square, red star, blue ball, blue square, blue star, right?

959
00:51:26.388 --> 00:51:31.593
I can now, with the same number of neurons, all I have to do is just calculate where I am on this plane and this plane.

960
00:51:32.113 --> 00:51:32.413
Yes.

961
00:51:32.534 --> 00:51:33.154
You see what I'm saying?

962
00:51:33.174 --> 00:51:33.554
Yeah, I do.

963
00:51:33.815 --> 00:51:34.876
It's very cool.

964
00:51:34.956 --> 00:51:35.216
Yes.

965
00:51:35.616 --> 00:51:36.097
Please continue.

966
00:51:36.117 --> 00:51:36.337
Sorry.

967
00:51:36.377 --> 00:51:36.717
But yes.

968
00:51:36.897 --> 00:51:37.098
Yes.

969
00:51:37.338 --> 00:51:37.958
Okay.

970
00:51:38.919 --> 00:51:40.140
So it's cool to think about.

971
00:51:40.501 --> 00:51:40.721
Yes.

972
00:51:40.861 --> 00:51:45.985
But what truly impressed me about this paper is how they actually went about proving this.

973
00:51:46.166 --> 00:51:46.346
Right.

974
00:51:46.366 --> 00:51:49.308
Because this is quite like very abstract.

975
00:51:49.509 --> 00:51:49.829
Yeah.

976
00:51:50.089 --> 00:51:50.289
Right.

977
00:51:50.670 --> 00:51:52.611
So like how are we actually doing this?

978
00:51:52.751 --> 00:51:53.132
Prove it.

979
00:51:53.312 --> 00:51:53.452
Yeah.

980
00:51:53.692 --> 00:51:54.172
Exactly.

981
00:51:55.093 --> 00:51:57.153
So they used two adult Reus monkeys.

982
00:51:57.594 --> 00:52:05.697
The Reus monkeys are chosen because the prefrontal cortex, the neocortex of a Reus monkey is very similar to a human neocortex.

983
00:52:05.757 --> 00:52:05.997
Okay.

984
00:52:06.337 --> 00:52:13.480
So, you know, for it's better than, for example, a lot of the other model organisms that we use like rodents because...

985
00:52:14.300 --> 00:52:18.524
This brain can be involved with rule switching, abstract categorization.

986
00:52:18.564 --> 00:52:19.805
So it's very close that way.

987
00:52:19.845 --> 00:52:23.909
And we can actually try to probe these higher order abstract ideas.

988
00:52:24.289 --> 00:52:24.489
Right.

989
00:52:25.510 --> 00:52:32.436
And the stimuli that the subjects were responding to were of two kinds.

990
00:52:32.616 --> 00:52:34.838
OK, there's shape and then there's color.

991
00:52:35.279 --> 00:52:39.883
The shape that they were responding to is a bunny versus a tee.

992
00:52:39.903 --> 00:52:40.063
Right.

993
00:52:41.170 --> 00:52:44.031
Okay, and the bunny ears became a T. Okay, got it.

994
00:52:44.331 --> 00:52:49.633
Okay, so that's the sort of shape axis that we're trying to discern, right?

995
00:52:50.113 --> 00:52:53.834
And then on the other hand, we have the color axis, which is green versus red.

996
00:52:54.515 --> 00:53:01.157
And one of the things that they were very careful about is that these axes are circular.

997
00:53:01.797 --> 00:53:02.757
They don't have edges.

998
00:53:03.117 --> 00:53:03.317
Okay.

999
00:53:03.337 --> 00:53:03.538
Okay.

1000
00:53:03.818 --> 00:53:09.339
So you smoothly go from a T to a bunny back to a T and from green to red back to green.

1001
00:53:09.659 --> 00:53:09.899
Okay.

1002
00:53:10.239 --> 00:53:13.040
The reason why this is important is because you don't have any edge effects.

1003
00:53:13.740 --> 00:53:13.940
Right?

1004
00:53:14.000 --> 00:53:22.483
Like if you have like a square, let's say, then there might be some artifact from being in the corner here because there's nowhere to go over on this side.

1005
00:53:23.063 --> 00:53:25.346
So your brain state might be doing something weird, right?

1006
00:53:25.466 --> 00:53:25.586
Yes.

1007
00:53:25.606 --> 00:53:28.789
But here in this case, you've got a circle, and then you've got another circle.

1008
00:53:28.810 --> 00:53:29.590
There's no edges.

1009
00:53:29.690 --> 00:53:29.891
Yes.

1010
00:53:30.051 --> 00:53:32.534
So every point in the circle is equivalent.

1011
00:53:32.554 --> 00:53:32.794
Yes.

1012
00:53:32.954 --> 00:53:33.154
Right?

1013
00:53:33.294 --> 00:53:33.415
Yes.

1014
00:53:33.435 --> 00:53:38.000
So we're really trying to figure out the topology of this manifold that we're trying to look at.

1015
00:53:38.060 --> 00:53:38.280
Yes.

1016
00:53:38.440 --> 00:53:38.640
Right?

1017
00:53:39.021 --> 00:53:41.143
And the manifold turns out to be a donut.

1018
00:53:41.263 --> 00:53:43.986
It's a Clifford Taurus because it's a circle.

1019
00:53:44.816 --> 00:53:50.219
which is a circle like this way, and then a circle around the donut, and then like through the...

1020
00:53:50.259 --> 00:53:52.240
There's two circles in a donut.

1021
00:53:52.520 --> 00:53:56.582
So you're constraining your stimulus to a torus.

1022
00:53:56.622 --> 00:53:57.042
To a torus.

1023
00:53:57.122 --> 00:53:57.382
Okay?

1024
00:53:57.743 --> 00:53:57.983
So...

1025
00:53:58.870 --> 00:54:03.915
presumably the brain should also have some kind of representation of this Taurus.

1026
00:54:04.296 --> 00:54:06.618
And the task that they're doing is the following, okay?

1027
00:54:06.859 --> 00:54:19.632
So you've got a stimulus, the subject looks at the stimulus, and then he does a saccade, which is the eyes go in one particular direction based on whether it's a tea or a bunny or if it's green or if it's red, okay?

1028
00:54:19.652 --> 00:54:19.832
Okay.

1029
00:54:20.786 --> 00:54:29.892
So the subjects are trained to categorize shape, in which case it's a bunny or a T, and they respond in one axis.

1030
00:54:30.032 --> 00:54:34.034
So the axis being like, you know, upper left, lower right.

1031
00:54:34.054 --> 00:54:34.235
Mm-hmm.

1032
00:54:35.019 --> 00:54:36.500
That's going to be the shape axis.

1033
00:54:37.400 --> 00:54:42.803
And then there's going to be another task, which is color, and they're going to do the opposite direction, which is upper right, lower left.

1034
00:54:43.123 --> 00:54:43.344
Okay?

1035
00:54:43.624 --> 00:54:44.044
That's key.

1036
00:54:44.564 --> 00:54:45.985
So we need to understand that.

1037
00:54:46.205 --> 00:54:48.526
And task design is everything.

1038
00:54:48.967 --> 00:54:49.267
Okay?

1039
00:54:49.787 --> 00:55:00.213
In a lot of these systems, neuroscience, behavior, and abstraction experiments, I remember when I was in my PhD, like, we used to spend entire lab meetings just discussing the task.

1040
00:55:05.357 --> 00:55:09.140
how to probe these very weird questions that we're asking, right?

1041
00:55:09.621 --> 00:55:17.327
Because once we have the task, then we can figure out how to analyze what the neuron was doing in this task versus this task and so on and so forth, right?

1042
00:55:17.668 --> 00:55:18.969
So the task is as follows.

1043
00:55:19.289 --> 00:55:21.591
There's a shape axis, which is, let's say, this way.

1044
00:55:22.552 --> 00:55:25.614
And then there's a color axis, which is this way, okay?

1045
00:55:26.055 --> 00:55:29.638
And then what they have to do is they have to switch the axes, right?

1046
00:55:30.420 --> 00:55:30.680
Okay?

1047
00:55:30.780 --> 00:55:30.980
Okay.

1048
00:55:31.000 --> 00:55:33.921
So before, the shape axis was upper left, lower right.

1049
00:55:34.261 --> 00:55:36.361
The color axis was upper right, lower left.

1050
00:55:36.741 --> 00:55:44.183
And then now what they have to do is they have to say, actually, I'm going to respond to color, but using the shape axis.

1051
00:55:45.123 --> 00:55:45.423
Okay.

1052
00:55:46.284 --> 00:55:46.704
Okay.

1053
00:55:46.844 --> 00:55:47.104
Okay?

1054
00:55:47.344 --> 00:55:47.624
Okay.

1055
00:55:47.944 --> 00:55:50.164
So let's go through this again.

1056
00:55:50.264 --> 00:55:50.484
Okay.

1057
00:55:50.644 --> 00:55:50.885
Okay?

1058
00:55:50.965 --> 00:55:51.445
Yeah, yeah, yeah.

1059
00:55:51.705 --> 00:55:52.345
With shape...

1060
00:55:53.123 --> 00:55:56.645
If I see the shape one way, I'm going to go to the lower right or the upper right.

1061
00:55:57.445 --> 00:56:01.727
If I see the color, I'm going to go to the upper left or lower left.

1062
00:56:02.368 --> 00:56:05.449
But now in the third task, I'm mixing the two.

1063
00:56:06.070 --> 00:56:06.370
Okay?

1064
00:56:06.730 --> 00:56:12.813
So I'm looking at color, but I'm actually responding with the motion of the shape.

1065
00:56:13.453 --> 00:56:13.773
Okay.

1066
00:56:14.194 --> 00:56:14.394
Okay?

1067
00:56:14.414 --> 00:56:14.594
Yes.

1068
00:56:15.847 --> 00:56:18.468
Here's the key hypothesis, okay?

1069
00:56:18.989 --> 00:56:28.834
If this cognitive Legos is correct, then the brain should learn that mixing task, but it shouldn't learn it as a brand new problem.

1070
00:56:29.992 --> 00:56:31.094
It should be just...

1071
00:56:31.174 --> 00:56:40.586
It should be a cognitive Lego of the shape task from the motion side and a cognitive Lego from the color task combining them together.

1072
00:56:40.646 --> 00:56:40.827
Yes.

1073
00:56:41.047 --> 00:56:45.993
The neural representations of those two should be resilient.

1074
00:56:46.013 --> 00:56:46.734
Okay.

1075
00:56:46.754 --> 00:56:46.975
Okay.

1076
00:56:47.707 --> 00:56:48.528
Like it does.

1077
00:56:48.608 --> 00:56:49.528
It's not ephemeral.

1078
00:56:50.389 --> 00:56:50.849
It's not.

1079
00:56:51.330 --> 00:56:52.631
When you say resilient, how do you mean?

1080
00:56:52.931 --> 00:56:55.293
What I mean by resilient is the same neurons.

1081
00:56:55.453 --> 00:56:55.633
Yes.

1082
00:56:55.773 --> 00:56:56.994
That were responding to color.

1083
00:56:57.174 --> 00:56:57.354
Yes.

1084
00:56:57.474 --> 00:56:59.255
Are now going to still be responding to color.

1085
00:56:59.536 --> 00:56:59.696
Yes.

1086
00:56:59.716 --> 00:57:03.639
The same neurons that were responding to this axis are still going to be responding to this axis.

1087
00:57:03.959 --> 00:57:04.099
Yes.

1088
00:57:04.179 --> 00:57:07.662
But the subject is still is going to be doing a completely new task.

1089
00:57:07.862 --> 00:57:08.022
Right.

1090
00:57:08.042 --> 00:57:08.302
Right.

1091
00:57:08.843 --> 00:57:11.205
Using the same old neurons.

1092
00:57:11.225 --> 00:57:11.505
Yes.

1093
00:57:11.926 --> 00:57:14.668
Using the same sort of procedure, right?

1094
00:57:14.688 --> 00:57:26.320
The color processing module developed for one task and the response, the axis motor response for another task, it's going to just snap them together like Legos.

1095
00:57:26.360 --> 00:57:27.080
Literally like Legos.

1096
00:57:27.361 --> 00:57:27.521
Yeah.

1097
00:57:27.861 --> 00:57:30.524
And then that gives it the ability to then...

1098
00:57:31.945 --> 00:57:58.015
do the derivation without ascribing more neurons to do exactly yeah yeah so it's it's it's learning something but it's actually it's it's it's able to learn because it's able to snap older older learn habits together but at the neural level yes the neurons are now working together yes yes yes so yes the the idea is if you okay i'm gonna pause this one this one is this one is

1099
00:57:58.728 --> 00:57:59.448
This one's weird.

1100
00:57:59.468 --> 00:58:00.509
This one's weird.

1101
00:58:00.549 --> 00:58:01.449
This is weird.

1102
00:58:01.509 --> 00:58:04.451
Neuroscience is really cool to think about that way.

1103
00:58:04.831 --> 00:58:07.692
I just... Yeah, let's pause and really think about this.

1104
00:58:07.773 --> 00:58:10.574
Because I know we have a lot to cover, but I'm really trying to...

1105
00:58:11.034 --> 00:58:20.559
The point is, right, you have these... A neuron has a learning about whether it's shape or color in this context on these axes.

1106
00:58:20.759 --> 00:58:21.019
Yeah.

1107
00:58:21.519 --> 00:58:25.862
And let's say, as an analogy, the color...

1108
00:58:26.882 --> 00:58:33.130
is a red Lego brick, and the shape is a blue Lego brick.

1109
00:58:34.071 --> 00:58:42.322
And so rather than saying when... Rather than having a new yellow Lego brick, you can just take those two

1110
00:58:43.223 --> 00:58:50.368
put them together now and they are now able to get to this, all this reversed response task that you're talking about.

1111
00:58:50.508 --> 00:58:52.669
This like combined, this, this, yeah.

1112
00:58:52.689 --> 00:58:57.012
And the, and the key is that we're combining two different modalities here.

1113
00:58:57.112 --> 00:58:59.974
We're, we're combining it's, it's, it's, it's,

1114
00:59:01.262 --> 00:59:27.402
responding to color right but the response is with the shape response it's it's using the the the the axis that it's using it used to use for shape yeah it's a totally different task but it is able to reuse an old a previously paved path for a different uh use case without losing the ability to still do the old thing yeah

1115
00:59:28.153 --> 00:59:30.735
And also not losing the ability to continue to do the new thing.

1116
00:59:30.755 --> 00:59:31.696
Yes, exactly.

1117
00:59:31.736 --> 00:59:35.339
And one of the things they did was they randomized, like, which task they were doing.

1118
00:59:35.399 --> 00:59:39.303
They found that the performance didn't decrease at all.

1119
00:59:39.543 --> 00:59:42.606
And that's something that, like, biological brains do all the time.

1120
00:59:42.886 --> 00:59:50.552
But the key thing is what they can do is they can actually do simultaneous single-unit recordings of five different brain regions at the same time.

1121
00:59:52.013 --> 00:59:52.233
Okay.

1122
00:59:52.273 --> 00:59:52.873
So this is...

1123
00:59:52.893 --> 00:59:53.133
Okay.

1124
00:59:53.494 --> 00:59:55.414
And this is where the neurons come in, right?

1125
00:59:55.474 --> 01:00:03.956
So we can now do electrophysiology, which is literally putting electrodes into the brain and then listen to thousands of neurons at the same time.

1126
01:00:04.036 --> 01:00:06.357
I think it's on the order of a thousand in this study.

1127
01:00:06.677 --> 01:00:15.839
From five different brain regions, the parietal, the lateral prefrontal cortex, the frontal eye fields, the temporal cortex, the striatum.

1128
01:00:16.727 --> 01:00:16.907
Right.

1129
01:00:16.947 --> 01:00:23.873
So this is where there's sensory motor information, reinforcement learning, pre-motor information.

1130
01:00:24.213 --> 01:00:28.717
There's an executive hub, which is the prefrontal cortex, which is really the task rule.

1131
01:00:28.777 --> 01:00:31.159
It's trying to figure out which task am I on in the first place.

1132
01:00:31.539 --> 01:00:31.740
Right.

1133
01:00:32.800 --> 01:00:35.262
And that is what is really cool.

1134
01:00:35.803 --> 01:00:37.805
OK, is the fact that we could use all of these things.

1135
01:00:38.245 --> 01:00:43.970
And so how do we actually make sure that like how do we show that these cognitive Legos are actually working?

1136
01:00:45.055 --> 01:00:46.536
How do you demonstrate existence?

1137
01:00:46.556 --> 01:00:46.757
Yes.

1138
01:00:47.157 --> 01:00:50.019
How do you demonstrate that there's these manifolds and all this other stuff?

1139
01:00:50.379 --> 01:00:50.980
Here's what they did.

1140
01:00:51.280 --> 01:00:51.500
Okay.

1141
01:00:51.820 --> 01:00:55.503
So you get a classifier that is trained on color.

1142
01:00:55.523 --> 01:00:56.664
Okay.

1143
01:00:57.084 --> 01:00:57.344
Okay.

1144
01:00:57.685 --> 01:00:59.826
In task, in the color task.

1145
01:01:01.828 --> 01:01:10.274
And what you do is you say, okay, now in this new task where it's trying to find color, but the motor response is completely different.

1146
01:01:10.294 --> 01:01:10.394
Okay.

1147
01:01:11.494 --> 01:01:13.115
When you say motor response.

1148
01:01:13.155 --> 01:01:16.916
Meaning in the first task, the color response was upper left, lower right.

1149
01:01:16.977 --> 01:01:17.797
Oh, I guess.

1150
01:01:17.837 --> 01:01:22.359
But my motor response now in the second task is same color, but now I'm going in the opposite axis.

1151
01:01:22.379 --> 01:01:22.859
Yes, yes, yes.

1152
01:01:22.899 --> 01:01:23.079
Right?

1153
01:01:23.800 --> 01:01:33.244
If I train a classifier on the color task and I want to decode the color, meaning I look at the neurons and I try to guess the color.

1154
01:01:33.711 --> 01:01:35.152
Was the color red or blue?

1155
01:01:35.172 --> 01:01:35.312
No way.

1156
01:01:35.713 --> 01:01:35.973
Right?

1157
01:01:36.253 --> 01:01:36.433
Green?

1158
01:01:36.453 --> 01:01:37.755
No way.

1159
01:01:37.935 --> 01:01:39.616
And I don't retrain the classifier.

1160
01:01:39.636 --> 01:01:39.776
Right.

1161
01:01:39.796 --> 01:01:42.959
I just now test it on the second task.

1162
01:01:43.560 --> 01:01:43.820
Yes.

1163
01:01:43.920 --> 01:01:45.281
Which is a totally different motor thing.

1164
01:01:45.301 --> 01:01:45.441
Yes.

1165
01:01:45.882 --> 01:01:47.023
It can still guess the color.

1166
01:01:47.063 --> 01:01:47.783
No way.

1167
01:01:47.803 --> 01:01:48.143
You see?

1168
01:01:48.164 --> 01:01:49.124
This is the key.

1169
01:01:49.204 --> 01:01:50.365
This is the key.

1170
01:01:50.425 --> 01:01:50.766
Yeah, yeah.

1171
01:01:51.506 --> 01:01:52.267
Now it clicked, right?

1172
01:01:52.327 --> 01:01:52.527
Yes.

1173
01:01:52.567 --> 01:02:00.513
This result is very profound because it means that the geometry of the representation in this high dimensional space is stable, right?

1174
01:02:01.354 --> 01:02:06.618
That's actually, no, that, no, that is, and the way they got to, it's different.

1175
01:02:08.707 --> 01:02:09.807
I love these neuroscience.

1176
01:02:09.947 --> 01:02:11.528
People are so clever.

1177
01:02:11.688 --> 01:02:12.429
Yeah.

1178
01:02:12.589 --> 01:02:13.850
Because that's what that means.

1179
01:02:13.970 --> 01:02:19.393
If you've trained a classifier here, and then now this classifier is not trained on this second task.

1180
01:02:19.453 --> 01:02:19.654
Right.

1181
01:02:19.734 --> 01:02:24.997
But because the neural representation is stable, I can take all of the learning I've done here and just apply it here.

1182
01:02:25.277 --> 01:02:25.998
And I can guess...

1183
01:02:26.867 --> 01:02:28.628
If it was looking at green or red or whatever.

1184
01:02:28.688 --> 01:02:31.751
Which means that underlying manifold structure is the same.

1185
01:02:32.471 --> 01:02:34.653
It's like a real thing.

1186
01:02:34.693 --> 01:02:35.193
It's a thing.

1187
01:02:35.333 --> 01:02:39.496
It's like a real mathematical thing in this high dimensional space.

1188
01:02:41.238 --> 01:02:41.318
Wow.

1189
01:02:41.458 --> 01:02:42.278
Very cool, right?

1190
01:02:42.298 --> 01:02:43.259
This one is very good.

1191
01:02:43.339 --> 01:02:44.860
And they did the same thing for the shape.

1192
01:02:45.160 --> 01:02:45.281
Yes.

1193
01:02:45.301 --> 01:02:49.183
They did the same thing for the direction of the motor and all that other stuff.

1194
01:02:49.203 --> 01:02:50.584
That's like a really big deal.

1195
01:02:50.604 --> 01:02:51.205
Mm-hmm.

1196
01:02:52.767 --> 01:02:55.230
Yeah, I thought it was very, very cool.

1197
01:02:55.591 --> 01:03:01.279
This is why everyone was rushing to get out their manifold stories early this year, because it seems to be a confluence of... Yeah, it is.

1198
01:03:01.359 --> 01:03:05.124
It is kind of cool, because the next story we're going to talk about is about manifolds as well, right?

1199
01:03:05.184 --> 01:03:05.765
Oh, my God.

1200
01:03:06.787 --> 01:03:13.753
One of the other cool things that they did was, so, you know, if like they have a compression, they actually what they found was.

1201
01:03:13.833 --> 01:03:14.174
Sorry.

1202
01:03:14.694 --> 01:03:14.914
Yeah.

1203
01:03:15.054 --> 01:03:16.776
This is cool.

1204
01:03:16.876 --> 01:03:17.636
So cool.

1205
01:03:17.837 --> 01:03:18.837
It's cool.

1206
01:03:18.917 --> 01:03:20.959
They had also this thing called the compression index.

1207
01:03:20.979 --> 01:03:24.142
What they found was like whatever the task is not yet.

1208
01:03:24.322 --> 01:03:24.902
Oh, sorry.

1209
01:03:25.122 --> 01:03:28.765
But like whatever the task is relevant for, like suppose it was paying attention to color.

1210
01:03:29.106 --> 01:03:29.306
Yes.

1211
01:03:29.366 --> 01:03:31.708
Then the shape manifold would actually like shrink.

1212
01:03:31.728 --> 01:03:31.908
Yeah.

1213
01:03:32.733 --> 01:03:33.033
Okay.

1214
01:03:33.254 --> 01:03:33.514
Okay?

1215
01:03:33.574 --> 01:03:39.080
Like, if you look at the neurons that were doing the shape stuff, they wouldn't be as active anymore.

1216
01:03:39.140 --> 01:03:42.483
Because the subject is paying attention to the color first.

1217
01:03:43.004 --> 01:03:43.404
And then...

1218
01:03:43.885 --> 01:03:48.349
So even the manifolds themselves are dynamic in that sense.

1219
01:03:48.389 --> 01:03:48.629
Yes.

1220
01:03:48.689 --> 01:03:48.870
Right?

1221
01:03:48.890 --> 01:03:53.594
The neurons that are in this plane are now a little bit shut off because they're not as important right now.

1222
01:03:53.635 --> 01:03:53.755
Right?

1223
01:03:54.235 --> 01:03:55.075
Or right now.

1224
01:03:55.436 --> 01:03:57.116
You know, in this particular task.

1225
01:03:57.136 --> 01:03:57.336
Yes.

1226
01:03:57.796 --> 01:03:59.477
I mean, it's kind of like muscles, right?

1227
01:03:59.837 --> 01:04:00.297
Very much.

1228
01:04:00.377 --> 01:04:02.138
Like, if you're not using them, they kind of get a little smaller.

1229
01:04:02.198 --> 01:04:02.398
Yeah.

1230
01:04:02.418 --> 01:04:03.939
But then they're still kind of there.

1231
01:04:04.059 --> 01:04:08.320
Yeah, but this is like, you know, on the order of minutes that they're like responding.

1232
01:04:08.701 --> 01:04:13.903
Because the subject knows which task is happening.

1233
01:04:13.923 --> 01:04:15.363
Happening in real time.

1234
01:04:15.383 --> 01:04:22.266
And then subsequently which one to then prioritize or have active, in a more active state.

1235
01:04:22.466 --> 01:04:22.686
Yeah.

1236
01:04:24.187 --> 01:04:25.068
Very cool.

1237
01:04:25.568 --> 01:04:30.271
It's very cool to see just how our brain has done this.

1238
01:04:30.331 --> 01:04:37.435
Millions of years of evolution, and we're getting very close to solving stuff like the binding problem, which is a very big deal.

1239
01:04:38.056 --> 01:04:41.218
And this has a lot of implications for artificial intelligence.

1240
01:04:41.578 --> 01:04:42.578
I mean, immediately.

1241
01:04:42.699 --> 01:04:45.140
Because catastrophic forgetting is very much a thing, right?

1242
01:04:45.280 --> 01:04:45.440
Right.

1243
01:04:45.480 --> 01:04:47.101
So there was actually this...

1244
01:04:47.742 --> 01:04:52.264
There's a new way of trying to figure out how to...

1245
01:04:53.565 --> 01:05:16.970
not do catastrophic forgetting so if we go to photo 15 right there's something called orthogonal subspace learning which is the idea that if i want to train on one task task a and then i also want to train on task b what i can do is i can make the two subspaces orthogonal inside my neural network and now when i do back propagation which we're going to get into later but it's just

1246
01:05:24.406 --> 01:05:30.433
The model is going to train for task A, and the way that the model is going to train for task B are going to be orthogonal.

1247
01:05:30.453 --> 01:05:30.633
Yes.

1248
01:05:30.813 --> 01:05:32.395
So they're not going to interfere with each other.

1249
01:05:32.415 --> 01:05:32.595
Yes.

1250
01:05:32.876 --> 01:05:35.058
And you can enforce that constraint.

1251
01:05:35.398 --> 01:05:35.739
I get you.

1252
01:05:35.759 --> 01:05:41.025
Like, if I'm on task A, I'm only going to move along this plane in my parameter space.

1253
01:05:41.045 --> 01:05:41.205
Yes.

1254
01:05:41.585 --> 01:05:45.410
And if I'm trying to do task B, I'm only going to move along this plane in my parameter space.

1255
01:05:45.830 --> 01:05:45.990
Right.

1256
01:05:46.010 --> 01:05:48.352
And that way I don't forget how I did task A.

1257
01:05:48.392 --> 01:05:48.592
Right.

1258
01:05:48.773 --> 01:05:52.396
If I now all of a sudden start trying to learn task B.

1259
01:05:52.516 --> 01:06:04.025
It's putting in roads and traffic lights into the pathways of how the processes proceed in a way that they don't interfere with each other.

1260
01:06:04.045 --> 01:06:04.346
Mm-hmm.

1261
01:06:05.291 --> 01:06:06.632
This is so good.

1262
01:06:06.972 --> 01:06:08.472
So, yeah, it's really cool.

1263
01:06:08.492 --> 01:06:12.714
I mean, there's this new AI architecture called Orthogonal Low-Rank Adaptation.

1264
01:06:13.374 --> 01:06:14.514
Oh, Laura.

1265
01:06:14.955 --> 01:06:15.715
Or Oh, Laura.

1266
01:06:16.335 --> 01:06:17.496
It very recently came out.

1267
01:06:17.516 --> 01:06:20.577
I think it was last year where they're trying to do this.

1268
01:06:20.797 --> 01:06:29.200
Because the orthogonal idea, this like orthogonal subspace learning, conceptually, that maps to the same thing you were just talking about earlier about when you have...

1269
01:06:29.600 --> 01:06:33.501
plane, this plane, and this plane.

1270
01:06:33.541 --> 01:06:34.562
That's the orthogonal.

1271
01:06:34.662 --> 01:06:37.723
And they're applying it now in the context of training.

1272
01:06:38.203 --> 01:06:41.764
But it... We're able to experimentally see this.

1273
01:06:41.804 --> 01:06:42.225
See this.

1274
01:06:42.585 --> 01:06:43.785
That's what's so cool.

1275
01:06:43.825 --> 01:06:46.506
That is... And like, straight up confirm it.

1276
01:06:47.086 --> 01:06:47.286
Right.

1277
01:06:47.907 --> 01:06:48.647
That is very good.

1278
01:06:48.727 --> 01:06:49.827
I thought it was a very cool paper.

1279
01:06:49.887 --> 01:06:51.048
I mean, it's hard to beat

1280
01:06:52.032 --> 01:06:52.452
Princeton.

1281
01:06:52.513 --> 01:06:52.753
Yeah.

1282
01:06:52.993 --> 01:06:53.514
At anything.

1283
01:06:53.774 --> 01:06:55.456
I mean, maybe football.

1284
01:06:55.816 --> 01:06:56.877
You might get us in football.

1285
01:06:57.378 --> 01:06:58.038
Like all the time.

1286
01:06:58.218 --> 01:07:10.912
But when it comes to being the leaders in a variety of hard sciences, my brain is currently having a lot of orthogonal subspace.

1287
01:07:10.932 --> 01:07:11.112
Yeah.

1288
01:07:11.452 --> 01:07:11.733
Yeah.

1289
01:07:12.993 --> 01:07:15.494
Yeah, every time we get on this podcast, I have to do that.

1290
01:07:15.874 --> 01:07:17.514
A lot of orthogonal subspace learning.

1291
01:07:18.114 --> 01:07:19.375
But it's very cool.

1292
01:07:19.415 --> 01:07:21.575
And I think, you know, in terms of...

1293
01:07:21.595 --> 01:07:25.296
So this is not just for fundamental science, right?

1294
01:07:25.336 --> 01:07:26.436
You might be asking, okay, like...

1295
01:07:26.957 --> 01:07:27.457
Cool guys.

1296
01:07:27.477 --> 01:07:28.617
Cool guys, right?

1297
01:07:28.717 --> 01:07:37.899
But, you know, cognitive inflexibility is one of the symptoms of psychiatric disorders like schizophrenia and ASD, right?

1298
01:07:38.280 --> 01:07:39.240
And, you know...

1299
01:07:40.200 --> 01:07:49.768
Given this kind of fundamental research, it could be that that inflexibility comes from the failure to orthogonally separate and compress subspaces in your head.

1300
01:07:50.169 --> 01:07:56.975
So you can imagine like non-invasive neuromodulation therapies that would like orthogonalize.

1301
01:07:58.458 --> 01:07:59.219
Sub spaces.

1302
01:07:59.239 --> 01:08:00.940
Like make it more perpendicular.

1303
01:08:00.960 --> 01:08:03.143
You have these headbands on that would be... Yeah, yeah.

1304
01:08:03.163 --> 01:08:03.923
That would like...

1305
01:08:04.023 --> 01:08:04.544
Exactly.

1306
01:08:04.564 --> 01:08:05.565
That would do the thing.

1307
01:08:05.585 --> 01:08:05.985
Yeah.

1308
01:08:06.025 --> 01:08:06.926
But the point is there's a...

1309
01:08:07.086 --> 01:08:08.348
I mean, that's far out.

1310
01:08:08.468 --> 01:08:11.591
But like you got to get to this kind of fundamental research.

1311
01:08:11.611 --> 01:08:13.433
You have to understand that the manifold is there.

1312
01:08:13.453 --> 01:08:13.853
Yeah.

1313
01:08:13.873 --> 01:08:14.053
Yeah.

1314
01:08:15.134 --> 01:08:15.615
Yeah.

1315
01:08:15.895 --> 01:08:22.260
And I think fundamentally what I really liked about this paper is because it provides a physics of meaning.

1316
01:08:23.281 --> 01:08:26.364
It's like meaning is literally like geometry in your brain.

1317
01:08:26.684 --> 01:08:35.551
Like your brain is living in this high dimensional subspace and there's like a geometry underlying it.

1318
01:08:35.571 --> 01:08:35.832
Mm-hmm.

1319
01:08:37.174 --> 01:08:39.235
that gives stuff meaning.

1320
01:08:40.215 --> 01:08:41.175
I thought that was very cool.

1321
01:08:41.315 --> 01:08:45.636
It's sort of the whole, this is why math is discovered, not invented.

1322
01:08:46.216 --> 01:08:47.117
It's even in our brain.

1323
01:08:47.337 --> 01:08:47.517
Right.

1324
01:08:47.657 --> 01:08:47.877
Yeah.

1325
01:08:48.157 --> 01:08:48.617
Very cool.

1326
01:08:49.057 --> 01:08:49.777
Fascinating.

1327
01:08:49.857 --> 01:08:57.019
This is our story number two out of Princeton neuroscience.

1328
01:08:57.519 --> 01:08:58.219
I'm really struggling.

1329
01:08:58.239 --> 01:08:59.160
I'm normally not like this.

1330
01:08:59.580 --> 01:09:00.740
That story really kind of...

1331
01:09:01.969 --> 01:09:11.247
I'm going to have to grok that one for a minute, but I have a good follow-up for it now because we are going to go into story number three, which is another deep seek moment.

1332
01:09:11.267 --> 01:09:11.328
Um,

1333
01:09:12.515 --> 01:09:14.196
Jan 1, they came into 2020.

1334
01:09:14.216 --> 01:09:15.436
January 1st, dude.

1335
01:09:15.516 --> 01:09:17.897
Just like all guns blazing.

1336
01:09:17.917 --> 01:09:18.857
Guns blazing.

1337
01:09:19.017 --> 01:09:23.518
It's all over the news, especially if you're in tech, et cetera, and all these other spaces.

1338
01:09:23.778 --> 01:09:24.039
Yeah.

1339
01:09:24.139 --> 01:09:31.401
But there's something about manifold constraints, and they broke the scaling laws, and everyone's hair on fire.

1340
01:09:31.801 --> 01:09:32.701
How are they doing it?

1341
01:09:33.361 --> 01:09:35.022
Ref, somebody, please.

1342
01:09:35.042 --> 01:09:35.622
Yeah.

1343
01:09:36.009 --> 01:09:38.511
Well, they're trying to beat scaling laws, right?

1344
01:09:38.591 --> 01:09:42.313
Because they're GPU poor at the end of the day.

1345
01:09:43.574 --> 01:09:50.459
Scaling laws are very effectively more compute, more parameters, better performance.

1346
01:09:51.020 --> 01:09:53.141
If you have bigger, better stuff, you spend a lot of money.

1347
01:09:53.541 --> 01:09:57.464
More chips, bigger, better, faster chips means you can get to the next stage of...

1348
01:09:58.685 --> 01:10:00.326
Whatever you're trying to scale up.

1349
01:10:00.446 --> 01:10:01.787
Yeah, yeah, exactly.

1350
01:10:01.907 --> 01:10:03.708
But there's a caveat, right?

1351
01:10:04.008 --> 01:10:07.590
Which is at massive scale, you've got like hundreds of billions of parameters.

1352
01:10:07.631 --> 01:10:09.592
I think now we're getting to trillions of parameters.

1353
01:10:09.832 --> 01:10:11.753
You're going to hit something called a stability limit.

1354
01:10:12.213 --> 01:10:12.453
Okay.

1355
01:10:12.994 --> 01:10:14.735
And in deep neural networks,

1356
01:10:15.854 --> 01:10:18.375
It's all about signal propagation, right?

1357
01:10:18.435 --> 01:10:22.237
Which is like, there's a signal of did I get something right or did I get something wrong?

1358
01:10:22.737 --> 01:10:26.799
And as that propagates through my network, it's very chaotic, right?

1359
01:10:27.079 --> 01:10:31.281
And it can lead to amplification, vanishing gradient problem, exploding gradient problem.

1360
01:10:31.301 --> 01:10:32.561
These are stuff that we're going to talk about.

1361
01:10:33.162 --> 01:10:33.382
So...

1362
01:10:34.182 --> 01:10:35.463
There's some kind of limit, right?

1363
01:10:35.823 --> 01:10:41.047
There might be some kind of like thermodynamic limit to how much we can do.

1364
01:10:42.267 --> 01:10:47.411
And the solution is what DeepSeek is purporting to have found.

1365
01:10:47.791 --> 01:10:48.031
Okay.

1366
01:10:48.051 --> 01:10:50.513
It's something called manifold constrained hyperconnections.

1367
01:10:51.733 --> 01:11:00.323
It's a new way to train models more efficiently and actually, like, straight up better.

1368
01:11:01.064 --> 01:11:01.304
Okay?

1369
01:11:01.585 --> 01:11:03.407
But it's a very small hack that they're doing.

1370
01:11:03.427 --> 01:11:04.848
And it's actually quite simple.

1371
01:11:05.469 --> 01:11:06.290
The hack that they're...

1372
01:11:07.071 --> 01:11:09.774
The little hack that they did is, like, really simple.

1373
01:11:10.415 --> 01:11:10.615
But...

1374
01:11:11.516 --> 01:11:12.737
Apparently, it's a very big deal.

1375
01:11:12.897 --> 01:11:13.178
Okay.

1376
01:11:13.218 --> 01:11:13.478
Okay.

1377
01:11:13.818 --> 01:11:14.759
So let's get into it.

1378
01:11:14.939 --> 01:11:15.199
Okay.

1379
01:11:15.419 --> 01:11:16.020
Neural networks.

1380
01:11:16.060 --> 01:11:18.462
First, let's just talk about neural networks real quick.

1381
01:11:18.582 --> 01:11:18.822
Okay.

1382
01:11:19.122 --> 01:11:24.106
Neural networks are a giant mathematical function that basically it's fancy linear algebra.

1383
01:11:24.767 --> 01:11:25.047
Okay.

1384
01:11:25.087 --> 01:11:30.251
This is a transformer architecture from the very famous paper.

1385
01:11:31.372 --> 01:11:32.493
Attention is all you need.

1386
01:11:33.114 --> 01:11:33.314
Right.

1387
01:11:33.534 --> 01:11:33.794
Sorry.

1388
01:11:33.814 --> 01:11:34.655
I repeated after him.

1389
01:11:34.675 --> 01:11:36.156
I almost got through the whole episode without doing it.

1390
01:11:37.077 --> 01:11:38.859
It's hard to unlearn 15 years of friendship, please.

1391
01:11:38.879 --> 01:11:39.599
Yeah, yeah, yeah.

1392
01:11:40.140 --> 01:11:46.506
Well, so all of this is showing all of these arrows are basically numbers going from one thing to the other.

1393
01:11:46.906 --> 01:11:49.448
And then each block is effectively a matrix.

1394
01:11:49.809 --> 01:11:54.653
You're multiplying matrices, then you add stuff, then you multiply a matrix, then you do a threshold, then you add stuff.

1395
01:11:55.214 --> 01:11:58.477
It's a giant mathematical function at the end of the day.

1396
01:11:58.797 --> 01:12:00.659
And each of the building blocks...

1397
01:12:01.808 --> 01:12:05.049
of these are something called an artificial neuron.

1398
01:12:05.689 --> 01:12:05.929
Okay?

1399
01:12:06.390 --> 01:12:22.655
And an artificial neuron is effectively kind of like a biological neuron in that it sums up its inputs, and then if that input is above a certain threshold, well, depending on the activation function, but let's just say it's above a certain threshold, then it lets it through.

1400
01:12:22.795 --> 01:12:25.276
If it's below a certain threshold, then it doesn't let anything through.

1401
01:12:25.596 --> 01:12:27.236
So that's that non-linearity part.

1402
01:12:27.697 --> 01:12:27.937
Okay?

1403
01:12:29.137 --> 01:12:35.318
And when we train a network, we've got these billions of parameters, billions of neurons that are all connected together.

1404
01:12:35.738 --> 01:12:38.179
And what we want to do is something called back propagation.

1405
01:12:38.559 --> 01:12:42.459
So you've got a network with weights, which are how these neurons are connected.

1406
01:12:44.220 --> 01:12:48.561
What you do is you initialize the network with a bunch of random stuff.

1407
01:12:48.981 --> 01:12:50.081
Usually it's like random weights.

1408
01:12:50.641 --> 01:12:52.301
You can get more complicated that way.

1409
01:12:53.301 --> 01:12:55.922
What it's going to do is it's going to have a guess at an answer.

1410
01:12:56.875 --> 01:13:05.146
It's going to compare its guess to the real thing, compute a loss, which is how different was my guess from the real thing, and then...

1411
01:13:06.258 --> 01:13:14.385
propagate that back through the network and change the weights such that the next time it's going to be a bit closer to the correct answer.

1412
01:13:14.765 --> 01:13:16.246
That's what back propagation is.

1413
01:13:16.467 --> 01:13:21.831
Literally, the movie Tenet is a movie about humans back propagating in like really.

1414
01:13:21.951 --> 01:13:25.214
It's like I just made that connection in my head right now, but that's literally what they were doing.

1415
01:13:25.254 --> 01:13:25.694
It's funny.

1416
01:13:26.095 --> 01:13:27.496
That's so funny, actually.

1417
01:13:27.936 --> 01:13:28.817
Yeah.

1418
01:13:29.237 --> 01:13:29.438
So.

1419
01:13:30.078 --> 01:13:32.659
So that's effectively what backpropagation is.

1420
01:13:32.779 --> 01:13:38.681
And really what you're doing is like the chain rule in calculus where you take the change here and you multiply it.

1421
01:13:39.382 --> 01:13:42.923
You take a derivative here and you multiply it to the next layer, to the next layer, to the next layer.

1422
01:13:42.963 --> 01:13:46.164
And that's how you calculate how much you want to change your weights by.

1423
01:13:46.304 --> 01:13:46.545
Okay.

1424
01:13:46.825 --> 01:13:48.885
It's a product of these.

1425
01:13:50.506 --> 01:14:14.882
derivatives right okay um something called the jacobian matrix we don't have to we don't have to get into what this jacobian matrix is but here is the key thing okay effectively whenever you're doing back propagation the amount that the weights here change is based on a product of all of these numbers right now if the numbers that you're multiplying are a bit less than one

1426
01:14:15.708 --> 01:14:16.648
Let's say 0.9.

1427
01:14:17.349 --> 01:14:19.629
But there's 100 layers.

1428
01:14:19.709 --> 01:14:26.112
0.9 times 0.9 times 0.9 100 times is something like 0.00003.

1429
01:14:26.932 --> 01:14:29.052
This is the vanishing gradient problem.

1430
01:14:29.753 --> 01:14:30.533
Interesting.

1431
01:14:30.553 --> 01:14:30.853
Okay?

1432
01:14:31.873 --> 01:14:42.077
The idea is the stuff that's changing, the weights that are going to change, if I do this very naively, is only going to be towards the output.

1433
01:14:43.188 --> 01:14:43.629
Okay.

1434
01:14:44.010 --> 01:14:44.851
Of my neural network.

1435
01:14:45.292 --> 01:14:53.045
All of the earlier layers that I have, like, you know, it goes input and then there's something, something, something, something all the way to output.

1436
01:14:53.165 --> 01:14:57.332
My change, my backpropagation is never going to make it to these early layers.

1437
01:14:58.358 --> 01:15:02.102
Because I've just been multiplying a number that's slightly smaller than one, smaller than one.

1438
01:15:02.142 --> 01:15:05.845
I mean, in effect, I just used the idea of 0.9 to the 100, right?

1439
01:15:06.086 --> 01:15:09.109
Sometimes it's 0.5, and then you're really screwed, right?

1440
01:15:09.769 --> 01:15:13.313
So that's the vanishing gradient problem, if you're really bad at it.

1441
01:15:14.174 --> 01:15:15.895
We solved this already, but I'm just letting you know.

1442
01:15:16.516 --> 01:15:18.637
This is still kind of a problem.

1443
01:15:18.657 --> 01:15:20.838
You couldn't just say, oh, backpropagating, we're fine.

1444
01:15:20.858 --> 01:15:23.479
It's like, well, only if it's not greater than 1.

1445
01:15:23.579 --> 01:15:23.839
Yeah.

1446
01:15:23.859 --> 01:15:24.959
And also...

1447
01:15:25.079 --> 01:15:30.201
So here's the problem that happens if that number that you're multiplying is a little bit smaller than 1.

1448
01:15:30.542 --> 01:15:32.983
If it's larger than 1, then you have the opposite problem, right?

1449
01:15:33.003 --> 01:15:35.404
Because 1.1 to the 100, 1.1 times 1.1 times 1.1 is something like 14,000.

1450
01:15:35.464 --> 01:15:36.344
And so you have an exploding gradient.

1451
01:15:36.364 --> 01:15:36.504
Mm-hmm.

1452
01:15:41.626 --> 01:15:41.846
Right.

1453
01:15:42.126 --> 01:15:45.248
Where like one, the system gets really chaotic.

1454
01:15:45.308 --> 01:15:48.970
It's kind of like a chain reaction in like nuclear physics.

1455
01:15:49.030 --> 01:15:49.210
Yes.

1456
01:15:49.410 --> 01:15:49.610
Right.

1457
01:15:50.370 --> 01:15:56.433
And there have been historical fixes like the long short term memory LSTM model.

1458
01:15:56.534 --> 01:15:58.555
It used something called the constant air carousel.

1459
01:15:58.955 --> 01:16:02.156
But it's it's that thing was limited to sequential data.

1460
01:16:02.497 --> 01:16:02.757
OK.

1461
01:16:03.037 --> 01:16:03.237
Right.

1462
01:16:03.457 --> 01:16:05.598
Along came in 2015 ResNet.

1463
01:16:05.978 --> 01:16:06.238
OK.

1464
01:16:06.278 --> 01:16:07.279
Residual network.

1465
01:16:07.739 --> 01:16:07.859
OK.

1466
01:16:11.441 --> 01:16:11.621
Okay.

1467
01:16:11.861 --> 01:16:12.101
Okay.

1468
01:16:12.381 --> 01:16:14.943
They said, I'm not only going to have a feed forward layer.

1469
01:16:15.103 --> 01:16:18.044
So it goes layer and then it does something goes to the next layer.

1470
01:16:18.404 --> 01:16:21.126
I'm also just going to pass along the input.

1471
01:16:22.106 --> 01:16:22.426
Oh, okay.

1472
01:16:22.526 --> 01:16:24.167
As in just the same thing.

1473
01:16:24.207 --> 01:16:24.427
Yes.

1474
01:16:24.607 --> 01:16:33.091
So, so the next, the next part of the network, the next block is going to get my original input plus whatever computation I've done.

1475
01:16:33.271 --> 01:16:33.431
Right.

1476
01:16:33.491 --> 01:16:33.812
Okay.

1477
01:16:33.872 --> 01:16:37.693
Your, your output from your part of the chain plus what you had as your input to get to that.

1478
01:16:37.733 --> 01:16:37.954
Yeah.

1479
01:16:37.994 --> 01:16:38.154
Yeah.

1480
01:16:38.214 --> 01:16:39.754
And that's that residual connection.

1481
01:16:39.794 --> 01:16:41.095
That's, that's why it's called the resnet.

1482
01:16:41.235 --> 01:16:41.435
Okay.

1483
01:16:41.455 --> 01:16:41.635
Okay.

1484
01:16:41.896 --> 01:16:43.237
I like to just call it the skip.

1485
01:16:43.597 --> 01:16:43.998
The skip.

1486
01:16:44.218 --> 01:16:44.518
Okay.

1487
01:16:44.618 --> 01:16:45.419
The skip connection.

1488
01:16:45.439 --> 01:16:45.539
Okay.

1489
01:16:45.859 --> 01:16:46.079
All right.

1490
01:16:46.520 --> 01:16:53.526
And the skip connection is very cool because this is basically doing something called Euler's method, which is something that you learn in calculus.

1491
01:16:53.886 --> 01:16:59.451
The idea is like if you've got a differential equation and you want to figure out where it's going, all you do is you say, okay, well, the slope is this way.

1492
01:16:59.471 --> 01:17:04.356
I'm going to make a step, but I'm going to keep going in that direction and then make a step and then keep going in that direction.

1493
01:17:04.396 --> 01:17:04.576
Right?

1494
01:17:04.996 --> 01:17:05.217
So...

1495
01:17:06.240 --> 01:17:12.347
That identity mapping, what it does is it ensures that you're not going to have that vanishing gradient problem.

1496
01:17:12.367 --> 01:17:12.487
Right.

1497
01:17:12.827 --> 01:17:13.008
Right?

1498
01:17:13.248 --> 01:17:17.813
Because even if it's like 0.99 to the... You're still adding... Like, it's still 1...

1499
01:17:18.904 --> 01:17:20.344
plus or minus something, right?

1500
01:17:20.545 --> 01:17:21.565
One plus or minus something.

1501
01:17:21.605 --> 01:17:26.126
And you can still have an exploding gradient because you could still have that plus, plus, plus.

1502
01:17:26.347 --> 01:17:30.688
But then you have stuff like batch norm and layer norm that can keep that in check, right?

1503
01:17:31.368 --> 01:17:33.689
So this kind of solved that, the ResNet.

1504
01:17:33.729 --> 01:17:36.030
The ResNet problem kind of solved that, okay?

1505
01:17:36.050 --> 01:17:36.130
Yes.

1506
01:17:36.770 --> 01:17:39.551
The problem with the ResNet is you want to do better.

1507
01:17:39.772 --> 01:17:40.912
You always want to do better.

1508
01:17:41.252 --> 01:17:41.492
Okay?

1509
01:17:41.793 --> 01:17:44.314
In this case, I've got one residual connection.

1510
01:17:44.354 --> 01:17:45.635
It's like a single-lane highway.

1511
01:17:45.695 --> 01:17:50.657
All I'm doing is I'm taking the input, I do a bunch of computation, and then I add it to the input again.

1512
01:17:50.697 --> 01:17:50.877
Right?

1513
01:17:51.317 --> 01:17:52.978
This thing is a single-lane highway.

1514
01:17:53.358 --> 01:17:54.859
I'm not doing anything fancy here.

1515
01:17:55.059 --> 01:17:55.620
Low bandwidth.

1516
01:17:55.800 --> 01:17:56.340
Low bandwidth.

1517
01:17:56.380 --> 01:17:57.921
I'm not doing anything fancy.

1518
01:17:58.041 --> 01:18:04.904
2024-2025, a paper by ByteDance, their AI, which is the TikTok company.

1519
01:18:04.924 --> 01:18:05.104
Yes.

1520
01:18:05.124 --> 01:18:05.645
Right?

1521
01:18:05.665 --> 01:18:05.745
Yes.

1522
01:18:05.785 --> 01:18:05.885
Yes.

1523
01:18:07.049 --> 01:18:13.226
They come up with something called hyper connections, which is what if we can start doing something fancy with that?

1524
01:18:14.777 --> 01:18:15.957
You know, that skip connection.

1525
01:18:16.097 --> 01:18:17.798
Instead of just like copy pasting.

1526
01:18:18.038 --> 01:18:18.198
Right.

1527
01:18:18.518 --> 01:18:21.298
What if I do something fancy in that lane?

1528
01:18:21.398 --> 01:18:21.598
Right.

1529
01:18:21.698 --> 01:18:21.959
Okay.

1530
01:18:22.239 --> 01:18:23.599
And this is what they did.

1531
01:18:23.679 --> 01:18:29.860
So on the left hand side is your original ResNet, which is just copy paste, copy paste, do some computation, copy paste like that.

1532
01:18:29.960 --> 01:18:30.120
Yes.

1533
01:18:30.180 --> 01:18:37.102
On the right hand side, what they're doing is you can imagine instead of a single lane highway, you've got multiple lanes and you're mixing traffic.

1534
01:18:37.502 --> 01:18:37.602
Okay.

1535
01:18:37.622 --> 01:18:37.842
Okay.

1536
01:18:38.322 --> 01:18:46.588
Okay, so you've got all of this information, and you want to find the optimal way to send this information to the next place, right?

1537
01:18:47.068 --> 01:18:49.450
This was the ByteDance idea from HyperConnections.

1538
01:18:49.470 --> 01:18:51.831
This was a paper in 2024, 2025, right?

1539
01:18:51.871 --> 01:18:52.972
This is good.

1540
01:18:53.653 --> 01:19:01.378
So the idea is to always have more and more computation in all of these little segregated spots because then that makes your model better, right?

1541
01:19:01.398 --> 01:19:01.518
Right.

1542
01:19:01.538 --> 01:19:02.018
The more sort of...

1543
01:19:03.549 --> 01:19:06.290
Fine tuning you're doing here and there and here and there.

1544
01:19:06.330 --> 01:19:06.551
Yeah.

1545
01:19:06.591 --> 01:19:06.951
Yeah.

1546
01:19:07.011 --> 01:19:07.911
You're getting better, right?

1547
01:19:07.971 --> 01:19:09.952
The model output is going to be better.

1548
01:19:10.052 --> 01:19:19.517
Letting it run and then trying to like send back a conclusion that has now had these intermediate steps where there's no feedback loop.

1549
01:19:19.577 --> 01:19:22.559
Yeah, there's no, and there's no, it's kind of just lazy.

1550
01:19:22.699 --> 01:19:23.059
Yeah, yeah.

1551
01:19:23.120 --> 01:19:23.340
Right?

1552
01:19:23.420 --> 01:19:24.781
It's just like copy-paste.

1553
01:19:24.801 --> 01:19:25.101
Yeah, yeah.

1554
01:19:25.161 --> 01:19:30.685
Well, but what if the next step could use some more finagling, right?

1555
01:19:30.745 --> 01:19:35.748
What if I could finagle this part to do a little bit better here, right?

1556
01:19:35.808 --> 01:19:38.110
So that I get a better answer all the way at the end.

1557
01:19:38.410 --> 01:19:38.871
Right?

1558
01:19:38.911 --> 01:19:39.711
That's the whole idea.

1559
01:19:39.791 --> 01:19:40.172
Makes sense.

1560
01:19:40.272 --> 01:19:40.452
Right?

1561
01:19:40.592 --> 01:19:44.174
So it's that multiple lanes kind of traffic merging.

1562
01:19:44.274 --> 01:19:49.078
And the point is during training, we figure out what the rules of the traffic merging are, right?

1563
01:19:49.138 --> 01:19:49.278
Mm-hmm.

1564
01:19:49.598 --> 01:19:57.763
Okay, the problem is kind of the same of what we had earlier, which is if you're...

1565
01:19:58.163 --> 01:20:03.907
The whole point of this skip connection was to make sure that I don't blow up and I don't vanish.

1566
01:20:04.267 --> 01:20:11.672
But now, if I don't have that identity, I could start blowing up and vanishing here on this end.

1567
01:20:12.692 --> 01:20:12.912
Right?

1568
01:20:12.972 --> 01:20:14.633
It's the same problem that I had earlier.

1569
01:20:14.753 --> 01:20:15.154
I gotcha.

1570
01:20:15.694 --> 01:20:15.994
Okay?

1571
01:20:16.574 --> 01:20:17.275
So, so...

1572
01:20:19.712 --> 01:20:21.834
It's hard to use, these hyperconnections.

1573
01:20:21.854 --> 01:20:24.856
Because they were just being let loose a little bit.

1574
01:20:24.876 --> 01:20:29.179
Like, maybe there was a wide road, but not lanes.

1575
01:20:29.400 --> 01:20:29.640
Yeah.

1576
01:20:29.840 --> 01:20:33.122
In between what, like, would be, like, lanes for different things to go down.

1577
01:20:33.142 --> 01:20:35.744
It was just like, ah, just everyone, like, the wide road, let's all go.

1578
01:20:36.125 --> 01:20:40.548
And if, like, and if the highway itself is now, like, magnifying the amount of traffic.

1579
01:20:40.588 --> 01:20:40.808
Right.

1580
01:20:41.048 --> 01:20:43.891
Then you're running into the same problem that you were earlier.

1581
01:20:43.911 --> 01:20:44.371
That makes sense.

1582
01:20:44.551 --> 01:20:44.791
Right.

1583
01:20:45.212 --> 01:20:45.492
Okay.

1584
01:20:45.952 --> 01:20:48.374
And that is where this particular paper comes in.

1585
01:20:48.674 --> 01:20:48.954
Okay.

1586
01:20:49.135 --> 01:20:50.916
With something called a manifold solution.

1587
01:20:51.116 --> 01:20:51.376
Okay.

1588
01:20:51.576 --> 01:20:51.837
Okay.

1589
01:20:52.217 --> 01:21:00.203
The idea is what you do is you say, okay, remember that hyperconnection, which is a way to change the...

1590
01:21:03.435 --> 01:21:05.716
how we're merging this traffic stuff, right?

1591
01:21:06.777 --> 01:21:09.198
That thing is just another matrix.

1592
01:21:09.358 --> 01:21:09.619
Okay.

1593
01:21:09.639 --> 01:21:18.604
That thing is just another block of numbers that's going to tell me how much of this vector is going to go into that one, how much of this vector is going to go into that one.

1594
01:21:19.044 --> 01:21:26.529
And the problem is that if this matrix amplifies or de-amplifies, then I get into that vanishing or exploding problem, right?

1595
01:21:27.049 --> 01:21:31.992
So what if I constrain that matrix to basically never do that?

1596
01:21:32.652 --> 01:21:33.453
And that's what this is.

1597
01:21:33.833 --> 01:21:36.194
They constrain the matrix to be doubly stochastic.

1598
01:21:36.494 --> 01:21:39.016
And what that means is the sum of all the rows is one.

1599
01:21:39.556 --> 01:21:41.277
The sum of all the columns is one.

1600
01:21:41.898 --> 01:21:44.139
The question is, how do you make a matrix do that?

1601
01:21:44.159 --> 01:21:47.081
It's not really a trivial way to do it.

1602
01:21:47.381 --> 01:21:53.945
But so it's kind of a brute force algorithm, something called the Sinkhorn-NOP algorithm.

1603
01:21:54.245 --> 01:21:56.547
All you do is you basically say, okay, take this row.

1604
01:21:57.624 --> 01:21:58.425
Divide by the sum.

1605
01:21:58.625 --> 01:21:59.726
So now everything sums to one.

1606
01:22:00.266 --> 01:22:01.528
Next row, divide by the sum.

1607
01:22:01.888 --> 01:22:02.088
Okay?

1608
01:22:02.348 --> 01:22:03.509
And then you do it for the rows.

1609
01:22:03.810 --> 01:22:05.151
Now you go along the columns.

1610
01:22:05.411 --> 01:22:06.592
Then you go for the rows again.

1611
01:22:06.632 --> 01:22:07.413
Then you do the columns.

1612
01:22:07.453 --> 01:22:12.858
And one by one, it's like you're folding a pastry over and over again until you get a very nice...

1613
01:22:14.003 --> 01:22:17.324
Matrix where all of the rows and all the columns sum up to one.

1614
01:22:17.925 --> 01:22:23.587
The key is once you do that, you're no longer going to blow up or shrink.

1615
01:22:24.708 --> 01:22:25.868
And that's the key insight.

1616
01:22:26.268 --> 01:22:26.669
That's it.

1617
01:22:26.769 --> 01:22:27.169
That's it.

1618
01:22:27.189 --> 01:22:31.551
It's just an iterative brute force method to constrain the hyperconnection.

1619
01:22:31.591 --> 01:22:31.991
That's it.

1620
01:22:32.813 --> 01:22:33.534
That was the idea.

1621
01:22:33.554 --> 01:22:34.354
That was the idea.

1622
01:22:35.075 --> 01:22:43.841
It's like, you know, I want to do this hyperconnection stuff, but I don't want to deal with all of the nonsense of blowing up and all of that stuff.

1623
01:22:43.921 --> 01:22:48.124
So I'm just going to put in a little hack where I make sure that these hyperconnections

1624
01:22:48.924 --> 01:22:51.125
are constrained so that the sum is one.

1625
01:22:51.725 --> 01:22:53.006
And so I'm not stretching.

1626
01:22:53.046 --> 01:22:53.826
I'm not shrinking.

1627
01:22:54.167 --> 01:22:58.829
So every time I'm not like doing 0.9 times 0.9, it's always one times one times one.

1628
01:22:59.069 --> 01:22:59.989
I'm just mixing.

1629
01:23:00.770 --> 01:23:02.911
I'm mixing it in a really nice way.

1630
01:23:04.182 --> 01:23:27.848
i'm i'm not mad sometimes simplicity it's a very simple hack it's one of those things where some people are probably looking at that paper and being like i can't believe you've done this yeah it's it seems pretty simple now um again within it's not obviously yeah yeah but obviously but like but um this is very important for deep seek though right because deep seek um bingo

1631
01:23:29.762 --> 01:23:32.745
It's hard for them to get GPUs.

1632
01:23:32.765 --> 01:23:42.433
There was this very recent news that actually came out a month ago where like two dudes got caught with $160 million.

1633
01:23:42.833 --> 01:23:48.919
This is, yeah, $160 million worth of GPUs that they were trying to smuggle into China.

1634
01:23:48.939 --> 01:23:49.299
Yeah.

1635
01:23:49.918 --> 01:23:50.659
This was a month ago.

1636
01:23:50.819 --> 01:23:51.460
They're never getting out.

1637
01:23:51.500 --> 01:23:52.301
So they're desperate.

1638
01:23:52.321 --> 01:23:52.621
Yeah.

1639
01:23:52.702 --> 01:23:52.942
Right?

1640
01:23:53.563 --> 01:24:02.854
And that's – this is – so for context, right, and I believe this started under the Biden administration, there was sort of this doctrine about how do we win the sort of great power war for AI?

1641
01:24:02.914 --> 01:24:07.180
And the idea was because people really buy into the scaling laws as like the –

1642
01:24:08.695 --> 01:24:14.617
most accessible lever for us to pull and like the fastest way to sort of accelerate the progress.

1643
01:24:15.877 --> 01:24:19.919
We continue to double down on domestic chip production.

1644
01:24:20.499 --> 01:24:25.560
We have the lead on that because of the likes of people like NVIDIA who just have something no one else has.

1645
01:24:26.801 --> 01:24:30.022
And we control all exports.

1646
01:24:30.903 --> 01:24:35.610
of these, not only to countries like China, but a whole list of others.

1647
01:24:35.930 --> 01:24:41.318
And there was this really big conflict when Trump administration came in because the argument was being made that, uh,

1648
01:24:42.805 --> 01:24:45.047
That is not solving a problem.

1649
01:24:45.527 --> 01:24:46.327
Yeah.

1650
01:24:46.447 --> 01:24:51.430
And then it's also eliminating the ability to generate revenue that can then be applied and put in other places.

1651
01:24:51.470 --> 01:24:51.671
Yeah.

1652
01:24:51.891 --> 01:25:02.137
And so the Chinese and the research community there are having to be clever with not enough because we haven't seen Baidu or any of these larger –

1653
01:25:04.819 --> 01:25:07.680
tech companies within China get to that point.

1654
01:25:08.160 --> 01:25:09.421
They're getting, they're trying to get there.

1655
01:25:09.441 --> 01:25:11.642
They're trying to get there, they're trying to get there, but they have not yet.

1656
01:25:12.063 --> 01:25:13.223
And that's still the bottleneck.

1657
01:25:13.383 --> 01:25:13.543
Yeah.

1658
01:25:13.623 --> 01:25:21.748
And I think, I think very recently the Trump administration said that they're going to allow Nvidia to sell to China, but they're,

1659
01:25:23.115 --> 01:25:27.719
They'd have to pay like a 25% tariff that would go to the government.

1660
01:25:27.739 --> 01:25:30.422
So the government would like share in the profits, something like that.

1661
01:25:30.542 --> 01:25:31.823
I'm not quite clear.

1662
01:25:32.323 --> 01:25:39.710
But in any case, what we're trying to do is like choke out the Chinese AI industry by starving them of GPUs.

1663
01:25:40.010 --> 01:25:40.230
So –

1664
01:25:41.292 --> 01:25:45.017
the DeepSea guys have to be very clever in how they implement this.

1665
01:25:45.118 --> 01:25:46.720
Because here's the idea, right?

1666
01:25:47.200 --> 01:25:55.572
Modern GPUs, like the NVIDIA H100s, they're characterized by a massive disparity between GPUs,

1667
01:25:56.213 --> 01:26:04.959
how much they can compute, so the compute capability, which is a number of flops, floating point operations, and their memory bandwidth in terabytes per second.

1668
01:26:04.979 --> 01:26:22.132
This is called the memory wall, which refers to the latency and the energy costs associated with moving data from memory, which is this high bandwidth RAM, to your spam registers, which is the thing that the computing part of the GPU has access to.

1669
01:26:22.632 --> 01:26:25.194
And so it's really slow to transfer stuff,

1670
01:26:26.317 --> 01:26:40.090
And if you're starved of GPUs, you need to come up with clever ways to, if you want to do something very cool, like this manifold hyperconnection thingy, thingy, thingy, but you don't have a lot of GPUs, you got to get clever, right?

1671
01:26:40.470 --> 01:26:43.954
And that part of the paper I actually thought was really cool, right?

1672
01:26:44.274 --> 01:26:45.575
So you've got this bottleneck, right?

1673
01:26:45.935 --> 01:26:49.559
And if you want to naively do something like this hyperconnection stuff...

1674
01:26:52.354 --> 01:26:54.076
Before, you just had a single lane.

1675
01:26:54.436 --> 01:26:56.719
Now you've got, let's say, four lanes.

1676
01:26:57.359 --> 01:27:01.644
That's four times the compute that you have to do because you have to figure out.

1677
01:27:02.105 --> 01:27:04.247
You have to read your input.

1678
01:27:04.508 --> 01:27:07.551
Then you have to churn the numbers four times.

1679
01:27:08.312 --> 01:27:10.935
And then that's expanding a lot of...

1680
01:27:12.722 --> 01:27:13.142
Compute.

1681
01:27:13.202 --> 01:27:13.843
Compute cost.

1682
01:27:14.163 --> 01:27:18.605
It's four to five times higher than the original ResNet.

1683
01:27:18.705 --> 01:27:18.925
Right.

1684
01:27:19.185 --> 01:27:19.425
Okay?

1685
01:27:19.445 --> 01:27:25.468
And when you're starved with GPUs, four to five times higher input-output is not something that...

1686
01:27:25.668 --> 01:27:26.228
It's too inefficient.

1687
01:27:26.248 --> 01:27:28.730
Yeah, you can't just buy like 500 more.

1688
01:27:28.970 --> 01:27:29.170
Right?

1689
01:27:29.750 --> 01:27:35.573
So what DeepSeek did was reduce that computational overhead to something that was negligible.

1690
01:27:35.733 --> 01:27:36.553
Six to 6.7%.

1691
01:27:38.439 --> 01:27:38.639
Really?

1692
01:27:38.659 --> 01:27:39.921
Instead of four to five times.

1693
01:27:40.242 --> 01:27:41.103
And here's how they did it.

1694
01:27:41.223 --> 01:27:43.727
They used this language called TileLang.

1695
01:27:44.048 --> 01:27:44.368
Okay.

1696
01:27:44.669 --> 01:27:44.929
Okay?

1697
01:27:45.250 --> 01:27:50.899
And what they're doing is literally overcoming the memory wall by something called kernel fusion.

1698
01:27:50.979 --> 01:27:53.403
So whenever you're doing AI...

1699
01:27:54.886 --> 01:27:56.667
Let's say you do like some kind of operation.

1700
01:27:56.727 --> 01:27:58.108
You have to like start up a kernel.

1701
01:27:58.408 --> 01:27:58.968
Yes.

1702
01:27:59.088 --> 01:28:01.210
It's just like, it's kind of like a processing unit.

1703
01:28:01.230 --> 01:28:01.430
Yes.

1704
01:28:01.770 --> 01:28:02.030
Okay.

1705
01:28:02.390 --> 01:28:06.152
And individual kernels are started up for individual tasks.

1706
01:28:06.192 --> 01:28:09.694
Like there's matrix multiplication, there's the activation and stuff like that.

1707
01:28:10.315 --> 01:28:15.758
But what these guys did was customize it so that all of the math is done in a single kernel.

1708
01:28:16.518 --> 01:28:38.673
faster arithmetic and the intermediate results don't go to memory back and forth so you don't like you don't like do you don't like go to memory get some data do something put it back in memory go somewhere else put it back it's kind of like if you're like a chef like an industrial chef you don't go to the fridge all the time right it's like you get all your ingredients you put it on your stovetop the fridge is like your high bandwidth memory yeah

1709
01:28:38.933 --> 01:28:42.576
The stovetop is like your SRAM, which is the countertop.

1710
01:28:42.856 --> 01:28:47.779
So it's limited, but you want to do everything you can in this little time.

1711
01:28:48.260 --> 01:28:49.981
And the kernel is your instructions.

1712
01:28:50.321 --> 01:28:57.446
But if you can make your kernel very complicated and clever, then I don't have to go visit the fridge all the time.

1713
01:28:58.287 --> 01:28:59.528
And that's exactly what they did.

1714
01:29:00.148 --> 01:29:04.011
Reduce the global memory reads by a huge amount.

1715
01:29:04.291 --> 01:29:05.232
That's fascinating.

1716
01:29:06.122 --> 01:29:10.344
Which then opens up the bandwidth on memory.

1717
01:29:10.724 --> 01:29:11.065
Exactly.

1718
01:29:11.085 --> 01:29:18.108
So you're not constantly going back to store and retrieve in time as you're doing any number of different operations or processes.

1719
01:29:18.289 --> 01:29:18.709
Yeah.

1720
01:29:18.949 --> 01:29:19.989
And the proof is in the pudding.

1721
01:29:20.009 --> 01:29:20.990
They've got a smoking gun.

1722
01:29:21.030 --> 01:29:23.251
They used a 27 billion parameter model.

1723
01:29:23.611 --> 01:29:24.632
The loss is lower.

1724
01:29:24.972 --> 01:29:26.733
The gradients aren't blowing up.

1725
01:29:28.454 --> 01:29:29.895
So it is very cool.

1726
01:29:30.335 --> 01:29:30.415
Oh.

1727
01:29:31.055 --> 01:29:33.898
But it's kind of insane that it was such a simple trick.

1728
01:29:33.918 --> 01:29:37.120
It was just like, oh, just make the matrices add up to one.

1729
01:29:37.761 --> 01:29:39.122
Just make sure it doesn't blow up.

1730
01:29:39.142 --> 01:29:41.003
Yeah, no, that was the whole point of ResNet.

1731
01:29:41.043 --> 01:29:42.624
And they just did it again.

1732
01:29:44.065 --> 01:29:45.046
It's not in the same way.

1733
01:29:45.146 --> 01:29:45.747
But like...

1734
01:29:46.563 --> 01:29:49.066
Still, you know, the concept is very similar.

1735
01:29:49.406 --> 01:30:10.126
And it's very physics intuition as well, because, you know, in quantum mechanics, for example, this reminded me a lot of quantum mechanics, because in quantum mechanics, you have, for example, quantum states, and then you have something called operators that operate on these quantum states, like a time evolution operator would take your quantum state now and tell you what happens sometime later and sometime later and sometime later, right?

1736
01:30:10.767 --> 01:30:10.947
Yeah.

1737
01:30:12.824 --> 01:30:19.768
And these operators have a condition that they have to be unitary, meaning the probability has to add up to one.

1738
01:30:20.469 --> 01:30:20.669
Okay.

1739
01:30:20.689 --> 01:30:23.251
Because all of the outcomes need to be all of the outcomes.

1740
01:30:23.571 --> 01:30:28.554
You can't have an operator that expands probability so that the sum of all probabilities is two.

1741
01:30:29.815 --> 01:30:32.576
But there's a very central physical insight, right?

1742
01:30:32.636 --> 01:30:33.877
It's like conservation of probability.

1743
01:30:33.897 --> 01:30:35.178
You also have conservation of energy.

1744
01:30:35.458 --> 01:30:38.180
In this case, they have like conservation of norm or something like that.

1745
01:30:38.200 --> 01:30:41.302
You know, it's like a very simple insight.

1746
01:30:42.085 --> 01:30:52.375
And this goes back to the way you kind of always talk about and see the world, which is like you can apply the mental model that we have as physicists into other domains.

1747
01:30:52.415 --> 01:30:52.675
Yeah.

1748
01:30:53.175 --> 01:30:57.039
And it gives you, it's like a way of thinking about stuff and systems.

1749
01:30:57.339 --> 01:30:57.539
Yeah.

1750
01:30:57.640 --> 01:30:58.520
And, you know.

1751
01:30:59.341 --> 01:31:26.512
order and progress and and it because it it's funny that we again did the prince and neuro story yeah followed up by this one because they're there it's like two different types of manifolds yeah in this case the manifold is like constrain it into the size yes right there it was like a a giant smooth sort of manifold here it's like just make it small and and the what is and what was driving the

1752
01:31:27.652 --> 01:31:35.707
you know, with the sort of deep sick story, it's, it's obviously very much focused on like a commercial outcome.

1753
01:31:35.807 --> 01:31:36.849
The insight was, uh,

1754
01:31:37.762 --> 01:31:42.167
not particularly, uh, not didn't have a huge wow fact.

1755
01:31:42.247 --> 01:31:45.010
The outcome had wow fact, not the intuition.

1756
01:31:45.050 --> 01:31:45.250
Yeah.

1757
01:31:45.691 --> 01:31:49.455
Whereas I feel like with the, the Princeton neuroscience story, like the intuition intuition.

1758
01:31:49.495 --> 01:31:49.675
Yeah.

1759
01:31:49.715 --> 01:31:50.156
That was cool.

1760
01:31:50.216 --> 01:31:51.057
It was really, really cool.

1761
01:31:51.377 --> 01:31:51.777
Yeah.

1762
01:31:52.518 --> 01:31:57.764
We, I know every, a lot of the AI community and the sort of tech community, uh,

1763
01:31:58.971 --> 01:32:03.294
None of them knew what hyper...

1764
01:32:03.374 --> 01:32:06.436
Manifold constrained hyperconnections.

1765
01:32:06.496 --> 01:32:07.397
Everyone was like, what is...

1766
01:32:07.737 --> 01:32:09.158
I don't know what manifolds are.

1767
01:32:09.178 --> 01:32:11.159
I don't know what constrained means in this context.

1768
01:32:11.199 --> 01:32:12.640
I don't know what hyperconnections are.

1769
01:32:12.660 --> 01:32:13.601
But now you do, right?

1770
01:32:13.641 --> 01:32:15.823
Hyperconnections is the mixing of the traffic.

1771
01:32:16.183 --> 01:32:23.168
The manifold constraint is to just say the thing that is mixing the traffic better not amplify the traffic or shrink the traffic.

1772
01:32:23.768 --> 01:32:25.029
The traffic better be the same.

1773
01:32:25.749 --> 01:32:27.030
This is obviously going to make models.

1774
01:32:27.571 --> 01:32:30.733
Everyone's going to start in different ways and capacities using this.

1775
01:32:30.773 --> 01:32:31.213
Exactly.

1776
01:32:31.253 --> 01:32:33.715
It's going to make models much, much better.

1777
01:32:33.875 --> 01:32:34.116
Yeah.

1778
01:32:34.436 --> 01:32:41.681
And it's also like kind of a shift in the paradigm of just like it's just compute, compute, compute, right?

1779
01:32:41.701 --> 01:32:43.983
Because right here, this is algorithmic innovation.

1780
01:32:44.003 --> 01:32:44.543
Yes.

1781
01:32:44.563 --> 01:32:46.405
It's acting as a substitute for compute.

1782
01:32:46.645 --> 01:32:49.307
You bring up a really good point because that's been the...

1783
01:32:50.167 --> 01:32:59.729
People sort of say the argument is like, oh, the Americans are just being fat and lazy or being dumb and lazy in this context because they just want to throw money at the problem.

1784
01:32:59.969 --> 01:33:03.810
And then they're not actually doing any of the variety of areas of fundamental research.

1785
01:33:03.850 --> 01:33:07.851
I mean, some people argue that you can't just scale without – this is the Jan LeCun.

1786
01:33:07.891 --> 01:33:09.412
Yeah, Jan LeCun says this a lot.

1787
01:33:09.432 --> 01:33:09.552
Yeah.

1788
01:33:11.432 --> 01:33:19.454
whatever you want to define as AGI by scaling LLMs, you need some or some subset, even Dario at Anthropoc kind of says the same thing.

1789
01:33:19.474 --> 01:33:22.675
You need some subset of other insights or breakthroughs around it.

1790
01:33:24.456 --> 01:33:31.498
But, you know, this is a perfect example of identifying that there's a lot of green space still.

1791
01:33:31.858 --> 01:33:32.539
Yeah.

1792
01:33:32.819 --> 01:33:41.545
In these areas that people might not be focusing on because the rat race around being the first AGI has people just like, what is the easiest way to get there quickly?

1793
01:33:41.625 --> 01:33:41.886
Yeah.

1794
01:33:41.946 --> 01:33:42.786
Throw money at the problem.

1795
01:33:42.826 --> 01:33:43.046
Yeah.

1796
01:33:43.667 --> 01:33:44.648
And so we don't get clever.

1797
01:33:44.988 --> 01:33:45.708
Very interesting.

1798
01:33:45.889 --> 01:33:48.330
Again, the US-China nexus is not going anywhere.

1799
01:33:48.571 --> 01:33:48.751
Nope.

1800
01:33:49.311 --> 01:33:50.012
It is what it is.

1801
01:33:50.132 --> 01:33:50.912
Yep.

1802
01:33:51.973 --> 01:33:53.815
And so we've hit Rome.

1803
01:33:54.975 --> 01:33:56.296
We went really far back in time.

1804
01:33:57.738 --> 01:34:00.860
Then we went real, real deep on the brain.

1805
01:34:00.880 --> 01:34:00.980
Yeah.

1806
01:34:02.465 --> 01:34:10.689
And then we kind of went a little bit, we went shallow seek on AI, the fake brain.

1807
01:34:11.569 --> 01:34:19.112
And now we're going to move to our last story of the day, which we have aptly titled Econophysics.

1808
01:34:19.712 --> 01:34:20.233
That's right.

1809
01:34:20.313 --> 01:34:24.214
As you can see, we are donned in our finance bro garb.

1810
01:34:24.374 --> 01:34:25.375
Yeah.

1811
01:34:25.635 --> 01:34:27.916
I got my Jackson Hole, you know.

1812
01:34:27.936 --> 01:34:28.036
Yeah.

1813
01:34:28.528 --> 01:34:30.149
Bunch of finance bros there, let me tell you.

1814
01:34:30.810 --> 01:34:31.531
Last time I went.

1815
01:34:32.351 --> 01:34:36.335
And I have no idea where this is going to go.

1816
01:34:37.736 --> 01:34:41.639
Yeah, this one, I was pleasantly surprised.

1817
01:34:41.759 --> 01:34:46.703
And it caught my eye because this was a paper in PRL, physical review letters.

1818
01:34:47.224 --> 01:34:55.250
This is the story journal that has published, you know, superconductivity, the black hole papers, Einstein, Wheeler,

1819
01:34:56.663 --> 01:35:19.210
cosmic microwave background and then a universal law about stock prices and the impact that orders have on the price of an asset and i was like what is this doing in a physics journal okay so here's the central question okay um you know when when you trade stocks

1820
01:35:20.270 --> 01:35:20.750
Stonks.

1821
01:35:20.990 --> 01:35:21.390
Stonks.

1822
01:35:21.750 --> 01:35:22.031
Yeah.

1823
01:35:22.491 --> 01:35:28.052
When you trade stocks, right, the trade itself affects the price of the asset.

1824
01:35:28.812 --> 01:35:29.052
Okay?

1825
01:35:29.513 --> 01:35:32.413
The question is how does that price change?

1826
01:35:33.334 --> 01:35:33.554
Okay?

1827
01:35:33.814 --> 01:35:41.616
There's something called the square root law, which says that if I put in an order for a certain amount—

1828
01:35:42.597 --> 01:35:48.863
The amount that the price is going to change as a response to that order is proportional to the square root of the size of my order.

1829
01:35:49.503 --> 01:35:49.763
Okay?

1830
01:35:50.304 --> 01:35:56.049
And it's sort of just been observed, but this was a paper that was trying to figure out, is it a square root?

1831
01:35:56.409 --> 01:36:02.535
Meaning, is the exponent a one-half, or is it one-fourth, or is it like 0.4, is it 0.6?

1832
01:36:03.095 --> 01:36:03.535
What is it?

1833
01:36:03.916 --> 01:36:04.156
Okay?

1834
01:36:04.516 --> 01:36:04.676
And...

1835
01:36:05.957 --> 01:36:08.818
The field of econophysics is actually quite old.

1836
01:36:09.158 --> 01:36:15.260
Okay, all the way back in 1900, there was this guy, Louis Bachelier.

1837
01:36:15.360 --> 01:36:19.921
He wrote his senior thesis, his PhD thesis, La Speculation.

1838
01:36:20.961 --> 01:36:21.621
That's my French.

1839
01:36:21.781 --> 01:36:22.622
I was going to say it.

1840
01:36:25.302 --> 01:36:30.984
But it's a physics thesis about speculation in the markets.

1841
01:36:31.952 --> 01:36:34.414
He actually predated Brownian motion.

1842
01:36:34.674 --> 01:36:35.475
Like this is 1900.

1843
01:36:35.735 --> 01:36:38.997
So this is before Einstein's 1905 paper on Brownian motion.

1844
01:36:39.017 --> 01:36:39.798
Yes.

1845
01:36:40.158 --> 01:36:41.639
Derived the diffusion equation.

1846
01:36:41.679 --> 01:36:43.420
It's a precursor to Brownian motion.

1847
01:36:43.580 --> 01:36:46.562
All of this stuff just to get rich off speculation.

1848
01:36:46.923 --> 01:36:47.163
Right.

1849
01:36:48.404 --> 01:36:48.844
We love it.

1850
01:36:49.024 --> 01:37:00.712
And that was sort of the birth of like trying to use statistical mechanics and insight from physics to try and talk about agents in a stock market, in a market with, you know.

1851
01:37:01.793 --> 01:37:03.375
people that are trying to buy and sell stuff.

1852
01:37:03.415 --> 01:37:03.595
Yes.

1853
01:37:03.915 --> 01:37:04.195
Okay?

1854
01:37:05.377 --> 01:37:24.755
In the 1990s, there was a guy, Eugene Stanley, he actually figured out something called the inverse cubic law of price returns, which means if you plot the returns that you get and the probability distribution of the returns based on, like, all of the assets everywhere, right?

1855
01:37:24.775 --> 01:37:24.875
Yeah.

1856
01:37:25.789 --> 01:37:30.471
It goes down like returns to the negative three power.

1857
01:37:30.652 --> 01:37:30.932
Okay.

1858
01:37:31.032 --> 01:37:31.312
Okay?

1859
01:37:31.392 --> 01:37:41.617
Meaning like if you want ten times the returns, the probability of getting ten times the returns is one divided by ten to the three compared to just getting the same return.

1860
01:37:41.637 --> 01:37:42.698
Does that make sense?

1861
01:37:42.718 --> 01:37:43.218
Yes, it does.

1862
01:37:43.258 --> 01:37:43.538
It does.

1863
01:37:43.758 --> 01:37:43.978
Okay.

1864
01:37:44.338 --> 01:37:50.582
So, and it's the same for like whether you wait one day, whether you wait two days, whether you wait three days.

1865
01:37:50.702 --> 01:37:51.362
It's kind of weird.

1866
01:37:51.582 --> 01:37:52.423
That is a little weird.

1867
01:37:52.463 --> 01:37:54.624
Like the probability distribution remains the same.

1868
01:37:54.825 --> 01:37:55.605
Yeah, I don't like that.

1869
01:37:55.705 --> 01:37:55.965
Okay.

1870
01:37:56.926 --> 01:38:00.669
Another one was the probability density of transaction sizes.

1871
01:38:00.729 --> 01:38:20.404
If you look at all of the transactions and the New York Stock Exchange, in the Paris Exchange, in the London Stock Exchange, and you plot the trade size on the x-axis and like how many trades happen at a certain size, at another certain size, then the exponent is negative 2.5.

1872
01:38:22.207 --> 01:38:22.467
Okay?

1873
01:38:22.667 --> 01:38:22.827
Yeah.

1874
01:38:22.847 --> 01:38:28.789
Which means that no matter what stock market you're looking at, it's the same trend.

1875
01:38:28.969 --> 01:38:29.509
I don't like that.

1876
01:38:29.569 --> 01:38:30.509
It's really weird.

1877
01:38:30.529 --> 01:38:31.069
Yeah, I don't like that.

1878
01:38:31.089 --> 01:38:31.289
Right?

1879
01:38:31.889 --> 01:38:37.531
And whenever physicists look at something like this, it reminds them of something called universality.

1880
01:38:37.971 --> 01:38:38.191
Okay?

1881
01:38:38.251 --> 01:38:42.292
Universality means there's some weird underlying physics that's happening.

1882
01:38:42.452 --> 01:38:42.652
Yes.

1883
01:38:43.452 --> 01:38:47.013
That is invariant of the thing that is...

1884
01:38:47.782 --> 01:38:51.183
involved in the data collection.

1885
01:38:51.584 --> 01:38:52.104
Here's what I mean.

1886
01:38:52.344 --> 01:38:56.526
Like in universality, what we a lot of times think about is phase transitions.

1887
01:38:56.946 --> 01:39:11.311
So if you think about gases or any substance that's going from liquid to gas, and it's at something called the critical point, which is where the liquid and the gas phase are like sort of same, same, right?

1888
01:39:12.932 --> 01:39:23.481
And you plot something like the specific heat of this substance, okay, as you change some parameter, like temperature or density or something like that.

1889
01:39:24.562 --> 01:39:36.413
Regardless of whether you're using helium or oxygen or this or that, the shape of that curve is going to be exactly the same, which means there's some underlying physics that is invariant of...

1890
01:39:37.391 --> 01:39:40.434
The actual mechanics of the particles.

1891
01:39:40.474 --> 01:39:41.715
It goes itself.

1892
01:39:41.775 --> 01:39:41.915
Yeah.

1893
01:39:41.935 --> 01:39:43.276
There's something much deeper.

1894
01:39:43.336 --> 01:39:43.476
Yes.

1895
01:39:43.696 --> 01:39:43.956
Okay?

1896
01:39:43.976 --> 01:39:46.839
So whenever we see stuff like that, we're like, there's something.

1897
01:39:47.359 --> 01:39:47.739
Okay?

1898
01:39:47.839 --> 01:39:50.462
And that's, I think, why this made it to PRL.

1899
01:39:50.722 --> 01:39:51.162
That makes sense.

1900
01:39:51.182 --> 01:39:51.442
Okay?

1901
01:39:51.602 --> 01:39:51.943
That makes sense.

1902
01:39:52.263 --> 01:39:54.104
I think that's why this made it to the physical review.

1903
01:39:54.485 --> 01:39:54.705
Okay.

1904
01:39:55.085 --> 01:40:01.490
So the empirical law is that square root law.

1905
01:40:01.811 --> 01:40:02.051
Okay?

1906
01:40:02.591 --> 01:40:03.972
It's that if I...

1907
01:40:05.405 --> 01:40:16.051
I put in a buy order of some size, let's say X, and then I put in and then somebody else puts in a buy order that's four times, right?

1908
01:40:16.491 --> 01:40:18.833
My buy order is going to move the price sum amount.

1909
01:40:20.089 --> 01:40:23.291
But their buy order is not going to move the price by four times.

1910
01:40:23.511 --> 01:40:26.212
It's only going to move the price by twice what mine moved.

1911
01:40:26.412 --> 01:40:26.712
Okay.

1912
01:40:26.892 --> 01:40:29.634
Because they did four and the square root of four is two.

1913
01:40:29.954 --> 01:40:34.256
If somebody put in nine times by my order, it would only move the price by three.

1914
01:40:34.876 --> 01:40:47.122
And it kind of makes sense because, you know, if some asset is like trading at $10 and then I put in like I want to buy something at like $10.1.

1915
01:40:48.847 --> 01:40:49.067
Right.

1916
01:40:49.967 --> 01:40:52.428
There's not going to be a lot of people who are willing to sell it to me.

1917
01:40:53.208 --> 01:40:55.589
So the price is really going to go to 10.1.

1918
01:40:56.029 --> 01:40:56.209
Right.

1919
01:40:56.549 --> 01:41:03.431
But if I if I put in an order for like I want to buy like a bunch, a bunch at like 12 or something.

1920
01:41:03.852 --> 01:41:04.052
Right.

1921
01:41:04.272 --> 01:41:07.313
There's going to be a lot of sellers that are going to be really keen.

1922
01:41:07.593 --> 01:41:11.654
And so the price is not going to move that much because a bunch of sellers just like fill that order.

1923
01:41:11.734 --> 01:41:11.934
Yes.

1924
01:41:12.094 --> 01:41:12.254
Right.

1925
01:41:12.494 --> 01:41:14.755
So that's the intuition behind it.

1926
01:41:14.775 --> 01:41:15.115
Okay.

1927
01:41:15.155 --> 01:41:18.639
The idea is that it's nonlinear in that sense, right?

1928
01:41:19.480 --> 01:41:23.786
The more volume that I want to buy or sell, the less the price is going to get impacted.

1929
01:41:23.826 --> 01:41:24.507
It's not linear.

1930
01:41:24.567 --> 01:41:26.869
Linear would mean that it's directly proportional.

1931
01:41:26.889 --> 01:41:29.252
Here, it's like a diminishing return type thing.

1932
01:41:29.333 --> 01:41:29.493
Yes.

1933
01:41:30.154 --> 01:41:30.414
Okay.

1934
01:41:31.095 --> 01:41:31.335
So...

1935
01:41:32.624 --> 01:41:34.685
The question is, is this truly universal, right?

1936
01:41:35.025 --> 01:41:35.285
Yes.

1937
01:41:35.445 --> 01:41:39.246
Now, in order to actually measure something like this, right?

1938
01:41:39.946 --> 01:41:43.408
I mean, there's a lot of people who think that it's not totally universal, right?

1939
01:41:43.508 --> 01:41:45.308
Maybe it depends on the microstructure.

1940
01:41:45.368 --> 01:41:50.250
Like, there's inventory risk models, like how much inventory is out there in the market.

1941
01:41:50.490 --> 01:41:55.512
Like, if there's large traders, like what if there's like a giant whale, you know, when we talk about Bitcoin whales?

1942
01:41:55.532 --> 01:41:56.132
Yeah, yeah, yeah.

1943
01:41:56.892 --> 01:42:01.415
If there's a market where there's only big orders, right?

1944
01:42:01.476 --> 01:42:02.897
No one's buying, like, small stuff.

1945
01:42:02.977 --> 01:42:03.157
Right.

1946
01:42:03.297 --> 01:42:05.118
And it doesn't even matter if you're buying small stuff.

1947
01:42:05.478 --> 01:42:08.120
Then does that change the dynamics of the square root law, right?

1948
01:42:08.681 --> 01:42:09.561
Okay.

1949
01:42:09.601 --> 01:42:10.202
Which makes sense.

1950
01:42:10.222 --> 01:42:11.103
Which totally makes sense.

1951
01:42:11.163 --> 01:42:15.566
But the universality hypothesis says that no, it's still 0.5.

1952
01:42:15.586 --> 01:42:17.247
Regardless of any of those.

1953
01:42:17.267 --> 01:42:19.028
The exponent is always going to be square root.

1954
01:42:19.709 --> 01:42:19.929
Okay.

1955
01:42:19.949 --> 01:42:20.109
Okay.

1956
01:42:20.409 --> 01:42:21.951
So, um...

1957
01:42:23.047 --> 01:42:29.210
To discern between whether it's one or the other, you have to get some kind of very good data, okay?

1958
01:42:29.230 --> 01:42:33.771
Because normally when we want to analyze data, there's these things called meta orders, right?

1959
01:42:34.192 --> 01:42:38.413
Like there's a hidden parent order, and then there's anonymous like child orders.

1960
01:42:38.433 --> 01:42:42.595
Like if I want to buy like 10 stocks of a particular asset...

1961
01:42:43.840 --> 01:42:49.907
I could just put in an order for 10 or I could be like, okay, I want to buy two and then I'm going to buy three and then I'm going to buy two and then I'm going to buy three.

1962
01:42:50.227 --> 01:42:54.613
That's four different little child orders for one big meta order.

1963
01:42:54.633 --> 01:42:54.793
Right.

1964
01:42:54.873 --> 01:42:55.734
Which is that you bought 10.

1965
01:42:55.894 --> 01:42:56.154
Yeah.

1966
01:42:56.234 --> 01:42:56.735
Yeah.

1967
01:42:56.875 --> 01:43:03.343
And in normal sort of trading in the data that you get if you want to analyze this kind of stuff.

1968
01:43:03.763 --> 01:43:05.544
You only have access to the little orders.

1969
01:43:05.584 --> 01:43:07.806
You don't have access to the guy who's, right?

1970
01:43:08.046 --> 01:43:13.570
Because each of these people have like a virtual server.

1971
01:43:13.730 --> 01:43:17.693
So each of the little orders might have like a virtual server ID.

1972
01:43:18.034 --> 01:43:22.097
And so it's going to be like pretty impossible to reconstruct who did which one, right?

1973
01:43:22.177 --> 01:43:22.357
Right.

1974
01:43:22.637 --> 01:43:22.877
Okay.

1975
01:43:23.158 --> 01:43:26.280
So this is where the Sato Kanazawa study came.

1976
01:43:26.560 --> 01:43:31.882
that was just published in PRL in 2025, and December 2025 comes in.

1977
01:43:31.982 --> 01:43:33.522
Strict universality.

1978
01:43:34.002 --> 01:43:39.184
They show that this is a very strict universality, and it's a complete survey of the Tokyo Stock Exchange.

1979
01:43:39.504 --> 01:43:47.467
So the Tokyo Stock Exchange, what they did was they got a bunch of data from that stock exchange for eight years, so from 2012 all the way to 2019.

1980
01:43:49.480 --> 01:44:00.068
And the key innovation is they figured out a way to stitch together the effective trader IDs, so the big meta orders, from these virtual server IDs.

1981
01:44:00.328 --> 01:44:01.649
They kind of de-anonymized it.

1982
01:44:01.849 --> 01:44:02.169
Yes.

1983
01:44:02.630 --> 01:44:02.850
Yeah.

1984
01:44:02.990 --> 01:44:06.953
It's still anonymous, but they can pool them together directly.

1985
01:44:07.113 --> 01:44:09.714
to recognize a single trader.

1986
01:44:10.014 --> 01:44:13.595
And the way they did it, the reconstruction method is actually pretty cool.

1987
01:44:14.175 --> 01:44:14.915
So here's what they did.

1988
01:44:14.955 --> 01:44:29.899
They developed a heuristic that said, okay, I'm going to aggregate IDs such that if server A submits an order and then server B cancels that order, the only way that's possible is if both of these servers are connected to the same trader.

1989
01:44:31.140 --> 01:44:32.520
So every other time...

1990
01:44:33.740 --> 01:44:37.061
I'm going to treat that, right, as it's pretty good.

1991
01:44:37.121 --> 01:44:38.262
That's really good.

1992
01:44:38.862 --> 01:44:41.023
And so you get these effective trader IDs.

1993
01:44:41.043 --> 01:44:44.364
You get the trader desk, right, in these, like, firms.

1994
01:44:44.404 --> 01:44:44.704
Yes.

1995
01:44:44.944 --> 01:44:45.165
Right?

1996
01:44:45.745 --> 01:44:55.349
And this allows them to analyze the behavior of all the active traders across all liquid stocks, all 2,000 stocks for nearly a decade.

1997
01:44:55.369 --> 01:44:56.329
It's like a God view.

1998
01:44:57.490 --> 01:44:58.111
Of the market.

1999
01:44:58.251 --> 01:44:58.911
I love that.

2000
01:44:59.011 --> 01:45:00.412
Of the Tokyo Stock Exchange, right?

2001
01:45:00.573 --> 01:45:00.813
Yeah.

2002
01:45:01.233 --> 01:45:01.433
Yeah.

2003
01:45:01.854 --> 01:45:02.574
Very cool.

2004
01:45:02.794 --> 01:45:03.315
And that's clever.

2005
01:45:03.335 --> 01:45:08.139
And again, it doesn't take a large leap to get that connection.

2006
01:45:08.159 --> 01:45:08.879
Yeah, yeah, yeah.

2007
01:45:09.120 --> 01:45:10.661
But they did the due diligence, right?

2008
01:45:10.681 --> 01:45:10.901
Yeah, yeah.

2009
01:45:10.921 --> 01:45:11.702
And they actually did it.

2010
01:45:11.762 --> 01:45:12.062
Yeah, yeah.

2011
01:45:12.122 --> 01:45:16.986
And so when they actually go through and do the math, they find that the mean exponent...

2012
01:45:18.512 --> 01:45:22.015
is about 0.489, very close to 0.5.

2013
01:45:22.615 --> 01:45:25.117
The standard deviation is 0.071.

2014
01:45:25.497 --> 01:45:28.059
And there on the left, you can see all of the different stocks they use.

2015
01:45:28.099 --> 01:45:32.122
Toyota, NTT, I don't know that stock.

2016
01:45:32.422 --> 01:45:36.084
But all of them are the same line.

2017
01:45:36.164 --> 01:45:36.345
Yes.

2018
01:45:36.365 --> 01:45:36.945
Yes.

2019
01:45:37.065 --> 01:45:38.606
And they're all centered at 0.5.

2020
01:45:38.626 --> 01:45:42.628
And if you look across all 2,000 stocks, the spread is quite big, I'll admit.

2021
01:45:42.648 --> 01:45:44.949
It's like from 0.4 all the way to 0.6.

2022
01:45:45.369 --> 01:45:48.151
But the mean of that spread is very sharply at 0.5.

2023
01:45:48.251 --> 01:45:48.631
Okay?

2024
01:45:48.651 --> 01:45:49.831
Mm-hmm.

2025
01:45:50.712 --> 01:45:50.892
So—

2026
01:45:52.425 --> 01:45:55.826
From this data set, they can actually falsify some of the earlier models.

2027
01:45:55.866 --> 01:46:04.348
There's this one model called the GGPS model, which is the, I don't know, GoBikes, Gopi Krishnan, Pluro, Stanley model from 2003.

2028
01:46:04.368 --> 01:46:06.968
This is the model of inventory risk.

2029
01:46:07.008 --> 01:46:12.109
This is the one that's saying that, like, okay, market makers, they provide liquidity, but they face the risk, right?

2030
01:46:12.149 --> 01:46:19.831
Because, like, if I'm a market maker, meaning, like, I have, like, I'm the one, if you're trying to sell, I'll buy from you, and if you're trying to buy, I'm going to,

2031
01:46:21.298 --> 01:46:24.679
I'll sell to you and I'm going to make money on the spread.

2032
01:46:24.719 --> 01:46:24.919
Yes.

2033
01:46:25.079 --> 01:46:30.741
But if the price like goes off, then I'm just left with the bag, the bag.

2034
01:46:30.781 --> 01:46:31.021
Right.

2035
01:46:31.081 --> 01:46:33.202
And I, that I didn't want, or maybe I did want.

2036
01:46:33.742 --> 01:46:33.982
Okay.

2037
01:46:34.262 --> 01:46:38.764
So they charge a premium and that price impact is going to compensate for that risk.

2038
01:46:39.664 --> 01:46:40.164
Okay.

2039
01:46:40.204 --> 01:46:47.627
So what if, what if this stuff's and the scaling argument is the size of the trades follows a power law, but to limit their risk,

2040
01:46:48.822 --> 01:46:53.063
They're going to adjust the price to balance the probability of large orders.

2041
01:46:53.863 --> 01:46:54.123
Okay?

2042
01:46:54.683 --> 01:47:02.245
And there's another sort of numerical coefficient that comes in beta that you can figure out, which is sort of the probability of these large orders.

2043
01:47:02.565 --> 01:47:11.627
And they say that the alpha, which is the exponent of my price impact, the one half that we're talking about here, the square root, is related to this beta.

2044
01:47:12.447 --> 01:47:15.368
So these guys plotted those two, and there's no correlation.

2045
01:47:16.128 --> 01:47:16.548
It's flat.

2046
01:47:16.568 --> 01:47:16.908
It's flat.

2047
01:47:18.933 --> 01:47:19.174
Okay?

2048
01:47:19.515 --> 01:47:22.865
So this 2003 model, this GGPS model, completely wrong.

2049
01:47:22.965 --> 01:47:23.386
Nonsense.

2050
01:47:23.467 --> 01:47:23.667
Yeah.

2051
01:47:24.269 --> 01:47:24.369
Okay?

2052
01:47:24.389 --> 01:47:24.530
Okay?

2053
01:47:25.345 --> 01:47:27.266
Like most things in economics.

2054
01:47:27.466 --> 01:47:29.326
But okay.

2055
01:47:29.346 --> 01:47:29.667
You know?

2056
01:47:29.687 --> 01:47:31.387
There's something wrong with the mic.

2057
01:47:32.288 --> 01:47:33.528
So zero correlation there.

2058
01:47:33.548 --> 01:47:36.829
There was also another fair pricing efficiency model.

2059
01:47:36.849 --> 01:47:37.630
That's crazy.

2060
01:47:37.670 --> 01:47:37.830
Yeah.

2061
01:47:38.050 --> 01:47:38.270
Yeah.

2062
01:47:39.010 --> 01:47:39.870
Zero correlation.

2063
01:47:40.110 --> 01:47:40.291
Right.

2064
01:47:40.551 --> 01:47:40.731
Yeah.

2065
01:47:40.991 --> 01:47:46.953
So there's this FGLW model, which is the Farmer Gehrig Lilo Weiland Broic model.

2066
01:47:47.533 --> 01:47:49.214
That also was falsified.

2067
01:47:49.234 --> 01:47:50.134
I'm not going to get into that.

2068
01:47:50.394 --> 01:47:52.335
But it's about market efficiency and fair pricing.

2069
01:47:53.496 --> 01:47:58.562
So I was reading this and I was like, okay, I would like to know mechanistically, though, what's going on.

2070
01:47:58.803 --> 01:47:58.983
Right.

2071
01:47:59.363 --> 01:47:59.664
Okay?

2072
01:47:59.784 --> 01:48:00.444
I'm a physicist.

2073
01:48:00.465 --> 01:48:01.846
It's like, what's actually going on?

2074
01:48:01.906 --> 01:48:02.086
Right.

2075
01:48:02.547 --> 01:48:02.727
Right?

2076
01:48:02.867 --> 01:48:04.329
Because there's clearly some underlying...

2077
01:48:04.509 --> 01:48:04.750
Yes.

2078
01:48:04.890 --> 01:48:06.211
The underlying physics is there.

2079
01:48:06.252 --> 01:48:06.712
That's there.

2080
01:48:06.732 --> 01:48:07.793
There's some explanation.

2081
01:48:07.813 --> 01:48:09.175
That's none of these other things.

2082
01:48:09.215 --> 01:48:09.435
Yes.

2083
01:48:09.576 --> 01:48:09.816
Okay.

2084
01:48:10.136 --> 01:48:10.376
So...

2085
01:48:10.957 --> 01:48:14.058
The victor comes from something called latent order book models.

2086
01:48:14.098 --> 01:48:19.739
This is from Bouchard, Donier, and Mastro Mateo in 2015.

2087
01:48:20.039 --> 01:48:27.001
Bouchard actually holds a PhD in theoretical physics from ENS, which is like the top university in Paris.

2088
01:48:27.721 --> 01:48:33.462
And his analogy is reaction diffusion in chemical reactions.

2089
01:48:33.802 --> 01:48:35.943
Imagine you've got two different types of particles.

2090
01:48:36.383 --> 01:48:38.684
You've got a bi-particle and you've got a cell particle.

2091
01:48:38.764 --> 01:48:40.404
If they meet up, they annihilate.

2092
01:48:41.224 --> 01:48:41.425
Right.

2093
01:48:41.505 --> 01:48:44.928
So they they they they poof out of existence.

2094
01:48:45.008 --> 01:48:45.208
Right.

2095
01:48:46.749 --> 01:48:50.332
The bi particles are on one side and the cell particles are on the other side.

2096
01:48:50.652 --> 01:48:52.514
OK. And they all want to get to each other.

2097
01:48:53.034 --> 01:48:57.538
But because of the nature of the reaction in the middle, there's not going to be any.

2098
01:48:57.939 --> 01:49:00.201
Because if they were in the middle, they would annihilate.

2099
01:49:00.321 --> 01:49:00.521
Right.

2100
01:49:00.881 --> 01:49:04.784
So you're going to get a density profile that is roughly linear.

2101
01:49:04.804 --> 01:49:04.925
Right.

2102
01:49:05.873 --> 01:49:07.554
On the right-hand side and the left-hand side.

2103
01:49:07.574 --> 01:49:13.276
And I think the next picture shows this very well, which is the density profile being very much linear.

2104
01:49:13.576 --> 01:49:19.238
So you're asking for a certain price above the current price, and that's in the red.

2105
01:49:19.638 --> 01:49:22.620
So this is like how much I would sell for.

2106
01:49:22.900 --> 01:49:23.060
Yes.

2107
01:49:23.380 --> 01:49:25.421
And then in the blue is how much I would buy for.

2108
01:49:25.441 --> 01:49:25.821
Mm-hmm.

2109
01:49:26.321 --> 01:49:28.283
And it's roughly linear, right?

2110
01:49:28.923 --> 01:49:29.184
Okay.

2111
01:49:29.624 --> 01:49:36.871
So now imagine with this linear sort of density profile of my order book.

2112
01:49:37.291 --> 01:49:38.352
These are the order books, right?

2113
01:49:39.353 --> 01:49:43.436
Suppose some order of size Q comes in.

2114
01:49:44.557 --> 01:49:44.838
Okay?

2115
01:49:45.518 --> 01:49:48.581
How much of it is going to go away?

2116
01:49:49.101 --> 01:49:50.343
It's going to be that triangle.

2117
01:49:50.363 --> 01:49:50.503
Okay?

2118
01:49:51.318 --> 01:49:51.558
Right.

2119
01:49:52.278 --> 01:49:53.899
And what's the area of the triangle?

2120
01:49:53.919 --> 01:49:59.841
The area of the triangle is one half times the length and the width.

2121
01:49:59.901 --> 01:50:00.121
Right.

2122
01:50:00.161 --> 01:50:04.062
So it's the total size is my area.

2123
01:50:04.702 --> 01:50:08.123
And that's proportional to the price impact squared.

2124
01:50:10.764 --> 01:50:15.286
So the price impact is proportional to the area square root.

2125
01:50:15.986 --> 01:50:16.186
Yep.

2126
01:50:17.978 --> 01:50:19.040
It's strictly universal.

2127
01:50:19.220 --> 01:50:20.944
It's like it makes sense.

2128
01:50:21.064 --> 01:50:24.731
It has to do with the shape of the order book.

2129
01:50:25.072 --> 01:50:25.312
Right.

2130
01:50:25.512 --> 01:50:25.713
Right.

2131
01:50:25.813 --> 01:50:28.218
The area is the price impact squared.

2132
01:50:28.238 --> 01:50:28.318
Yes.

2133
01:50:28.398 --> 01:50:28.598
Yes.

2134
01:50:29.758 --> 01:50:31.999
And so the price impact is the area square root.

2135
01:50:32.539 --> 01:50:34.619
And the area is the size of your order.

2136
01:50:35.139 --> 01:50:36.000
So can we all go home?

2137
01:50:36.460 --> 01:50:37.380
It's, like, pretty simple.

2138
01:50:37.960 --> 01:50:39.581
I thought that was, like, kind of cute.

2139
01:50:39.641 --> 01:50:45.062
No, that is – because it – you know, there's always a lot of hand-waving.

2140
01:50:45.242 --> 01:50:45.582
Yeah.

2141
01:50:45.702 --> 01:50:49.383
The thing I love about this is, like, no, but – Yeah, how does it work, though?

2142
01:50:49.423 --> 01:50:49.923
Show me.

2143
01:50:49.963 --> 01:50:50.403
Like, yeah.

2144
01:50:50.583 --> 01:50:52.684
Don't – well, like, we have to, you know –

2145
01:50:53.224 --> 01:50:54.225
some risk of that.

2146
01:50:54.325 --> 01:50:56.566
Yeah, no, it's like, no, just, oh, it's okay.

2147
01:50:56.867 --> 01:50:57.687
It's just an area.

2148
01:50:58.428 --> 01:51:04.332
That's why you guys don't want to talk about it, because you're trying to take your premiums, because you know, anyway.

2149
01:51:05.173 --> 01:51:07.354
This is actually really cool, econophysics.

2150
01:51:07.454 --> 01:51:11.797
Yeah, and it makes sense now that it was in PRL.

2151
01:51:11.877 --> 01:51:19.163
It's a pretty cool study, like the way that they batched the orders together to figure out what the trading desk was doing, things like that.

2152
01:51:19.243 --> 01:51:20.564
I think it was a really cool study.

2153
01:51:21.587 --> 01:51:46.258
um you know the practical impact i don't know somebody's probably going to make billions in slippage well yeah based on this or it's probably going to be priced in because now this is now that this it's it probably already is priced in because like the economists don't actually care about like the the finance guys don't actually care about the underlying like oh why is it doing it's like oh it's already been an empirical observation so they're gonna put it in their algorithms anyway so you know i i don't know if like

2154
01:51:47.018 --> 01:51:50.603
You're going to make a lot of money off this insight, but it's a cool insight nonetheless.

2155
01:51:50.883 --> 01:51:52.686
This is not financial advice.

2156
01:51:52.806 --> 01:51:53.026
No.

2157
01:51:53.207 --> 01:51:54.328
Yeah.

2158
01:51:54.348 --> 01:51:55.610
Don't go do experiments.

2159
01:51:55.630 --> 01:51:57.212
Hey, I wonder if I buy 100 Bitcoin.

2160
01:51:57.252 --> 01:51:58.434
What's going to happen?

2161
01:51:59.215 --> 01:52:01.218
We're just some physics nerds over here.

2162
01:52:01.298 --> 01:52:01.618
Yeah.

2163
01:52:01.678 --> 01:52:02.059
Yeah.

2164
01:52:02.219 --> 01:52:04.061
So don't come cry.

2165
01:52:05.964 --> 01:52:08.107
Because I think you can find there's structure all around us.

2166
01:52:08.308 --> 01:52:09.389
Yeah, exactly.

2167
01:52:09.449 --> 01:52:10.110
Yes, very good.

2168
01:52:10.130 --> 01:52:12.934
There's structure all around us.

2169
01:52:13.215 --> 01:52:19.444
It's accessible, but sometimes it may be a little opaque or discreet.

2170
01:52:20.265 --> 01:52:23.886
In terms of how we engage and move around in the world.

2171
01:52:23.926 --> 01:52:24.286
Yeah, yeah.

2172
01:52:24.546 --> 01:52:25.586
And then how it becomes obvious.

2173
01:52:26.086 --> 01:52:26.847
That's a very good point.

2174
01:52:26.867 --> 01:52:28.227
There is structure all around us.

2175
01:52:28.307 --> 01:52:28.967
I like that.

2176
01:52:29.207 --> 01:52:31.628
I mean, they're all of our stories.

2177
01:52:31.648 --> 01:52:33.708
We started with concrete, right?

2178
01:52:34.409 --> 01:52:36.009
And that's pretty good with structure.

2179
01:52:36.049 --> 01:52:38.550
Then we went to cognitive Legos and the structure of manifolds.

2180
01:52:38.590 --> 01:52:39.990
Yeah, in our brain.

2181
01:52:40.010 --> 01:52:41.630
In our brain and biological beings.

2182
01:52:42.111 --> 01:52:46.992
Then we went to the similar kind of structure for manifolds and constrained hyperconnections.

2183
01:52:48.132 --> 01:53:05.404
in the the human brain we're trying to replicate in silica and then the structure of this whole thing people view as like you need to be galaxy brain to beat the markets and all of this it's like it's actually just it's just a area of a triangle

2184
01:53:08.563 --> 01:53:10.404
We hope you guys are excited.

2185
01:53:10.825 --> 01:53:13.407
This season two, we have a lot cooking.

2186
01:53:13.887 --> 01:53:17.610
As you can tell, we've tried to do some updates and upgrades.

2187
01:53:18.151 --> 01:53:21.854
Not a perfect execution, but we've learned a lot in this first episode.

2188
01:53:22.294 --> 01:53:30.101
We are trying to do longer overlays now with better viewability because we do know many of you want to actually digest the graphics and see it.

2189
01:53:30.461 --> 01:53:33.364
For our listeners who are listening on the pod, we are a video podcast.

2190
01:53:33.384 --> 01:53:34.465
We're available on Spotify.

2191
01:53:34.505 --> 01:53:35.706
We're available on Spotify.

2192
01:53:36.086 --> 01:53:40.549
We're available on YouTube for our full-length episodes and clips all over the socials.

2193
01:53:40.990 --> 01:53:47.474
If you really want to kind of grok what we're talking through, especially some of the stories in today's episode, the graphics are super, super helpful.

2194
01:53:47.695 --> 01:53:53.139
Christian, you always do a good job of trying to verbalize what's being seen, but there's so much detail that's lost in it.

2195
01:53:53.839 --> 01:53:59.661
And I did mean to bring this up at the beginning of the episode, but there were too many new variables that I was worried about that I forgot to.

2196
01:54:00.021 --> 01:54:03.322
However, if you made it to the end, you get to know this before anybody else.

2197
01:54:03.862 --> 01:54:08.703
Our new website has just gone live and there's a sort of a couple of interesting things.

2198
01:54:09.044 --> 01:54:11.564
So obviously, if you want to support the pod, you can go there and do so.

2199
01:54:11.584 --> 01:54:12.845
But all of our episodes are there.

2200
01:54:13.605 --> 01:54:13.805
And...

2201
01:54:14.165 --> 01:54:17.308
We are backlogging all of the research papers for season one.

2202
01:54:17.648 --> 01:54:25.395
And on an ongoing basis, we'll be putting all of the research papers with key takeaways, the related episodes, all in one place.

2203
01:54:25.555 --> 01:54:31.480
So you can listen to us while you're on your commute to work or while you're in the middle of the gym workout.

2204
01:54:31.761 --> 01:54:35.023
And if you're like, I actually am really kind of, I kind of really picked my brain.

2205
01:54:35.043 --> 01:54:36.164
I want to dig in more.

2206
01:54:36.765 --> 01:54:43.751
You have an easy access to the source, the sort of nature paper that it's from or the journal, the publisher, impact factor.

2207
01:54:44.131 --> 01:54:51.837
We have some really, really cool ideas for how we can start to do some interesting things with the leaderboard and rankings.

2208
01:54:51.897 --> 01:54:52.818
I won't leak too much.

2209
01:54:53.018 --> 01:54:54.539
Yeah.

2210
01:54:54.719 --> 01:54:58.943
We also did get confirmation for our first on-site podcast.

2211
01:55:00.728 --> 01:55:01.588
I'll zip the lip.

2212
01:55:01.648 --> 01:55:02.189
Yeah, yeah, yeah.

2213
01:55:02.229 --> 01:55:02.569
That's it.

2214
01:55:03.209 --> 01:55:03.829
It's all you get.

2215
01:55:04.029 --> 01:55:06.130
But you can only get it if you go to the end of the podcast.

2216
01:55:06.470 --> 01:55:07.711
Or maybe we'll sprinkle it in the middle.

2217
01:55:07.771 --> 01:55:08.151
Who knows?

2218
01:55:08.211 --> 01:55:10.452
So you got to listen because you never know what you're going to get.

2219
01:55:11.593 --> 01:55:14.334
This was a good, long, two-hour epic.

2220
01:55:14.634 --> 01:55:15.194
Oh, wait.

2221
01:55:15.214 --> 01:55:18.736
We need to tell the listeners if they made it this far what to comment.

2222
01:55:19.016 --> 01:55:19.376
Ah, yes.

2223
01:55:19.756 --> 01:55:19.996
Yes.

2224
01:55:20.396 --> 01:55:20.737
Okay.

2225
01:55:20.757 --> 01:55:21.217
This is good.

2226
01:55:21.737 --> 01:55:24.678
If you have an idea, I'll let you cook on it.

2227
01:55:25.118 --> 01:55:26.019
I don't know if I...

2228
01:55:28.078 --> 01:55:34.781
Yeah, if you guys write down finance and physics.

2229
01:55:35.741 --> 01:55:36.462
No, that isn't.

2230
01:55:36.542 --> 01:55:37.042
No, no.

2231
01:55:37.062 --> 01:55:37.542
Don't do that.

2232
01:55:37.562 --> 01:55:38.042
Don't do that.

2233
01:55:38.202 --> 01:55:38.883
That's so bad.

2234
01:55:38.903 --> 01:55:39.803
We're going to edit that out.

2235
01:55:43.282 --> 01:55:44.283
This is why I don't do that.

2236
01:55:44.303 --> 01:55:47.466
So I don't know why you gave me the fucking, you know, like.

2237
01:55:48.868 --> 01:55:49.929
You're the promo guy.

2238
01:55:49.949 --> 01:55:50.449
Come on.

2239
01:55:50.469 --> 01:55:52.571
Dude, dude.

2240
01:55:54.493 --> 01:55:56.655
My brain, you fried my brain so hard today.

2241
01:55:56.775 --> 01:55:56.995
Yeah.

2242
01:55:57.096 --> 01:56:01.079
We've been working like really hard trying to get all of this set up for this season.

2243
01:56:01.160 --> 01:56:02.961
So maybe let's do this.

2244
01:56:05.404 --> 01:56:06.905
There's structure all around us.

2245
01:56:07.265 --> 01:56:07.546
Yeah.

2246
01:56:08.585 --> 01:56:10.566
And a pun on that or something like that.

2247
01:56:10.586 --> 01:56:11.446
Because last time...

2248
01:56:12.307 --> 01:56:13.587
There were some comedians.

2249
01:56:13.647 --> 01:56:15.708
Some of y'all were hilarious.

2250
01:56:15.968 --> 01:56:19.190
We still have to put a compilation for that feedback.

2251
01:56:19.210 --> 01:56:21.751
But there's structure all around us or something like that.

2252
01:56:21.771 --> 01:56:23.152
Or something along those lines.

2253
01:56:23.352 --> 01:56:24.913
The last thing I'll say is...

2254
01:56:26.073 --> 01:56:32.018
We are hoping to, in short order, put a submissions page on that new website that I mentioned.

2255
01:56:32.338 --> 01:56:45.328
So if you have a peer-reviewed research paper in a journal that has an impact factor that is greater than we will tell you soon, but it's going to be pretty high just because we have to kind of keep things there.

2256
01:56:45.348 --> 01:56:49.311
Please send it to our submission page, or if there's just something you're interested in, send it there too.

2257
01:56:49.612 --> 01:56:51.133
If it is not in a journal, please.

2258
01:56:51.833 --> 01:56:54.316
that has a significantly high impact factor.

2259
01:56:54.696 --> 01:56:55.637
We will not cover it.

2260
01:56:55.737 --> 01:56:56.718
It is not personal.

2261
01:56:57.058 --> 01:56:58.039
It may be amazing.

2262
01:56:58.259 --> 01:57:03.423
And if it is amazing, it will ultimately end up in one of those journals and we will ultimately cover it.

2263
01:57:03.563 --> 01:57:04.624
We are just the two of us.

2264
01:57:04.644 --> 01:57:05.445
We're trying to do what we can.

2265
01:57:05.465 --> 01:57:07.026
But we know so many of you send us stuff.

2266
01:57:07.347 --> 01:57:12.431
We're trying to start the process of creating an ecosystem and community to begin to do so.

2267
01:57:12.951 --> 01:57:13.211
So...

2268
01:57:14.012 --> 01:57:16.073
A lot of fun, exciting things.

2269
01:57:16.433 --> 01:57:16.753
Go ahead.

2270
01:57:16.834 --> 01:57:27.059
Yeah, and I was just going to say, like, the one thing that I was going to say with the impact factor stuff is, like, we cover papers from even, like, lower impact factor.

2271
01:57:28.165 --> 01:57:30.147
To me, peer-reviewed journals.

2272
01:57:30.708 --> 01:57:31.669
Okay?

2273
01:57:31.749 --> 01:57:34.393
It's got to be peer-reviewed.

2274
01:57:34.693 --> 01:57:35.794
We need reviewer number two.

2275
01:57:35.814 --> 01:57:36.675
Yeah, yeah.

2276
01:57:37.116 --> 01:57:38.317
Archive doesn't work, guys.

2277
01:57:38.337 --> 01:57:39.278
Yeah.

2278
01:57:39.299 --> 01:57:39.679
Sorry.

2279
01:57:39.819 --> 01:57:41.101
It's not today.

2280
01:57:41.441 --> 01:57:42.122
Yeah.

2281
01:57:42.402 --> 01:57:44.445
But, hey, if you support the pod and help us grow...

2282
01:57:44.745 --> 01:57:45.726
Maybe one day.

2283
01:57:45.786 --> 01:57:46.046
Yeah.

2284
01:57:46.286 --> 01:57:47.087
We'll expand.

2285
01:57:47.107 --> 01:57:52.211
We'll have a big board behind us where we can get the infographics from the infographic team.

2286
01:57:52.251 --> 01:57:55.114
And we'll have the 3D animations like MKBHC.

2287
01:57:55.134 --> 01:57:57.916
We'll be scaled down to the nanoscale.

2288
01:57:59.117 --> 01:58:02.000
Also, we have a blank wall behind us now because of the new angle.

2289
01:58:02.440 --> 01:58:04.302
If you have anything you would like to put on the wall...

2290
01:58:04.742 --> 01:58:05.443
please DM us.

2291
01:58:06.143 --> 01:58:14.209
As always, I am your host, Lester Nare, joined by the one, the only, our resident PhD, Krishna Chowdhury.

2292
01:58:14.750 --> 01:58:17.132
We really appreciate you guys joining us and coming back.

2293
01:58:17.172 --> 01:58:22.756
And if you're a new fan who heard about us on Rogan, you're in for a good treat.

2294
01:58:23.276 --> 01:58:25.598
This is From First Principles.
