WEBVTT

1
00:00:00.211 --> 00:00:06.756
It's called wetware engineering, and it's effectively neurons, human neurons, plus electronics.

2
00:00:07.176 --> 00:00:08.277
So you cut out the middleman.

3
00:00:08.317 --> 00:00:12.400
The learning mechanism is literally minimizing surprise.

4
00:00:12.801 --> 00:00:21.827
If, on the other hand, it misses, it's going to get four seconds at 20 volts of highly chaotic electrical activity.

5
00:00:21.927 --> 00:00:25.430
So now I can very easily ask, who is actually doing the learning?

6
00:00:25.690 --> 00:00:26.030
Correct.

7
00:00:26.211 --> 00:00:28.212
Is it my reinforcement learning mechanism?

8
00:00:28.377 --> 00:00:30.938
Network, or is it the actual biological neurons that I have?

9
00:00:30.958 --> 00:00:31.699
Doesn't count.

10
00:00:32.039 --> 00:00:32.840
Hello, Internet.

11
00:00:32.860 --> 00:00:41.384
This is your captain speaking, Lester Nare, joined as always by my co-host and our resident PhD, Krishna Chowdhury.

12
00:00:42.065 --> 00:00:56.393
And today we're going to be talking about a topic that sounds like it's something straight out of science fiction, which is, did a dish full of human neurons successfully play Doom, the video game?

13
00:00:57.053 --> 00:01:06.322
Now, this is something that has been hyped in the news over the last two to three weeks with everyone trying to understand, is this real or is this hype?

14
00:01:06.743 --> 00:01:16.833
We're going to dive into the origins of how this came about through a research paper in Neuron out of Cortical Labs in Melbourne, Australia, published in 2022.

15
00:01:16.873 --> 00:01:17.013
Okay.

16
00:01:18.474 --> 00:01:21.757
And look at that paper versus the recent news.

17
00:01:22.378 --> 00:01:29.525
And we might even touch on some of the other big fundamentals that made this concept possible.

18
00:01:29.925 --> 00:01:36.111
We are going to learn about the science from the ground up today because this is From First Principles.

19
00:01:51.793 --> 00:01:58.237
So in our world of AI, we are very much used to artificial neural networks, right?

20
00:01:58.937 --> 00:02:06.161
And these are biomimetic AI in the sense that they mimic biology, right?

21
00:02:07.022 --> 00:02:10.724
Biological brains became artificial neural networks.

22
00:02:11.777 --> 00:02:19.303
The subject of this deep dive today is about a possible transition from this biomimetic AI to straight-up biocomputing.

23
00:02:19.863 --> 00:02:26.388
It's called wetware engineering, and it's effectively neurons, human neurons, plus electronics.

24
00:02:26.808 --> 00:02:27.889
So you cut out the middleman.

25
00:02:27.909 --> 00:02:29.750
You don't even make any artificial neurons.

26
00:02:29.770 --> 00:02:33.954
You're using now biological neurons to help you in computing.

27
00:02:34.594 --> 00:02:37.399
Specifically, there's an Australian company called Cortical Labs.

28
00:02:38.000 --> 00:02:41.406
They were recently in the news because their wetware computer could play Doom.

29
00:02:43.395 --> 00:02:45.296
And it was all hype, right?

30
00:02:45.476 --> 00:02:48.737
I've seen it literally everywhere.

31
00:02:49.077 --> 00:02:49.317
Yeah.

32
00:02:49.437 --> 00:02:51.978
The tech bros are going crazy.

33
00:02:52.038 --> 00:02:52.238
Yeah.

34
00:02:52.298 --> 00:02:54.679
All the doomsday people are like, should we be doing this?

35
00:02:54.779 --> 00:02:55.000
Right.

36
00:02:55.180 --> 00:02:59.441
So, you know, I naturally asked how much of this is real and how much of this is hype.

37
00:03:00.081 --> 00:03:03.263
And it turns out there's a lot of both.

38
00:03:03.283 --> 00:03:04.023
Okay.

39
00:03:04.143 --> 00:03:05.223
And that's what we're going to get into.

40
00:03:05.243 --> 00:03:05.864
Okay.

41
00:03:05.884 --> 00:03:06.044
Okay.

42
00:03:06.284 --> 00:03:09.125
So let's do a little bit of brief background.

43
00:03:09.145 --> 00:03:09.265
Okay.

44
00:03:10.300 --> 00:03:15.862
Let's talk about the biological neuron, the one that is the substrate of our own nervous system in our human brain.

45
00:03:16.522 --> 00:03:20.103
A biological neuron is a cell, a biological cell.

46
00:03:20.803 --> 00:03:22.043
We see that on the left here.

47
00:03:22.303 --> 00:03:23.304
It's got a cell body.

48
00:03:24.484 --> 00:03:29.545
It's got dendrites, which are where other neurons come in and give you input.

49
00:03:30.506 --> 00:03:35.367
And in the cell body, what you do is you integrate, meaning you sum up all the inputs.

50
00:03:39.624 --> 00:03:46.308
exceeds some kind of threshold, then you fire what's called an action potential, which is, in other words, a spike.

51
00:03:46.768 --> 00:03:54.612
It's effectively saying my neuron went from off to on because all of the other neurons that were talking to me, there were enough.

52
00:03:54.872 --> 00:04:00.015
There were enough neurons that were talking to me with enough input that I decided to go from off to on.

53
00:04:00.295 --> 00:04:06.319
When I do that, I go downstream and the action potential goes downstream to other dendrites of other neurons.

54
00:04:06.999 --> 00:04:08.320
And that's how we get

55
00:04:09.169 --> 00:04:10.190
a biological neuron.

56
00:04:10.450 --> 00:04:12.972
Now, in the artificial neuron, very much similar.

57
00:04:13.312 --> 00:04:17.615
You've got incoming neurons and their activity.

58
00:04:18.435 --> 00:04:25.600
The GPU of NVIDIA sums that up and then fires if it fires like, you know,

59
00:04:26.682 --> 00:04:33.166
an output to downstream neurons if the input exceeds a certain threshold, right?

60
00:04:33.426 --> 00:04:38.829
So we've basically taken the biological thing, and we've turned it into a very simple math equation.

61
00:04:38.849 --> 00:04:53.078
Now, obviously, in that approximation, we're not doing it complete justice because in biological neurons, there's structure, there's proteins, there's all sorts of little things, and it's not really just straight-up integrate and fire, though that's the simplest kind of...

62
00:04:53.638 --> 00:04:54.779
model that you can create.

63
00:04:55.180 --> 00:05:00.825
But it turns out for a lot of the AI that we see, I mean, it's doing pretty well, right?

64
00:05:01.586 --> 00:05:10.134
So even though we sort of have a simplification of this representation of biological neurons in our artificial neural network systems...

65
00:05:11.035 --> 00:05:18.941
even that simple approximation in and of itself has valuable, productive outcomes.

66
00:05:19.262 --> 00:05:19.482
Yes.

67
00:05:19.522 --> 00:05:27.088
We don't have to replicate the whole of the brain in order to get some benefits from the concept.

68
00:05:27.168 --> 00:05:27.588
Exactly.

69
00:05:27.648 --> 00:05:33.253
And the reason why is because of the neural network, the idea that we can connect a bunch of these neurons together.

70
00:05:34.388 --> 00:05:38.129
into a giant network where everything is connected to one another.

71
00:05:38.189 --> 00:05:41.790
Not everything is connected, but you can have layers of connections from one to the other.

72
00:05:42.091 --> 00:05:45.952
So in the biological case, again, on the left, you've got one neuron that's connected to the other.

73
00:05:46.272 --> 00:05:50.613
So the output of one neuron is connected to the dendrites of the other through something called a synapse.

74
00:05:51.013 --> 00:05:55.155
That's the gap in between neurons that let the neurons talk to one another.

75
00:05:55.535 --> 00:05:58.916
And in the artificial case, you've got hidden layers of neurons.

76
00:05:59.356 --> 00:06:01.257
And from one layer to the next, you've got...

77
00:06:01.917 --> 00:06:02.778
inputs going in.

78
00:06:03.218 --> 00:06:08.662
Again, each neuron is going to integrate, meaning sum up and then fire if it exceeds some threshold.

79
00:06:09.062 --> 00:06:15.147
So you've got these nonlinearities, and then that's going to create the output, which is going to create the next output, which is going to create the next output.

80
00:06:15.367 --> 00:06:24.454
And if you have trillions of parameters and, you know, millions and billions of neurons, then you can actually approximate any function you want, including, let's say, thought.

81
00:06:24.794 --> 00:06:28.137
And that's how you get large language models and all sorts of very, very cool things.

82
00:06:28.437 --> 00:06:28.977
So that's how

83
00:06:29.758 --> 00:06:33.220
We've mostly done artificial neural networks, right?

84
00:06:34.081 --> 00:06:40.205
Now, how is the brain different from these artificial neural networks?

85
00:06:40.265 --> 00:06:41.826
That's something that we should focus on here.

86
00:06:42.006 --> 00:06:45.268
Because these guys are trying to use biological neurons.

87
00:06:45.629 --> 00:06:50.812
If the artificial neural network is just that good, why would we want to try to use biological neural networks?

88
00:06:50.912 --> 00:06:57.717
There is an advantage that cortical labs and other people doing this technology say.

89
00:06:58.787 --> 00:07:05.491
The idea is silicon models, which are the things in NVIDIA and things like that, they use the von Neumann architecture of computing.

90
00:07:06.012 --> 00:07:08.413
The biological brain is just the biological brain.

91
00:07:08.834 --> 00:07:11.535
The von Neumann architecture is the following.

92
00:07:12.036 --> 00:07:16.058
Your processor is separate from your memory, right?

93
00:07:16.358 --> 00:07:17.879
The memory is where you store stuff.

94
00:07:18.260 --> 00:07:19.821
The processing is where you compute.

95
00:07:20.501 --> 00:07:23.383
Those are two separate things in a conventional computer, right?

96
00:07:23.403 --> 00:07:26.286
You've got your, like, RAM, and then you've got your CPU.

97
00:07:26.926 --> 00:07:37.895
And over the years, there's a huge gap between how fast you can compute things in your CPU or your GPU and how fast you can access the memory where you store stuff.

98
00:07:38.496 --> 00:07:40.317
In the brain, both are co-located.

99
00:07:40.717 --> 00:07:41.018
Okay.

100
00:07:41.038 --> 00:07:41.838
They're the same thing.

101
00:07:41.858 --> 00:07:44.821
They're all stored in the synapses, the weights of these things.

102
00:07:44.861 --> 00:07:47.583
The memory itself is stored in the weights, right?

103
00:07:47.883 --> 00:07:53.207
And the way that you process stuff is you go from one neuron to the other in this neural network.

104
00:07:53.287 --> 00:07:56.609
But in an artificial neural network, there's that difference, right?

105
00:07:57.770 --> 00:08:03.013
So the idea is that perhaps this gives the brain an advantage.

106
00:08:03.334 --> 00:08:06.316
Another advantage that the brain has is that it's massively parallel.

107
00:08:07.136 --> 00:08:11.101
And it only uses like 12 to 20 watts of power for 100 billion neurons.

108
00:08:11.442 --> 00:08:17.670
On the other hand, state-of-the-art GPUs and neural nets use gigawatts of power for computing, right?

109
00:08:17.730 --> 00:08:24.479
So there's a clear difference also in the amount of energy needed to create that computing.

110
00:08:25.240 --> 00:08:29.526
So the differences between the two, one is efficiency.

111
00:08:29.566 --> 00:08:32.510
We've created a simplified abstraction of what the brain does.

112
00:08:33.231 --> 00:08:38.559
And it takes a lot more energy to even do some of the things that the brain does.

113
00:08:38.999 --> 00:08:40.962
So it's a highly inefficient abstraction.

114
00:08:41.202 --> 00:08:47.767
As compared to the brain, naturally, artificial intelligence is highly inefficient or highly energy consumptive.

115
00:08:47.867 --> 00:08:49.908
Yes.

116
00:08:50.309 --> 00:09:00.195
Also, the way in which the brain is structured, it can do significantly larger amounts of parallel processing.

117
00:09:00.215 --> 00:09:00.296
Yes.

118
00:09:01.396 --> 00:09:03.998
Which enables that energy efficiency in part.

119
00:09:04.238 --> 00:09:04.438
Yeah.

120
00:09:04.559 --> 00:09:09.702
And also like each neural, each neuron itself is like highly efficient as a computational unit.

121
00:09:10.003 --> 00:09:10.163
Right.

122
00:09:10.183 --> 00:09:11.604
It doesn't require a lot of energy.

123
00:09:11.684 --> 00:09:16.487
We think, you know, our brain does consume 20% of the energy of our body.

124
00:09:16.527 --> 00:09:19.029
But still, that's not actually a lot when you think about things.

125
00:09:19.069 --> 00:09:19.509
Right.

126
00:09:19.629 --> 00:09:23.913
So perhaps the idea is we can use this natural biological wetware.

127
00:09:24.693 --> 00:09:29.702
in place of our von Neumann architecture hardware, the GPUs that take up all that energy.

128
00:09:29.923 --> 00:09:31.105
That's the argument, okay?

129
00:09:31.345 --> 00:09:35.954
But in order to do that, we need to be able to read and write to biological neurons.

130
00:09:36.645 --> 00:09:39.807
Right, and also where do these biological neurons exist?

131
00:09:39.867 --> 00:09:40.407
Yeah, yeah.

132
00:09:40.467 --> 00:09:41.188
Where do they come from?

133
00:09:41.228 --> 00:09:42.448
They're usually in our brain, right?

134
00:09:42.929 --> 00:09:48.092
So let's get into how we actually read and write to real biological neurons in a dish.

135
00:09:48.132 --> 00:09:57.037
This was actually pioneered in 1972 by Thomas G. Pine at Caltech when he developed the first multi-electrode array.

136
00:09:57.337 --> 00:09:57.998
Here's the idea.

137
00:09:58.498 --> 00:10:01.700
So neurons produce electricity when they talk to one another.

138
00:10:01.800 --> 00:10:02.660
That's how they talk.

139
00:10:03.081 --> 00:10:04.922
They use ion channels, which are...

140
00:10:05.920 --> 00:10:10.942
little holes in their membranes, the boundary between the inside and the outside of the cell.

141
00:10:11.462 --> 00:10:24.606
And when those little membrane proteins, those channels open up or close, that facilitates the transfer of ions like potassium ions, chlorine ions, sodium ions, from the inside to the outside of the cell.

142
00:10:24.826 --> 00:10:27.407
Ions are charged atoms in some sense.

143
00:10:27.667 --> 00:10:28.868
And so when they move around...

144
00:10:29.573 --> 00:10:30.774
they're going to create electricity.

145
00:10:30.794 --> 00:10:32.056
They're going to create electric fields.

146
00:10:32.356 --> 00:10:39.583
And we can sense that if we have an electrode array, which is just a bunch of tips of metal that is connected to some kind of amplifier.

147
00:10:40.504 --> 00:10:47.932
And as each neuron produces a little electric field blip, I can sense that with my electrode array, right?

148
00:10:48.172 --> 00:10:50.154
I can also supply electricity

149
00:10:50.935 --> 00:10:54.716
to then influence the behavior of the neurons near my electrode.

150
00:10:54.956 --> 00:11:08.320
Like if I've got a little metal tip here and I supply electricity, that is going to change the electric field around that metal tip, which is then going to influence the ions and thus the proteins on that neuron that are nearby that metal tip.

151
00:11:08.540 --> 00:11:12.381
And so I can now toggle between on and off based on that stimulation.

152
00:11:12.621 --> 00:11:16.542
So I've got a read, which is when the neurons fire, they're going to release a little electric...

153
00:11:17.122 --> 00:11:24.070
field, you know, spike that I can read and then I can write by supplying electricity and that's going to influence the neurons around it.

154
00:11:24.588 --> 00:11:26.049
I have a quick clarifying question.

155
00:11:26.069 --> 00:11:26.270
Yeah.

156
00:11:26.770 --> 00:11:33.016
When you talked about the spike just now, you're referring to the concept you talked about earlier of the action potential.

157
00:11:33.056 --> 00:11:33.276
Yes.

158
00:11:33.476 --> 00:11:43.806
This sort of like surge of electrical signal that then the neuron utilizes to do its next action or its next step.

159
00:11:44.466 --> 00:11:47.449
And so when we have the ability to provide that

160
00:11:52.408 --> 00:12:00.414
energy or source material necessary for the neuron to trigger its action potential artificially.

161
00:12:01.875 --> 00:12:06.898
It's not only a capability that is not reproducible artificially.

162
00:12:06.938 --> 00:12:07.718
Yes, exactly.

163
00:12:07.818 --> 00:12:13.041
Yeah, it's something that we can we can provide a signal to whatever metal contact is near that neuron.

164
00:12:13.442 --> 00:12:19.445
And then, you know, that's going to create a change in electric field, which is going to then drive ions around that neuron.

165
00:12:19.485 --> 00:12:21.446
It's going to drive the proteins around that neuron.

166
00:12:21.786 --> 00:12:25.168
And then the neurons will be sensitive to that change in the electric field.

167
00:12:25.268 --> 00:12:25.448
Right.

168
00:12:25.488 --> 00:12:27.550
They're going to think that it's another neuron somewhere.

169
00:12:28.230 --> 00:12:29.110
That's like doing that.

170
00:12:29.310 --> 00:12:32.411
So we've to just make a fine point on it.

171
00:12:32.931 --> 00:12:39.413
We can translate into the language that these neurons speak using this kind of methodology.

172
00:12:39.453 --> 00:12:40.273
Yes, exactly.

173
00:12:40.293 --> 00:12:40.473
Yes.

174
00:12:40.533 --> 00:12:41.833
So now we can read and write.

175
00:12:42.033 --> 00:12:42.233
Yes.

176
00:12:42.373 --> 00:12:44.734
OK, so can we do biocomputing?

177
00:12:44.794 --> 00:12:48.735
Well, early biocomputing actually started in the late 1990s, early 2000s.

178
00:12:49.135 --> 00:12:50.915
There was this guy, Steve M. Potter.

179
00:12:51.215 --> 00:12:55.216
He was at the Georgia Tech, Georgia Institute of Technology.

180
00:12:58.297 --> 00:13:08.703
which is a hybrid robot, he used cultured cortical neurons from rats on a multi-electrode array to actually do just, to control like a robot.

181
00:13:09.123 --> 00:13:09.363
Okay?

182
00:13:09.723 --> 00:13:12.645
So again, this is a little plate that has a bunch of

183
00:13:13.565 --> 00:13:38.928
silicon electrodes let's say it could be something else i don't actually know what what but today we use a lot of silicon in our electrodes obviously um and he could grow rat cortical neurons on that electrode like in a petri dish and then sense and then write you know so you've got to read in and then uh write out you can listen to what's happening and you can talk to it to tell it stuff

184
00:13:39.048 --> 00:13:40.089
Yeah, yeah.

185
00:13:40.109 --> 00:13:41.550
So this was him controlling a robot.

186
00:13:41.930 --> 00:13:43.512
In 2004, this was in the news.

187
00:13:43.532 --> 00:13:44.212
That was pretty cool.

188
00:13:44.533 --> 00:13:53.180
Thomas DeMars cultured 25,000 rat cortical neurons in a Petri dish, and that acted as an autopilot for an F-22 flight simulator.

189
00:13:54.080 --> 00:13:56.963
This was all over the news as well back then.

190
00:13:57.403 --> 00:13:59.165
And so that's pretty cool, right?

191
00:13:59.205 --> 00:14:03.108
Like almost 20 years ago, more than 20 years ago, 30 years ago is when we're doing this stuff.

192
00:14:03.468 --> 00:14:07.511
So how come now is when we can actually...

193
00:14:08.212 --> 00:14:08.973
Play video games.

194
00:14:09.453 --> 00:14:09.673
Okay.

195
00:14:10.314 --> 00:14:20.461
The reason for that is, one, we've had a huge improvement in how many electrodes we can fit inside our multi-electrode array.

196
00:14:20.521 --> 00:14:22.283
Back then, we could only fit about 60.

197
00:14:22.663 --> 00:14:23.744
Now we can do thousands.

198
00:14:24.264 --> 00:14:26.466
So we can sense a lot finer detail.

199
00:14:27.166 --> 00:14:28.347
The bandwidth is greater.

200
00:14:28.407 --> 00:14:28.867
Yeah, yeah.

201
00:14:28.887 --> 00:14:31.568
It's like the density of the electrodes is literally greater.

202
00:14:31.608 --> 00:14:32.588
We can make them smaller.

203
00:14:32.628 --> 00:14:33.848
We can make them more sensitive.

204
00:14:34.429 --> 00:14:38.130
So we can now sense individual neurons rather than global populations.

205
00:14:38.990 --> 00:14:44.332
The other thing is that we need to keep these neurons alive, right, in the Petri dish.

206
00:14:44.392 --> 00:14:46.193
And we've had a tremendous amount of...

207
00:14:47.472 --> 00:15:11.872
scientific innovation in how to keep neurons in a petri dish alive for an extended period of time right so now with all of that we can get to cortical labs which is a new startup that came out of australia that is doing this stuff that is trying to make neurons in a petri dish play pong and now most recently play doom but before that allegedly allegedly we'll see right we'll see

208
00:15:12.592 --> 00:15:13.793
Before that?

209
00:15:13.873 --> 00:15:21.797
We are just so grateful and excited for so many of you that continue to join us on this journey of curiosity and discovery.

210
00:15:22.237 --> 00:15:24.538
For those of you who are returning, welcome back.

211
00:15:25.019 --> 00:15:31.822
We greatly appreciate your attendance at our multi-week now show release schedule.

212
00:15:32.262 --> 00:15:40.907
As we mentioned in our previous episodes, we are now testing going from just doing one weekly episode to multiple, with the rundown being it's on episode type.

213
00:15:41.427 --> 00:15:46.989
So if you're looking at your podcast feed or your Spotify feed and you're like, what's going on?

214
00:15:47.429 --> 00:15:50.770
You're getting more of us more often, which has been in high demand.

215
00:15:51.051 --> 00:16:06.056
And if you do like what you hear, whether you've been here with us for a long time or you're a new listener, one of the greatest ways that you can support the show is to give it a like, share, comment, follow on the platform that you happen to discover us on or listen to us on.

216
00:16:06.836 --> 00:16:19.910
And if you want to go above and beyond, we do have a donation portal at the website ffppod.com backslash donate where you can support the show with a recurring or one-time donation.

217
00:16:20.251 --> 00:16:22.233
We are a two-man operation.

218
00:16:22.653 --> 00:16:25.396
We've set up this unbelievable organization.

219
00:16:25.757 --> 00:16:30.040
production through the ingenuity of the both of us.

220
00:16:30.180 --> 00:16:35.584
And it is really a pleasure to be able to talk to you guys about the wonders of science multiple times a week.

221
00:16:36.145 --> 00:16:43.030
And we just, again, really, really, really cannot express how important it is to fight the billionaire algorithm

222
00:16:43.610 --> 00:17:00.423
to just engage in any way that you can, even if it's not financially, it's really helpful for us to get to more people so we can talk about the wonders of what the researchers and scientists all over the globe are providing us in terms of the future of humanity.

223
00:17:00.563 --> 00:17:04.266
And with that housekeeping note, we can jump back into Cortical Labs.

224
00:17:04.686 --> 00:17:05.026
All right.

225
00:17:05.306 --> 00:17:06.767
So now let's talk about Cortical Labs.

226
00:17:07.007 --> 00:17:07.967
This is the subject.

227
00:17:08.428 --> 00:17:11.449
This is a startup out of Melbourne, Australia.

228
00:17:11.509 --> 00:17:15.110
I just recently watched the Formula One Grand Prix in Melbourne.

229
00:17:16.111 --> 00:17:16.551
Ferrari.

230
00:17:16.691 --> 00:17:17.271
No, it didn't.

231
00:17:18.011 --> 00:17:18.712
Mercedes is.

232
00:17:19.492 --> 00:17:23.273
Was that the one where Leclerc and Hamilton were battling each other?

233
00:17:23.293 --> 00:17:25.594
No, that was the Chinese Grand Prix very recently.

234
00:17:25.734 --> 00:17:26.775
Anyways, I don't want to talk about it.

235
00:17:27.575 --> 00:17:52.003
um cortical labs founded in 2019 by hongwen chong the company gained international attention in 2022 because of a paper that we're going to cover it was in the journal neuron and it was describing this thing called dish brain which is a system in which in vitro neural cultures so that's in vitro meaning in a petri dish learned how to play the game pong in a closed loop environment

236
00:17:52.503 --> 00:17:52.683
Okay?

237
00:17:53.143 --> 00:18:02.866
So that is the paper that we are going to be talking about because that is the bedrock upon which their technology is built, and it'll help us understand how exactly they did do.

238
00:18:03.446 --> 00:18:03.686
Okay?

239
00:18:04.086 --> 00:18:07.287
And it's actually very cool, some of the stuff that they've done.

240
00:18:07.567 --> 00:18:07.808
Okay?

241
00:18:08.648 --> 00:18:13.869
So, first of all, you got to get neurons for your wetware engineering.

242
00:18:14.009 --> 00:18:14.269
Yes.

243
00:18:14.329 --> 00:18:15.050
How do you get neurons?

244
00:18:15.430 --> 00:18:18.131
Well, first you get primary mouse cortical neurons.

245
00:18:18.671 --> 00:18:19.131
Those are...

246
00:18:20.233 --> 00:18:20.753
fine to get.

247
00:18:21.414 --> 00:18:26.456
You know, mice are very much a animal that we use in biological research.

248
00:18:26.816 --> 00:18:27.897
So that's not that hard.

249
00:18:28.257 --> 00:18:30.718
Human cortical cells, on the other hand, are hard to come by.

250
00:18:31.058 --> 00:18:43.324
So instead of actually, you know, waiting for a patient who has epilepsy, let's say to have a little part of their brain excised and get that kind of cortical neuron, you never know with those kinds of samples, like, okay, it's a brain that has epilepsy.

251
00:18:43.344 --> 00:18:44.365
That's not a wild type.

252
00:18:44.985 --> 00:19:10.006
type brain it's not a neurotypical as we should say type brain so instead what what they did was they generated human induced pluripotent stem cells this is something that we cover a lot it won the nobel prize because it's a technology where we can take somatic cells like stuff from the biopsy of a of skin we can reprogram them to become stem cells which are blank slate cells that can be anything that they want and then we can

253
00:19:11.425 --> 00:19:17.331
Give them a cocktail of chemicals to differentiate them into cortical tissue.

254
00:19:17.611 --> 00:19:22.316
So we take skin cells, we make them a blank slate, and then we make them cortical neurons.

255
00:19:22.676 --> 00:19:26.200
I just want to make a quick note that this was, I think this was the Yamanaka.

256
00:19:26.780 --> 00:19:27.080
Yes.

257
00:19:27.280 --> 00:19:27.481
Yeah.

258
00:19:27.641 --> 00:19:35.805
And this was really important because prior to that, embryonic stem cell research was highly controversial and it was the only way to acquire stem cells.

259
00:19:36.385 --> 00:19:44.289
And it really unlocked the ability to actually be able to do this type of work by removing the sort of moral, ethical issues.

260
00:19:44.409 --> 00:19:44.629
Yeah.

261
00:19:44.749 --> 00:19:45.370
Conundrum.

262
00:19:45.650 --> 00:19:49.614
So now the point being any human cell, you can reprogram to something else.

263
00:19:49.694 --> 00:19:49.894
Yeah.

264
00:19:49.994 --> 00:19:54.898
That wasn't, that's where, you know, that's a huge, that's a huge foundational discovery.

265
00:19:55.038 --> 00:19:55.759
Exactly.

266
00:19:55.899 --> 00:20:01.865
That, that now, cause I, I know the sort of stem cell piece for some folks, it's like they, they're that,

267
00:20:02.565 --> 00:20:18.626
historical point because some people still think oh aren't we still getting those from embryos no yeah this is just your skin cell becoming a stem cell it's pretty insane now of course with the pluripotent stem cells it's not a complete blank slate right the embryonic stem cell is like the gold standard because those things

268
00:20:19.367 --> 00:20:21.349
are built to be blank slates, right?

269
00:20:21.750 --> 00:20:25.874
So, but this comes pretty close for literally 90% of applications.

270
00:20:26.034 --> 00:20:34.363
So it's done a huge service for the biomedical industry because for 90% of applications, I don't need to worry about that.

271
00:20:34.403 --> 00:20:35.564
That's still a huge unlock.

272
00:20:35.604 --> 00:20:35.824
Yes.

273
00:20:35.924 --> 00:20:37.286
I just want to be very clear here.

274
00:20:37.706 --> 00:20:39.988
So let's talk about the culture that they made.

275
00:20:40.209 --> 00:20:40.989
They made 800,000 neurons.

276
00:20:42.711 --> 00:20:44.374
Co-cultured with astrocytes.

277
00:20:44.474 --> 00:20:45.736
This is pretty incredible here.

278
00:20:46.517 --> 00:20:52.666
Usually when we make stem cells, right, and we make a culture of cortical tissue, you're going to have all neurons.

279
00:20:53.755 --> 00:20:55.516
That actually is not ideal.

280
00:20:55.536 --> 00:20:56.236
Okay.

281
00:20:56.256 --> 00:20:57.517
For certain applications, it is.

282
00:20:57.677 --> 00:21:00.279
Like when we were discussing the ALS story.

283
00:21:00.379 --> 00:21:00.559
Yes.

284
00:21:00.619 --> 00:21:05.782
And we wanted to create an ALS mini brain on a Petri dish.

285
00:21:05.902 --> 00:21:08.063
We wanted it to be all motor neurons, right?

286
00:21:08.123 --> 00:21:10.624
Because then that's how we test the drugs and so on and so forth.

287
00:21:11.045 --> 00:21:15.007
In this case, we want like a live computing unit.

288
00:21:15.227 --> 00:21:15.427
Mm-hmm.

289
00:21:15.687 --> 00:21:20.109
That kind of mimics the human brain, like, or at least a small chunk of the human brain.

290
00:21:20.329 --> 00:21:22.269
And the human brain is not just neurons.

291
00:21:22.569 --> 00:21:24.050
The human brain has astrocytes.

292
00:21:24.130 --> 00:21:33.433
So if we look over here, what they've been able to do is grow neurons along with astrocytes and all of the supporting cells.

293
00:21:33.913 --> 00:21:34.173
Okay?

294
00:21:34.513 --> 00:21:37.835
So the blue, that DAPI.

295
00:21:37.995 --> 00:21:38.115
Yes.

296
00:21:38.335 --> 00:21:38.555
Yes.

297
00:21:39.435 --> 00:21:41.116
Those are marking the neurons.

298
00:21:41.836 --> 00:21:45.437
The GFAP and the red, those are marking supporting astrocytes.

299
00:21:45.537 --> 00:21:48.098
And those are critical for long-term functioning.

300
00:21:48.298 --> 00:21:48.678
Okay.

301
00:21:48.698 --> 00:21:48.898
Okay?

302
00:21:49.118 --> 00:21:55.580
So this is sort of a secret sauce that allows those cultures in that Petri dish to survive for a lot longer.

303
00:21:55.980 --> 00:21:58.261
So now they can learn over a longer timescale.

304
00:21:59.001 --> 00:22:04.263
And, you know, actually now we can take neurocomputing seriously.

305
00:22:04.283 --> 00:22:04.543
Okay.

306
00:22:04.743 --> 00:22:19.987
If you have a professional sports team with these high-skill athletes, they're nothing without the training and medical staff that enable them to continue to perform over longer periods of time given the amount of stress and things that are going on.

307
00:22:20.047 --> 00:22:20.487
Exactly.

308
00:22:20.547 --> 00:22:29.072
And so these are all these supporting cells which are crucial to creating a compute, like, you know, a little piece of computation on that Petri dish.

309
00:22:29.112 --> 00:22:30.133
It's not just all neurons.

310
00:22:30.393 --> 00:22:33.034
You've got to have sort of a mimicked brain environment.

311
00:22:33.274 --> 00:22:34.815
And they were very successful at doing that.

312
00:22:35.135 --> 00:22:37.937
So now that's the side of the cells.

313
00:22:38.657 --> 00:22:41.899
We've created a really nice culture with 800,000 neurons.

314
00:22:41.939 --> 00:22:42.759
That's a lot of neurons.

315
00:22:42.940 --> 00:22:43.080
Right.

316
00:22:43.140 --> 00:22:43.320
Okay.

317
00:22:43.840 --> 00:22:47.203
Now let's go into the multi-electrode array.

318
00:22:47.364 --> 00:22:47.804
I'm sorry.

319
00:22:47.824 --> 00:22:49.506
I just want to interrupt you really briefly.

320
00:22:50.026 --> 00:22:58.835
So the 800,000 neurons, would that be comparable to saying like the GPUs in your NVIDIA cluster as like the corollary in an artificial system?

321
00:22:59.315 --> 00:22:59.555
Yeah.

322
00:22:59.676 --> 00:22:59.896
Yeah.

323
00:22:59.956 --> 00:23:01.317
It's like the number of...

324
00:23:02.407 --> 00:23:04.528
It's the number of transistors one can think of.

325
00:23:04.588 --> 00:23:04.968
Okay.

326
00:23:05.008 --> 00:23:05.348
Maybe.

327
00:23:05.588 --> 00:23:06.128
Maybe.

328
00:23:06.148 --> 00:23:07.268
Maybe.

329
00:23:07.308 --> 00:23:11.189
No, it's not quite because it's not one transistor to one artificial neuron.

330
00:23:11.509 --> 00:23:17.891
But in your artificial neural network, you have billions and billions of neurons with trillions of weights in between them.

331
00:23:18.171 --> 00:23:19.932
Here you've got 800,000 neurons.

332
00:23:20.052 --> 00:23:23.533
It's still small, but it's incredible what they're able to do with it.

333
00:23:23.693 --> 00:23:24.073
Understood.

334
00:23:24.293 --> 00:23:24.513
You know?

335
00:23:25.242 --> 00:23:26.003
But that's the idea.

336
00:23:26.203 --> 00:23:26.363
Yes.

337
00:23:26.383 --> 00:23:26.603
You know?

338
00:23:26.984 --> 00:23:28.345
Okay, so that's our neuron piece.

339
00:23:28.745 --> 00:23:30.346
Now let's get into how are we going to sense them.

340
00:23:30.507 --> 00:23:31.888
That's our multi-electrodes, right?

341
00:23:31.928 --> 00:23:37.873
Those are the little metal leads that let us read the neurons and then write to the neurons.

342
00:23:38.233 --> 00:23:45.419
So here they did a high-density multi-electrode array using CMOS technology, 26,000 platinum electrodes.

343
00:23:45.820 --> 00:23:49.703
What you can see over there, all of these little rectangles is an individual electrode.

344
00:23:49.723 --> 00:23:49.843
Right.

345
00:23:50.223 --> 00:23:56.324
So that can be individually tuned to have voltage, you know, 5 volts, negative 2 volts, whatever you want it to be.

346
00:23:56.584 --> 00:24:02.546
It can read the electric field around it, and it can write an electric field around it.

347
00:24:02.926 --> 00:24:03.706
Okay?

348
00:24:03.826 --> 00:24:07.727
And you can see the sort of—this is a—I think it's an electron microscope.

349
00:24:08.147 --> 00:24:13.388
And you can see the cell bodies and all the dendrites and all of the connections that are just sort of growing—

350
00:24:14.368 --> 00:24:16.530
through that multi-electrode array.

351
00:24:16.770 --> 00:24:23.875
So when I have a connection, like a little axon that's connected from neuron A to B, I can follow that signal with my electrodes.

352
00:24:23.895 --> 00:24:24.115
Yes.

353
00:24:24.335 --> 00:24:25.496
Very, very cool, right?

354
00:24:25.556 --> 00:24:27.558
It looks like a map of LA from the sky.

355
00:24:27.718 --> 00:24:29.499
Yeah, yeah, it really does.

356
00:24:29.519 --> 00:24:32.962
The Spanish are very good at gridding, and that's what this looks like.

357
00:24:33.062 --> 00:24:35.584
If you're in Barcelona, it also looks like that, you know?

358
00:24:35.604 --> 00:24:36.465
Yes, yes.

359
00:24:37.485 --> 00:24:43.210
And out of the 26,000 platinum electrodes, there's only 1024 independent channels

360
00:24:43.970 --> 00:25:10.679
that are active okay so they can pick and choose which ones they want to read through right so that that's again a huge thing and one of the incredible things is that the latency between reading and then making a computation to figure out what to write right because i'm going to read the the the activity from the neurons then i'm gonna i don't know i guess like see what the game state of of pong is or doom is and then write back to the neurons and give them feedback and

361
00:25:10.899 --> 00:25:12.580
They got it down to five milliseconds.

362
00:25:12.600 --> 00:25:13.480
That's such a huge...

363
00:25:13.581 --> 00:25:15.562
Which is quite a small latency.

364
00:25:15.602 --> 00:25:15.862
Yes.

365
00:25:16.142 --> 00:25:17.202
If you really think about it.

366
00:25:17.983 --> 00:25:23.266
That is... And if we're trying to mimic the brain, we need that latency as close to zero as possible.

367
00:25:23.306 --> 00:25:23.666
Exactly.

368
00:25:23.706 --> 00:25:27.628
Because for us, our latency is around that.

369
00:25:27.648 --> 00:25:27.828
Right.

370
00:25:27.928 --> 00:25:28.128
Right?

371
00:25:29.269 --> 00:25:37.834
One of the common things that we think about is, like, our eyes are about 60 frames per second is the top level of how many, you know...

372
00:25:39.174 --> 00:25:42.576
little pictures it takes every second and relays back to our brain.

373
00:25:42.897 --> 00:25:47.720
And so 60 frames, let's just say 50 frames per second, because I want to make my math simpler.

374
00:25:47.800 --> 00:25:48.040
Yes.

375
00:25:48.100 --> 00:25:48.300
Right.

376
00:25:48.380 --> 00:25:54.504
So 50 frames per second, that's about 20 milliseconds is the, is the refresh rate.

377
00:25:54.524 --> 00:25:54.644
Yes.

378
00:25:54.664 --> 00:25:54.744
Yes.

379
00:25:55.224 --> 00:25:57.385
of my eye to the brain, right?

380
00:25:57.585 --> 00:25:59.225
And this is going down to five milliseconds.

381
00:25:59.245 --> 00:26:01.226
So it's even lower than that.

382
00:26:01.746 --> 00:26:03.426
That's the speed at which neurons talk.

383
00:26:03.466 --> 00:26:09.328
The action potential, which is this thing that I was telling you about when the neuron turns off, it actually immediately turns off right after.

384
00:26:09.648 --> 00:26:10.488
It turns on and off.

385
00:26:10.969 --> 00:26:13.829
And that action potential takes about one millisecond.

386
00:26:14.230 --> 00:26:15.450
So getting it down...

387
00:26:16.371 --> 00:26:23.034
is mimicking the sort of computational latency that the brain already has and what these neurons in your dish are used to.

388
00:26:23.254 --> 00:26:23.494
Yes.

389
00:26:23.594 --> 00:26:23.794
Right?

390
00:26:23.814 --> 00:26:27.496
So you don't have them, like, thinking harder than they need to.

391
00:26:27.516 --> 00:26:29.777
Than they need to, which makes it more energy-filled.

392
00:26:29.797 --> 00:26:29.997
Yeah.

393
00:26:30.817 --> 00:26:32.978
So now let's look at the dish-brain protocol.

394
00:26:33.478 --> 00:26:36.400
How do we play Pong with our...

395
00:26:37.270 --> 00:26:37.790
Video game.

396
00:26:37.850 --> 00:26:39.131
With our DBP.

397
00:26:39.351 --> 00:26:39.611
Yes.

398
00:26:40.111 --> 00:26:40.352
Okay.

399
00:26:40.632 --> 00:26:46.995
So over here, we've got in the middle our Petri dish that has on the bottom of it this multi-electrode array.

400
00:26:47.315 --> 00:26:51.497
These are little tiny electrodes that are going to sense the neuron culture that is on top.

401
00:26:51.537 --> 00:26:57.140
The neuron culture on top is getting bathed with electrolytes with glucose to keep it alive.

402
00:26:57.160 --> 00:26:59.241
They can keep it alive for as long as six months.

403
00:26:59.761 --> 00:27:04.744
And as these neurons in my Petri dish turn on and off, my multi-electrode array is going to sense that.

404
00:27:05.144 --> 00:27:06.185
It's going to read it in.

405
00:27:06.965 --> 00:27:10.526
It's going to sense the gameplay, and it's going to read it out into the neuron.

406
00:27:10.826 --> 00:27:17.528
Now, what we don't want to do is completely jumble up our read-in and our read-out.

407
00:27:17.948 --> 00:27:21.249
That was literally going to be my—the orchestration problem there is non-trivial.

408
00:27:21.269 --> 00:27:21.469
Right?

409
00:27:21.529 --> 00:27:28.071
Yeah, because we've got a bunch of electrodes, and the brain is not really an all-read-in, all-read-out, right?

410
00:27:28.451 --> 00:27:32.492
There are regions in the brain, like the occipital lobe that has the visual cortex—

411
00:27:34.361 --> 00:27:37.242
there's the prefrontal cortex that makes decisions.

412
00:27:37.602 --> 00:27:43.605
There's the motor cortex that actually outputs my hand when I'm like, let's say using a joystick, right?

413
00:27:43.865 --> 00:27:48.406
So there are segregated spots in my brain for where I'm reading in.

414
00:27:48.706 --> 00:27:51.707
That'll be my visual cortex and where I'm going and reading out.

415
00:27:52.007 --> 00:27:53.228
That would be my motor cortex.

416
00:27:53.588 --> 00:27:56.329
So they decided to create an artificial cortex,

417
00:27:57.121 --> 00:28:00.868
visual cortex and a motor cortex on the chip itself.

418
00:28:01.209 --> 00:28:01.469
Okay?

419
00:28:01.750 --> 00:28:06.038
So at the very top, on the right-hand side, what you can see is the electrode layout schematic.

420
00:28:06.359 --> 00:28:06.539
Okay.

421
00:28:06.920 --> 00:28:08.482
So you've got your rectangular...

422
00:28:10.001 --> 00:28:15.765
multi-electrode array that has 1024 electrodes that I can actively read out and read in from.

423
00:28:16.305 --> 00:28:24.970
On the very top, I'm going to make a rectangular box that's in the upper half, and eight of those electrodes I'm going to use as my, quote, visual cortex.

424
00:28:25.071 --> 00:28:28.793
It's really quite a sensory cortex, I should say, because that's what I'm...

425
00:28:30.767 --> 00:28:32.389
Writing into, right?

426
00:28:32.649 --> 00:28:34.911
The sensory cortex is getting information.

427
00:28:35.151 --> 00:28:43.920
When I play a video game, the sensory cortex is through my eyes and through, I guess, sound sometimes, getting information about the state of the game.

428
00:28:44.429 --> 00:28:48.091
If you talk to Call of Duty players, sound is as equally important.

429
00:28:48.111 --> 00:28:48.271
Yeah.

430
00:28:48.291 --> 00:28:51.473
With their high-quality headphones to sneak up on you as visual.

431
00:28:51.693 --> 00:28:51.913
Yeah.

432
00:28:52.013 --> 00:28:52.814
I'm just being annoying.

433
00:28:52.914 --> 00:29:01.139
I mean, when I was playing Halo, I used to sometimes have to, like, play on mute because I was, like, 2 a.m. And my parents were like, what are you doing?

434
00:29:01.479 --> 00:29:04.441
So I'm actually quite good at playing Halo on mute.

435
00:29:04.541 --> 00:29:05.641
So I don't know about you guys.

436
00:29:05.981 --> 00:29:10.004
But anyways, the idea is there's a sensory cortex that is getting information.

437
00:29:10.124 --> 00:29:10.344
Yeah.

438
00:29:10.464 --> 00:29:11.084
From the brain.

439
00:29:11.384 --> 00:29:20.648
So I'm going to just arbitrarily make that top half of my electrode, the sensory cortex, and that's where I'm going to write information to my neurons.

440
00:29:20.968 --> 00:29:27.831
And the idea here is we're segregating it from other section or sections such that it only has one thing to worry about.

441
00:29:27.851 --> 00:29:28.371
Yeah, exactly.

442
00:29:28.391 --> 00:29:33.854
It's kind of like, I mean, I don't want to give too much agency to these neurons in a dish.

443
00:29:34.114 --> 00:29:38.716
But what I'm saying is that those neurons in that upper half of my Petri dish,

444
00:29:39.396 --> 00:30:03.140
are acting like the sensory neurons they're going to sort of understand that hey i'm getting information about something and i think this goes back to the point you brought up earlier which is proximity creates stronger neural connections yes and so because you're putting them in proximity and having a similar input yeah the theory is you know they will naturally strengthen around that shared yes idea

445
00:30:03.320 --> 00:30:03.720
Very good.

446
00:30:03.740 --> 00:30:04.461
Yeah, exactly.

447
00:30:04.501 --> 00:30:07.243
Those neurons in that region, they're all connected to one another.

448
00:30:07.603 --> 00:30:09.684
And so they're going to somehow figure it out.

449
00:30:09.865 --> 00:30:10.025
Right.

450
00:30:10.045 --> 00:30:12.286
We're going to get into how, but that's the idea.

451
00:30:12.306 --> 00:30:13.487
They're going to somehow figure it out.

452
00:30:13.807 --> 00:30:20.412
And actually, if we put that back, the bottom half of that Petri dish has those up and down arrows.

453
00:30:20.492 --> 00:30:20.692
Yeah.

454
00:30:20.912 --> 00:30:22.173
That's my motor cortex.

455
00:30:22.493 --> 00:30:22.734
Okay.

456
00:30:22.834 --> 00:30:23.094
Okay.

457
00:30:23.374 --> 00:30:27.397
So neurons in the down region in that little block.

458
00:30:28.483 --> 00:30:32.766
If those fire, that's going to trigger my Pong paddle, I guess.

459
00:30:33.266 --> 00:30:34.067
It's a paddle, right?

460
00:30:34.287 --> 00:30:35.367
It's like a digital paddle.

461
00:30:35.527 --> 00:30:36.088
I know what you mean.

462
00:30:36.128 --> 00:30:37.409
You know what I mean, right?

463
00:30:37.589 --> 00:30:40.090
It's going to trigger the paddle to go down.

464
00:30:40.471 --> 00:30:46.874
And if the neurons in the up region fire for those electrodes, then it's going to trigger the neurons to go up.

465
00:30:47.255 --> 00:30:51.978
And sometimes you will have to calculate a difference between the up and down because, you know, maybe there's more...

466
00:30:52.598 --> 00:30:54.219
neurons in the down region versus the up.

467
00:30:54.239 --> 00:31:02.665
So I also have to normalize based on how many neurons there are, because I don't want to just artificially keep going down if there's randomly more neurons in that region.

468
00:31:02.865 --> 00:31:10.091
So there's a bunch of mathematical tricks you have to do, but at the end of the day, segregated regions, one set of neurons is going to fire if

469
00:31:10.691 --> 00:31:16.412
the culture wants to go down, and one set of neurons is going to fire if that petri dish culture is going to want to go up.

470
00:31:16.672 --> 00:31:16.873
Yes.

471
00:31:16.913 --> 00:31:17.113
Okay?

472
00:31:17.333 --> 00:31:20.613
So now we've figured out our read and our write mechanism.

473
00:31:20.673 --> 00:31:26.055
The write is the sensory cortex at the top, eight electrodes, and our read is getting it in.

474
00:31:26.415 --> 00:31:38.358
And I think an interesting point about this is that geographical or locational, where it's located and how it's segregated, like form is function, or form is following function in this context.

475
00:31:38.398 --> 00:31:39.058
Exactly, yeah.

476
00:31:39.358 --> 00:31:46.501
Now, there's one other thing that I want to touch upon, which is how exactly that writing mechanism is being implemented.

477
00:31:46.521 --> 00:31:46.761
Yeah.

478
00:31:46.801 --> 00:31:46.941
Right?

479
00:31:46.961 --> 00:31:51.183
Because what do we need to give this Petri dish of neurons information about?

480
00:31:51.323 --> 00:31:51.523
Yes.

481
00:31:51.563 --> 00:31:53.124
What is all the information we need to give it?

482
00:31:53.144 --> 00:31:55.825
Well, we need to give it a state of the game, right?

483
00:31:55.885 --> 00:31:59.026
Which is where is the paddle and where is the ball, right?

484
00:31:59.486 --> 00:32:00.307
In relation to that.

485
00:32:00.427 --> 00:32:00.627
Yes.

486
00:32:00.647 --> 00:32:00.727
Yes.

487
00:32:01.259 --> 00:32:05.521
Now, for us, when we play Pong, we've got a very nice visual system.

488
00:32:05.561 --> 00:32:06.922
So we see where the paddle is.

489
00:32:06.962 --> 00:32:07.962
We see where the ball is.

490
00:32:07.982 --> 00:32:09.623
And we also see how it's moving.

491
00:32:10.103 --> 00:32:20.488
And so we can sort of all of our billions of neurons in our brain can like understand spatially what's going on because we understand space because we've had so much pre-training.

492
00:32:20.508 --> 00:32:20.888
Yeah.

493
00:32:21.603 --> 00:32:23.345
And we can figure it out.

494
00:32:23.545 --> 00:32:23.725
Yeah.

495
00:32:24.066 --> 00:32:25.707
This is a neuron in a Petri dish.

496
00:32:25.827 --> 00:32:26.848
It does not have eyes.

497
00:32:26.969 --> 00:32:28.090
It does not have senses.

498
00:32:28.470 --> 00:32:29.831
We are creating the senses.

499
00:32:29.912 --> 00:32:30.893
So how are we going to do it?

500
00:32:30.953 --> 00:32:34.716
Well, they use something called rate and place coding.

501
00:32:34.917 --> 00:32:38.100
This is something that is very commonly observed in science.

502
00:32:39.397 --> 00:32:40.097
The human brain.

503
00:32:40.297 --> 00:32:40.577
Okay.

504
00:32:40.777 --> 00:32:41.017
Okay?

505
00:32:42.058 --> 00:32:46.919
For example, let's take the example of the human cochlea, which is in our ear.

506
00:32:46.979 --> 00:32:48.139
That's how we sense sound.

507
00:32:48.579 --> 00:32:50.080
We sense frequency.

508
00:32:50.180 --> 00:32:51.300
And we sense loudness.

509
00:32:51.760 --> 00:32:52.640
Right?

510
00:32:53.280 --> 00:32:55.221
I've touched upon the cochlea before.

511
00:32:55.781 --> 00:33:03.043
The idea with the cochlea is there's like this tube, a snail-like tube, that is embedded in our ear.

512
00:33:03.063 --> 00:33:03.363
Right?

513
00:33:04.102 --> 00:33:11.952
And what that tube is, if you were to unravel it, there would be neurons that are very thick at one end, and there would be neurons that are very thin at the other end.

514
00:33:12.312 --> 00:33:16.157
This thing gets wrapped up into kind of like a shell shape, right?

515
00:33:16.898 --> 00:33:21.463
But if I hear sound of a certain frequency, that is going to...

516
00:33:22.244 --> 00:33:27.766
excite a neuron along this axis that is at exactly the right thickness, right?

517
00:33:27.866 --> 00:33:36.009
If the sound is very low frequency, like a bass sound, that's going to excite the thicker neurons because they're bigger and the wavelength of the sound is bigger.

518
00:33:36.049 --> 00:33:40.471
So it's kind of just like, you know, the resonance is happening.

519
00:33:40.731 --> 00:33:43.272
And if the sound is really high, then it's going to...

520
00:33:44.237 --> 00:33:46.619
Excite the smaller neurons in my cochlea.

521
00:33:46.900 --> 00:33:47.060
Yes.

522
00:33:47.120 --> 00:33:48.301
So that's a place coding.

523
00:33:48.721 --> 00:33:49.983
That tells me the frequency.

524
00:33:50.203 --> 00:33:50.483
Okay.

525
00:33:50.543 --> 00:33:50.763
Okay.

526
00:33:50.863 --> 00:33:55.968
That's my place because my place along the axis is telling me the frequency of the sound.

527
00:33:56.128 --> 00:33:56.329
Yes.

528
00:33:56.809 --> 00:33:59.492
The rate coding is how loud is that sound?

529
00:33:59.512 --> 00:33:59.932
There it is.

530
00:34:00.212 --> 00:34:05.677
Because if it's very loud, then that neuron is going to fire at a very high frequency.

531
00:34:05.797 --> 00:34:08.579
If it's not that loud, it's going to fire at a low frequency.

532
00:34:08.799 --> 00:34:12.222
So I'm getting two pieces of information with place and rate.

533
00:34:12.583 --> 00:34:14.024
The place is telling me the frequency.

534
00:34:14.364 --> 00:34:16.326
The rate is telling me the loudness.

535
00:34:16.506 --> 00:34:19.088
And this is something that is very common in biology.

536
00:34:19.529 --> 00:34:23.092
Where the neurons are firing tells us one piece of information.

537
00:34:23.452 --> 00:34:26.735
And how much the neurons are firing tells me another piece of information.

538
00:34:27.015 --> 00:34:29.457
And those are simultaneously present in the world.

539
00:34:29.637 --> 00:34:33.379
when biological neural networks create computation.

540
00:34:34.019 --> 00:34:44.084
And just to continue to bring this back to the artificial comparison, I think what's interesting here is this is different than just being zeros or ones.

541
00:34:44.304 --> 00:34:44.544
Yes.

542
00:34:44.564 --> 00:34:47.606
Because you actually have two levels of input.

543
00:34:47.646 --> 00:34:50.267
You have the place, which you could...

544
00:34:52.168 --> 00:34:53.409
Let me not make that direct comparison.

545
00:34:53.489 --> 00:34:56.231
But the point is it is not as simple as a binary system.

546
00:34:56.251 --> 00:34:56.451
Yeah.

547
00:34:56.791 --> 00:35:00.093
It's more like of a two-state modulator.

548
00:35:00.213 --> 00:35:00.414
Yeah.

549
00:35:00.574 --> 00:35:06.337
No, I think you're on to something here in that the neuron is kind of an analog system, right, in that sense.

550
00:35:06.718 --> 00:35:12.381
Now, some people will argue that the neurons are still digital because they are sending out spikes.

551
00:35:12.782 --> 00:35:17.765
And if you were to condense time and discretize it into smaller and smaller segments—

552
00:35:18.105 --> 00:35:22.068
Then in each time window, the neuron is either active or inactive, right?

553
00:35:22.448 --> 00:35:27.791
And the idea of a high rate is just there's more time windows that it's active, right?

554
00:35:27.912 --> 00:35:36.137
So there is a really healthy debate in the neuroscience community about whether to interpret neurons as analog or digital, right?

555
00:35:36.437 --> 00:35:38.078
And I don't know where I fall on that.

556
00:35:38.198 --> 00:35:39.199
Okay, that's fair.

557
00:35:39.539 --> 00:35:42.601
But I think it's a fair point that you're making.

558
00:35:42.762 --> 00:35:44.583
Like, it's a very important point.

559
00:35:44.823 --> 00:35:45.704
Right, right, right.

560
00:35:45.744 --> 00:35:47.565
It's like that is still up for grabs.

561
00:35:47.725 --> 00:35:58.033
Because the answer to that will also then be relevant to, like, our conversation about LLMs versus other models that we had in the last episode in terms of if...

562
00:35:59.114 --> 00:35:59.995
Anyway, you get the idea.

563
00:36:00.055 --> 00:36:00.495
Yeah, yeah.

564
00:36:01.115 --> 00:36:04.198
So, to be clear, I just want to reiterate here.

565
00:36:06.115 --> 00:36:12.757
What we're doing is using something that is in place and something that the brain is already used to, which is place and rate coding.

566
00:36:13.057 --> 00:36:21.120
In the idea of the cochlea, the place along this axis was the frequency and the rate at which that neuron fired was the loudness.

567
00:36:22.420 --> 00:36:31.023
In this case, how are we actually, if we go back to photo 14, actually, how are we actually going to use that sensory frequency?

568
00:36:32.299 --> 00:36:36.220
Cortex, the eight electrodes that we're trying to write in our information.

569
00:36:36.520 --> 00:36:43.522
How are we going to use place and rate coding to inform my Petri dish brain about the state of the game?

570
00:36:43.862 --> 00:36:44.842
That's the key, right?

571
00:36:45.782 --> 00:36:49.183
What they did was they said something very interesting.

572
00:36:49.503 --> 00:36:56.905
The where the ball is or where the sorry, where the paddle is.

573
00:36:57.165 --> 00:36:57.345
Yes.

574
00:36:58.718 --> 00:36:59.938
is going to be my place.

575
00:37:01.158 --> 00:37:01.419
Okay?

576
00:37:01.899 --> 00:37:08.740
So if my paddle is lower on the screen, I'm going to fire the electrodes on the left.

577
00:37:08.820 --> 00:37:12.821
I'm going to make the left electrodes give some electricity to my brain.

578
00:37:13.241 --> 00:37:17.522
If the paddle is up on the screen, I'm going to give the right electrodes.

579
00:37:17.842 --> 00:37:19.042
Okay, so now I've got place.

580
00:37:19.302 --> 00:37:20.582
What's the other thing I need to code in?

581
00:37:20.622 --> 00:37:21.883
That's the distance to the ball.

582
00:37:22.203 --> 00:37:23.123
How close is the ball?

583
00:37:23.223 --> 00:37:24.063
How close is it getting?

584
00:37:24.863 --> 00:37:27.005
And so for that, they used frequency.

585
00:37:27.565 --> 00:37:30.187
They went all the way from 4 hertz to 40 hertz.

586
00:37:30.567 --> 00:37:32.068
4 hertz means it's very far away.

587
00:37:32.088 --> 00:37:33.809
40 hertz means it's getting close.

588
00:37:33.829 --> 00:37:36.570
You need to figure out which way to go.

589
00:37:38.512 --> 00:37:45.196
And the idea here is we are basically creating a translation between what we experience...

590
00:37:46.036 --> 00:37:47.958
As a visual input.

591
00:37:48.018 --> 00:37:48.298
Mm-hmm.

592
00:37:48.899 --> 00:37:53.524
Where we can do the rate and place just through our visual input.

593
00:37:53.564 --> 00:37:53.984
That's right.

594
00:37:54.625 --> 00:37:58.769
Into an electrical input such that... That is...

595
00:38:00.459 --> 00:38:24.882
regional as a means to create an extra layer of like degree of freedom in in in encoding more data in what is a very relatively flat yes system yeah exactly and we need to be able to give those neurons in my petri dish both of these pieces of information right and one thing i do want to be clear here is the neurons in this petri dish are not directly seeing the game right like we are this is kind of what i'm trying to right yeah

596
00:38:25.543 --> 00:38:26.003
poke at.

597
00:38:26.043 --> 00:38:26.304
Yes.

598
00:38:26.604 --> 00:38:28.525
They're not seeing the game like humans do.

599
00:38:28.825 --> 00:38:32.648
Instead, they're sort of seeing the game based on the paddle's point of view, in some sense.

600
00:38:33.008 --> 00:38:35.410
It's like, where am I if I'm the paddle?

601
00:38:35.730 --> 00:38:37.591
And also, how far away is the ball?

602
00:38:37.791 --> 00:38:37.992
Right.

603
00:38:38.272 --> 00:38:39.032
Those are the things.

604
00:38:39.813 --> 00:38:40.733
And the only...

605
00:38:41.594 --> 00:38:44.256
The resolution is only like eight pixels in some sense.

606
00:38:44.856 --> 00:38:45.076
Right?

607
00:38:45.657 --> 00:38:49.740
But from that, they're trying to make this Petri Dish play Pong.

608
00:38:50.160 --> 00:38:50.400
Okay?

609
00:38:50.740 --> 00:38:53.082
So, the other thing that we need to think about...

610
00:38:54.868 --> 00:38:55.248
It's fine.

611
00:38:55.288 --> 00:38:56.868
Now we've got a read and write mechanism.

612
00:38:57.048 --> 00:38:57.289
Yes.

613
00:38:57.609 --> 00:38:59.169
Now we need a training mechanism.

614
00:38:59.969 --> 00:39:00.229
Right.

615
00:39:00.649 --> 00:39:00.869
Right?

616
00:39:01.089 --> 00:39:01.229
Yeah.

617
00:39:01.249 --> 00:39:12.552
Because now all of a sudden, I mean, we're telling the Petri dish the state of the game, and we have some way of reading what the Petri dish wants to do, which is move the paddle up or move the paddle down.

618
00:39:12.892 --> 00:39:15.872
How does the Petri dish know whether it's doing well or not?

619
00:39:15.992 --> 00:39:16.292
Yes.

620
00:39:16.473 --> 00:39:16.673
Right?

621
00:39:17.213 --> 00:39:23.034
The challenge is actually quite non-trivial because in the brain, how do we get a reward for...

622
00:39:24.778 --> 00:39:26.259
You know, we feel happiness.

623
00:39:27.020 --> 00:39:28.821
Maybe we got a cookie, literally.

624
00:39:29.401 --> 00:39:37.747
That is going to fuel dopaminergic neurons in our brain that are going to send signals of dopamine, pleasure, and all this other kind of stuff.

625
00:39:38.148 --> 00:39:40.489
So we've got like emotional signals for reward.

626
00:39:40.529 --> 00:39:42.811
We've got other crazy signals for reward.

627
00:39:42.831 --> 00:39:44.092
This is a brain in a Petri dish.

628
00:39:44.552 --> 00:39:46.373
It does not have dopaminergic neurons.

629
00:39:46.573 --> 00:39:50.975
It does not have these centers and this complex brain anatomy to have reward.

630
00:39:51.215 --> 00:39:51.395
Yes.

631
00:39:51.935 --> 00:39:52.976
So how are we going to do that?

632
00:39:54.256 --> 00:39:57.918
And this, I think, is actually the coolest part of the paper to me.

633
00:39:58.118 --> 00:39:58.438
Okay.

634
00:39:58.498 --> 00:39:58.738
Okay?

635
00:39:59.939 --> 00:40:07.042
It's this idea of a closed-loop feedback architecture, and the learning mechanism is literally minimizing surprise.

636
00:40:08.283 --> 00:40:09.063
Here's what they're doing.

637
00:40:09.823 --> 00:40:13.505
If there is a successful hit, meaning the Pong...

638
00:40:14.998 --> 00:40:19.362
paddle thing, hit the ball, okay?

639
00:40:19.442 --> 00:40:20.382
And we're good to go.

640
00:40:20.783 --> 00:40:30.811
What it's going to do is all of the electrodes in my multi-electrode array is going to send out a predictable electrical pulse

641
00:40:32.045 --> 00:40:40.787
It's about 10 volts on every single electrode at 10 hertz, so 10 every second for about 50 milliseconds.

642
00:40:40.907 --> 00:40:43.088
Very short, very highly structured.

643
00:40:43.288 --> 00:40:43.568
Okay.

644
00:40:43.628 --> 00:40:43.868
Okay?

645
00:40:44.608 --> 00:40:53.630
If, on the other hand, it misses, it's going to get four seconds at 20 volts of highly chaotic electrical activity.

646
00:40:53.750 --> 00:40:53.910
Mm-hmm.

647
00:40:54.842 --> 00:40:57.165
And the idea is brains hate that.

648
00:40:57.586 --> 00:40:57.826
Yes.

649
00:40:57.866 --> 00:40:59.989
They want to be able to predict the future.

650
00:41:00.149 --> 00:41:00.370
Yes.

651
00:41:00.470 --> 00:41:00.750
Okay?

652
00:41:01.131 --> 00:41:11.645
And when you're giving them highly unpredictable stuff, even this naive network of biological neurons wants to decrease the amount of times it gets that.

653
00:41:12.639 --> 00:41:15.463
So if we look at the GIF, I just wanted to show one thing.

654
00:41:15.663 --> 00:41:17.365
So here, we've got the brain coming in.

655
00:41:17.405 --> 00:41:19.988
You see that bump in electrical activity everywhere?

656
00:41:20.248 --> 00:41:21.250
And now it's about to miss.

657
00:41:22.051 --> 00:41:22.651
It's going to miss.

658
00:41:23.432 --> 00:41:26.596
For four seconds, it's just like there's noise everywhere.

659
00:41:26.676 --> 00:41:26.896
Yes.

660
00:41:27.317 --> 00:41:29.400
Okay, we're feeding a bunch of electrical noise.

661
00:41:29.420 --> 00:41:30.281
The neurons hate that.

662
00:41:30.761 --> 00:41:31.082
Now...

663
00:41:32.458 --> 00:41:36.700
The, it's going to go, it's going to actually, oh, I guess it's going to recycle.

664
00:41:36.740 --> 00:41:38.321
Let's recycle.

665
00:41:38.341 --> 00:41:39.182
So it'll recycle.

666
00:41:39.202 --> 00:41:43.364
When it gets there, a bunch of predictable electrical signal.

667
00:41:43.424 --> 00:41:44.444
That's all we're doing.

668
00:41:44.644 --> 00:41:44.845
Yes.

669
00:41:45.445 --> 00:41:51.828
Our reward mechanism is simply, if you got it right, we're going to reward you with a bunch of predictable activity.

670
00:41:52.068 --> 00:41:52.248
Yep.

671
00:41:53.549 --> 00:41:55.270
10 hertz, 50 milliseconds.

672
00:41:55.950 --> 00:42:00.533
And if you get it wrong, four seconds of random chaotic activity.

673
00:42:00.553 --> 00:42:00.893
Okay.

674
00:42:01.422 --> 00:42:16.145
And I think this dovetails with some sort of historical science research that's been done around the, you know, when we look at how do we get ordered systems with entropy and this idea of everything goes to soup.

675
00:42:16.665 --> 00:42:21.086
And the way in which that works is by minimizing surprise.

676
00:42:21.226 --> 00:42:21.486
Yes.

677
00:42:21.506 --> 00:42:22.007
This concept.

678
00:42:22.027 --> 00:42:23.327
It's like a biological.

679
00:42:23.527 --> 00:42:27.968
It is how life fundamentally is able to instantiate itself.

680
00:42:27.988 --> 00:42:28.148
Yeah.

681
00:42:29.008 --> 00:42:40.235
And we're basically hijacking that natural kind of concept by artificially punishing the wrong decision.

682
00:42:40.515 --> 00:42:41.036
With chaos.

683
00:42:41.076 --> 00:42:43.537
With chaos, which makes the system work harder.

684
00:42:44.078 --> 00:42:46.879
And it is naturally going to want to not work harder.

685
00:42:46.959 --> 00:42:47.160
Yeah.

686
00:42:47.260 --> 00:42:49.161
And so that's why it chooses the right decision.

687
00:42:49.321 --> 00:42:49.921
Yeah.

688
00:42:50.041 --> 00:42:53.003
It doesn't really know that it's the right decision.

689
00:42:53.183 --> 00:42:53.384
Exactly.

690
00:42:53.404 --> 00:42:55.645
It's just optimizing for the less possible.

691
00:42:56.372 --> 00:42:58.694
The more energy efficient pathway.

692
00:42:58.714 --> 00:43:00.035
Yes, the less surprising.

693
00:43:00.055 --> 00:43:00.796
The less surprising.

694
00:43:00.856 --> 00:43:01.557
Sorry, the less surprising.

695
00:43:01.577 --> 00:43:05.280
Yeah, it's really the less surprising outcome is what it wants, right?

696
00:43:05.560 --> 00:43:07.141
And now let's look at the results.

697
00:43:07.321 --> 00:43:07.582
Okay.

698
00:43:07.702 --> 00:43:07.962
Okay?

699
00:43:08.382 --> 00:43:10.784
With this training paradigm, let's look at the results.

700
00:43:10.824 --> 00:43:13.527
The apparent learning is demonstrated in five minutes.

701
00:43:13.847 --> 00:43:16.849
So here what we're looking at is two sets of box plots.

702
00:43:17.390 --> 00:43:20.612
The green box plot is what happens from zero to five minutes.

703
00:43:20.873 --> 00:43:20.993
Okay.

704
00:43:21.013 --> 00:43:21.213
Okay.

705
00:43:21.653 --> 00:43:28.079
Um, we're looking at the change in rally length, which is like the, you know, the, how many times the rally happens for, for the pong.

706
00:43:28.099 --> 00:43:31.242
It's just like in tennis, like how many times do I keep the ball alive?

707
00:43:31.743 --> 00:43:39.571
Um, and you can see from six to 20 minutes, there's a giant change in the size of the rallies from zero to five.

708
00:43:39.591 --> 00:43:42.333
So within five minutes, this thing is learning quite well.

709
00:43:42.994 --> 00:43:43.374
There's a...

710
00:43:44.375 --> 00:44:00.623
up change in rally and that's for the stimulus for the controls which is silent and no feedback there's no change yeah okay so we're getting a significant change only when they're actually playing the game and they have this feedback yes now if we go to the next photo what we're also seeing is that human cortical neurons

711
00:44:01.427 --> 00:44:04.149
do a lot better than mouse cortical neurons.

712
00:44:04.169 --> 00:44:04.369
Yes.

713
00:44:04.769 --> 00:44:05.009
Okay?

714
00:44:05.169 --> 00:44:05.349
Yes.

715
00:44:05.390 --> 00:44:09.312
And that, I don't know, that makes me feel good that my neurons are better than a mouse's neurons.

716
00:44:09.332 --> 00:44:10.313
We are better.

717
00:44:10.393 --> 00:44:10.853
Right?

718
00:44:10.953 --> 00:44:12.334
And that kind of makes sense.

719
00:44:12.494 --> 00:44:16.757
We've had evolution, and our brains arguably are probably better than mouse.

720
00:44:17.677 --> 00:44:19.338
And the thing is, there was an improvement in both cases.

721
00:44:19.358 --> 00:44:22.540
It's just our improvement had a... Had a big... That's a good point.

722
00:44:22.620 --> 00:44:23.001
Very good.

723
00:44:23.021 --> 00:44:23.201
Yeah.

724
00:44:23.501 --> 00:44:26.743
It's that ours had a much bigger improvement than the mice.

725
00:44:27.083 --> 00:44:27.924
As compared to the mice.

726
00:44:28.304 --> 00:44:28.484
Right?

727
00:44:29.305 --> 00:44:29.525
So...

728
00:44:30.681 --> 00:44:36.807
Now that we have that, let's actually look at this free energy principle that you were alluding to a bit earlier.

729
00:44:37.067 --> 00:44:37.708
Oh, interesting.

730
00:44:37.768 --> 00:44:38.068
Okay.

731
00:44:38.128 --> 00:44:39.970
That's what it's called, the free energy principle.

732
00:44:40.330 --> 00:44:43.733
This was first pioneered by Carl Fristen.

733
00:44:44.647 --> 00:44:53.511
And the idea is, what you were saying, all self-organizing biological systems want to minimize something called the free energy, the variational free energy.

734
00:44:53.911 --> 00:45:01.715
In other sense, what we want to do is maximize the information entropy, and we want to minimize the surprise and the prediction error.

735
00:45:03.584 --> 00:45:09.866
What we can look at is how much surprise there is for what is the probability of a certain state.

736
00:45:10.566 --> 00:45:15.227
If the probability of a certain state is very low, there's a lot of surprise, right?

737
00:45:15.607 --> 00:45:22.949
And you can quantify that in the same way that we actually quantify entropy in physics, which is the log of the probability of stuff.

738
00:45:24.329 --> 00:45:28.650
And effectively, what the free energy principle is saying is that biological systems...

739
00:45:29.771 --> 00:45:39.294
want to maintain homeostasis, meaning like constancy over all the random stuff, and we want to minimize that surprise, right?

740
00:45:39.874 --> 00:45:42.115
And what do we do if we want to do that?

741
00:45:42.175 --> 00:45:55.520
Well, we've got some internal model of what the world is, and we are going to tune the parameters of that internal model such that that internal model mimics more closely the world that we live in, right?

742
00:45:55.860 --> 00:45:56.640
That is the idea.

743
00:45:58.077 --> 00:46:03.786
It's a stark departure, I should say, and highlight, from traditional reinforcement learning.

744
00:46:04.287 --> 00:46:04.567
Okay.

745
00:46:04.688 --> 00:46:04.948
Okay?

746
00:46:05.228 --> 00:46:12.239
In traditional reinforcement learning, you've got some algorithm, and that algorithm explicitly gives...

747
00:46:13.229 --> 00:46:17.051
negative feedback through backpropagation for wrong trials.

748
00:46:17.512 --> 00:46:22.235
Here, we are letting chaos itself be the negative feedback.

749
00:46:22.255 --> 00:46:22.675
Yeah, yeah, yeah.

750
00:46:22.895 --> 00:46:30.320
Now, one of the things that's kind of crazy is that the algorithms for reinforcement learning require thousands of epochs of trial and error.

751
00:46:31.000 --> 00:46:33.861
to update these weights through backpropagation.

752
00:46:33.981 --> 00:46:40.404
Biological systems will inherently self-organize to minimize this free energy, and they'll do it very, very quickly.

753
00:46:40.564 --> 00:46:45.987
They've just got really nice cellular mechanisms to find that sweet spot in their internal model.

754
00:46:46.007 --> 00:46:48.208
This goes back to the energy efficiency piece again.

755
00:46:48.228 --> 00:46:49.569
Yes, exactly, right?

756
00:46:49.809 --> 00:46:51.950
And so maybe that's why we want to use the...

757
00:46:52.730 --> 00:46:54.873
Wetware in the first place, right?

758
00:46:55.314 --> 00:47:00.180
So let's go back to this FEP in a dish, our free energy principle in a dish.

759
00:47:00.641 --> 00:47:04.746
Effectively, what's happening is you've got predictions and then you've got prediction errors.

760
00:47:05.207 --> 00:47:08.331
The predictions are where I want to move my...

761
00:47:09.817 --> 00:47:10.377
Paddle.

762
00:47:10.717 --> 00:47:16.220
And the prediction error is the chaos that I get when I get it wrong.

763
00:47:16.720 --> 00:47:16.940
Right?

764
00:47:17.820 --> 00:47:25.663
I'm trying to change the connections in my neurons on my Petri dish such that I minimize prediction error.

765
00:47:26.003 --> 00:47:28.484
And that's effectively what these neurons are doing.

766
00:47:29.145 --> 00:47:31.145
It's actually pretty incredible when you think about it.

767
00:47:31.205 --> 00:47:35.367
We're not giving them a bad and good signal.

768
00:47:35.687 --> 00:47:36.848
We're only giving them...

769
00:47:38.814 --> 00:47:41.376
chaos versus non-chaos signal.

770
00:47:41.497 --> 00:47:42.217
Right, right.

771
00:47:42.257 --> 00:47:47.702
And that itself, just because of the nature of information and the physics of information...

772
00:47:48.938 --> 00:48:13.111
is enough to like steer them like this is bad and that's good i think that's like quite deep the universe the universe never chooses violence yeah yeah it always chooses to billy when given i i'm being a little facetious no but i think i think i think that's a very good that's a very poignant statement here so now let's look at like some of the hype that they created around it okay let's look at some of the promo videos and this was for the pong

773
00:48:13.411 --> 00:48:14.412
This is just 2022.

774
00:48:14.472 --> 00:48:15.674
We're still on 2022.

775
00:48:15.694 --> 00:48:17.115
Okay, just confirm it.

776
00:48:17.135 --> 00:48:19.298
Because I know people are listening, waiting for the Doom.

777
00:48:19.558 --> 00:48:22.362
Yeah, but we need to really understand this in order to get to Doom.

778
00:48:22.462 --> 00:48:22.702
100%.

779
00:48:22.782 --> 00:48:24.985
Okay, so this is one of their...

780
00:48:26.280 --> 00:48:26.960
Promo videos.

781
00:48:27.240 --> 00:48:28.101
Look at how many.

782
00:48:28.541 --> 00:48:29.901
This is doing really well.

783
00:48:30.201 --> 00:48:31.862
It's already at like rallies of three.

784
00:48:31.882 --> 00:48:32.302
Yeah.

785
00:48:32.662 --> 00:48:33.723
Maybe it'll get to four.

786
00:48:33.863 --> 00:48:34.023
Yeah.

787
00:48:34.223 --> 00:48:34.443
Yep.

788
00:48:34.623 --> 00:48:34.823
Yep.

789
00:48:34.883 --> 00:48:35.723
Is it going to get to four?

790
00:48:36.284 --> 00:48:36.484
Yeah.

791
00:48:36.504 --> 00:48:37.284
It got to four.

792
00:48:37.824 --> 00:48:38.865
Four rallies, right?

793
00:48:38.945 --> 00:48:41.346
Like this is, it's really, it's really doing it.

794
00:48:41.786 --> 00:48:42.166
Five.

795
00:48:42.306 --> 00:48:43.166
It was getting to five.

796
00:48:43.226 --> 00:48:43.446
Yes.

797
00:48:43.506 --> 00:48:45.387
Now let's look at the results from their paper.

798
00:48:45.527 --> 00:48:45.847
Okay.

799
00:48:46.007 --> 00:48:46.287
Okay.

800
00:48:46.467 --> 00:48:46.607
Okay.

801
00:48:46.627 --> 00:48:49.469
From the results from their paper, if we look at the left-hand side.

802
00:48:49.489 --> 00:48:49.829
Yes.

803
00:48:49.849 --> 00:48:49.969
Yes.

804
00:48:50.869 --> 00:48:54.473
So figure B, that's the average rally length.

805
00:48:54.513 --> 00:48:55.595
And look at the y-axis.

806
00:48:56.696 --> 00:48:58.138
What is the y-axis on?

807
00:48:58.558 --> 00:49:00.100
It's on 1, right?

808
00:49:00.561 --> 00:49:02.483
Most of their trials are on 1.

809
00:49:03.004 --> 00:49:09.892
Even on their box plot, the edge of their trials for human cortical neurons, which is the right-hand side box plot, is going to 1.5.

810
00:49:09.932 --> 00:49:10.092
Mm-hmm.

811
00:49:11.113 --> 00:49:11.353
Right.

812
00:49:11.853 --> 00:49:17.435
Which is that's the I think it's the third quartile usually for box plots.

813
00:49:17.455 --> 00:49:20.976
So it's like the 75 percent of data falls within one point five.

814
00:49:20.996 --> 00:49:25.398
So clearly they're like showing a very nice example in their video.

815
00:49:25.498 --> 00:49:25.758
Right.

816
00:49:26.558 --> 00:49:28.199
It's still statistically significant.

817
00:49:28.539 --> 00:49:33.081
If you look at the human cortical neurons, which is the right box plot in figure B.

818
00:49:33.141 --> 00:49:33.241
Yes.

819
00:49:33.481 --> 00:49:33.721
Yes.

820
00:49:34.441 --> 00:49:40.647
From zero to five minutes, the average rally length is at 0.5, and now it's bumped all the way to one.

821
00:49:41.088 --> 00:49:44.851
And if you look at the spread of that data, clearly it's highly significant.

822
00:49:45.272 --> 00:49:51.898
If you look at all the other metrics, you can see for human cortical neurons, they do the best compared to mouse cortical neurons and all the other controls.

823
00:49:52.138 --> 00:49:56.242
So there's clearly an effect, but the Cortical Labs team are a startup.

824
00:49:56.522 --> 00:49:56.702
Yes.

825
00:49:56.963 --> 00:49:57.844
And so, you know...

826
00:49:58.604 --> 00:50:06.194
as is tradition in startups, not just in Silicon Valley, but around the world, even in Australia, they're going to hype up some of the results.

827
00:50:06.314 --> 00:50:06.935
And that's fine.

828
00:50:07.256 --> 00:50:08.497
I still think it's a very cool...

829
00:50:09.589 --> 00:50:10.289
thing that they did.

830
00:50:10.630 --> 00:50:13.431
Yes, and we just walked through why it's incredibly cool.

831
00:50:13.672 --> 00:50:14.052
Yeah.

832
00:50:14.312 --> 00:50:17.794
And if you think that AI is using too much energy, you should be all for wetware.

833
00:50:18.715 --> 00:50:19.155
Yeah, yeah.

834
00:50:19.235 --> 00:50:22.517
I mean, it maybe has a lot of applications.

835
00:50:22.898 --> 00:50:24.439
So now we're going to scale this up.

836
00:50:24.639 --> 00:50:24.979
Okay.

837
00:50:25.059 --> 00:50:31.803
We're going to scale it up to complexity level 1000 because we are now trying to play Doom.

838
00:50:32.504 --> 00:50:34.425
They built the CL1 platform.

839
00:50:34.865 --> 00:50:36.787
Cortical Labs first first

840
00:50:37.945 --> 00:50:41.108
code deployable biological computer in the world.

841
00:50:41.248 --> 00:50:41.709
I love that.

842
00:50:41.749 --> 00:50:42.509
That's what it looks like.

843
00:50:42.650 --> 00:50:53.440
It's a little box that at its center has the Petri dish with all of the biological fluids and all the things to keep the neurons active.

844
00:50:53.800 --> 00:50:55.362
You've got the multi-electrode array.

845
00:50:55.382 --> 00:50:55.482
Yeah.

846
00:50:55.902 --> 00:51:02.984
And you've got a little screen on the side that shows you the health of the neurons, the amount of oxygen there is, the amount of CO2 there is, and so on and so forth.

847
00:51:03.304 --> 00:51:04.725
So it can regulate temperature.

848
00:51:04.745 --> 00:51:06.405
It can manage gas exchange.

849
00:51:06.745 --> 00:51:10.866
It can clear metabolic waste because these neurons are alive, so they're going to be creating waste.

850
00:51:11.326 --> 00:51:12.947
All of this other stuff, and they can actually...

851
00:51:13.747 --> 00:51:16.748
survive in that in vitro Petri dish for up to six months.

852
00:51:16.768 --> 00:51:17.869
That's a very long time.

853
00:51:17.909 --> 00:51:18.189
Yeah.

854
00:51:18.229 --> 00:51:18.549
Okay.

855
00:51:18.569 --> 00:51:21.711
For a sort of little mini organoid to survive.

856
00:51:22.531 --> 00:51:24.112
So how about the software?

857
00:51:24.332 --> 00:51:25.052
This is very cool.

858
00:51:25.092 --> 00:51:28.414
So for the, for the pong, for the pong, for the pong,

859
00:51:29.514 --> 00:51:29.915
story.

860
00:51:30.955 --> 00:51:34.999
They had a custom built C, C++ machine language type thing.

861
00:51:35.139 --> 00:51:37.281
Nobody wants to code in C++.

862
00:51:37.642 --> 00:51:44.728
I get that you want the base layer of your software to be in C++ because we want very low latency.

863
00:51:44.808 --> 00:51:44.988
Yes.

864
00:51:45.049 --> 00:51:45.269
Right?

865
00:51:45.329 --> 00:51:47.631
And C is an extremely fast language.

866
00:51:47.651 --> 00:51:48.612
Python is quite slow.

867
00:51:48.912 --> 00:51:50.834
But at the high level, I want to code in Python.

868
00:51:50.854 --> 00:51:50.954
Yeah.

869
00:51:51.634 --> 00:51:58.038
And then maybe the Python gets transferred into C. And what they did was they created something called a cortical cloud and a Python API.

870
00:51:58.078 --> 00:52:00.599
So now you don't need C++.

871
00:52:00.619 --> 00:52:09.824
And you've got a cloud computer of a bunch of CL1 platforms that are each little tiny...

872
00:52:12.060 --> 00:52:17.903
brains on a dish that I can now just like get into my, you know, on my laptop, I can access.

873
00:52:18.283 --> 00:52:20.084
It's open platform.

874
00:52:20.364 --> 00:52:23.065
I mean, you have to like apply and say like, this is what I want to do with it.

875
00:52:23.345 --> 00:52:28.688
But this is, I think, very clever from Cortical Labs because even IBM, like one of the reasons why

876
00:52:29.208 --> 00:52:46.138
Kiskit, which is the quantum computing platform for IBM, like their software platform, that's used a lot because they've created a cloud platform where theorists and like people who want to do testing of quantum algorithms and things like that, they can go into Kiskit, they can apply for time.

877
00:52:46.478 --> 00:52:47.819
on the IBM quantum computer.

878
00:52:47.879 --> 00:52:51.222
They've got like a dilution refrigerator with that IBM quantum computer.

879
00:52:51.462 --> 00:52:55.645
And they can like test out their algorithms and like their error correction methodologies and things like that.

880
00:52:56.006 --> 00:53:02.311
So democratizing it is a really good way for like a startup or like a small computer lab to be like, hey,

881
00:53:02.891 --> 00:53:03.952
We've created the hardware.

882
00:53:04.033 --> 00:53:05.034
Why don't you guys go at it?

883
00:53:05.054 --> 00:53:05.234
Yes.

884
00:53:05.534 --> 00:53:05.695
Right?

885
00:53:05.715 --> 00:53:05.835
Yes.

886
00:53:06.496 --> 00:53:07.737
It's very Matrix-esque.

887
00:53:08.018 --> 00:53:20.273
If you remember the scene where Neo unplugs and he sees all the pods of humans to capture heat for the machines, that's what that effectively wetware server rack, AWS NVIDIA server rack.

888
00:53:20.353 --> 00:53:20.814
Exactly.

889
00:53:20.834 --> 00:53:24.838
Yeah, it's like AWS, but for like wetware.

890
00:53:24.999 --> 00:53:25.800
It's pretty crazy.

891
00:53:26.000 --> 00:53:28.463
And AWS for a brain on a chip.

892
00:53:28.543 --> 00:53:30.065
That is so outrageous.

893
00:53:30.325 --> 00:53:40.097
Amazon Web Services, for those who don't know, is the infrastructure layer that provides the back end for a large amount of all of the software applications that you use every day.

894
00:53:40.317 --> 00:53:41.197
Yeah, exactly.

895
00:53:41.477 --> 00:53:48.720
So with that in mind, now that we've got a cloud computing platform, now independent researchers and developers can mess around with it.

896
00:53:48.820 --> 00:53:51.641
And this is where we get into Sean Cole.

897
00:53:51.941 --> 00:53:56.303
He's an independent developer who wanted to use the CL1 platform

898
00:53:57.043 --> 00:53:58.864
to play Doom Freedom.

899
00:53:59.165 --> 00:53:59.565
Classic.

900
00:53:59.745 --> 00:54:00.025
Okay?

901
00:54:00.446 --> 00:54:02.347
And the... You know, one of the...

902
00:54:02.447 --> 00:54:08.592
I don't know why this is a meme, but, like, whenever you have, like, a new piece of computation or something, everyone's like, can it play Doom?

903
00:54:08.612 --> 00:54:09.212
Can it play Doom?

904
00:54:09.653 --> 00:54:09.853
100%.

905
00:54:09.893 --> 00:54:10.333
Right?

906
00:54:10.573 --> 00:54:12.195
Like, even the ESA has, like...

907
00:54:12.935 --> 00:54:16.558
Made satellites play Doom and things like that.

908
00:54:16.638 --> 00:54:17.980
So I guess it's a meme.

909
00:54:18.700 --> 00:54:22.624
And the guys at Cortical Labs knew this was a meme.

910
00:54:22.904 --> 00:54:23.124
Yes.

911
00:54:23.244 --> 00:54:28.309
And knew that if they could make their little thing play Doom, it would get a lot of hype.

912
00:54:28.529 --> 00:54:29.670
And it certainly worked.

913
00:54:30.110 --> 00:54:39.218
And I just want to give a shout out to John Carmack, who was the inventor of Doom at id Software and literally was one of the pioneers that created all the video game.

914
00:54:40.299 --> 00:54:51.569
Like that was like the 3D platform, like that concept and what they did at the time is what created all first person shooter style or like all these types of games.

915
00:54:51.609 --> 00:54:53.510
So is Doom before?

916
00:54:53.690 --> 00:54:55.092
What's the James Bond?

917
00:54:55.112 --> 00:54:55.832
James Bond Goldeneye.

918
00:54:55.972 --> 00:54:56.173
Yeah.

919
00:54:56.193 --> 00:54:56.293
Yeah.

920
00:54:57.033 --> 00:55:00.275
I don't have a chronological knowledge.

921
00:55:00.575 --> 00:55:04.498
But I feel like James Bond GoldenEye was also kind of... Like, from what I've...

922
00:55:04.618 --> 00:55:07.940
I've never played Doom, but I've seen the videos while I was researching this.

923
00:55:08.020 --> 00:55:09.761
And it looked kind of like...

924
00:55:09.881 --> 00:55:10.541
It's similar.

925
00:55:10.742 --> 00:55:10.982
Yeah.

926
00:55:11.142 --> 00:55:11.802
But it's not Doom.

927
00:55:11.982 --> 00:55:13.243
Yeah, it's not Doom, I guess.

928
00:55:13.263 --> 00:55:14.304
I was a 64 guy.

929
00:55:14.424 --> 00:55:15.845
I played GoldenEye all the time.

930
00:55:15.865 --> 00:55:15.965
Yeah.

931
00:55:16.045 --> 00:55:16.905
And I loved the music.

932
00:55:17.185 --> 00:55:19.567
But Doom is sort of... Doom is kind of like this OG...

933
00:55:19.647 --> 00:55:20.307
It's its own thing.

934
00:55:20.327 --> 00:55:20.487
Yeah.

935
00:55:20.608 --> 00:55:20.888
So...

936
00:55:21.828 --> 00:55:23.389
It is way more complex than Pong.

937
00:55:23.949 --> 00:55:36.975
So, you know, one of the things that I, as I was like researching Doom and I was like watching some videos of people playing Doom, you know, it's really, you're transitioning from a 2D linear game like Pong to now it's kind of like this 2.5 dimensional.

938
00:55:37.015 --> 00:55:41.758
I wouldn't really say three dimensional because it's like first person and like,

939
00:55:43.079 --> 00:55:46.342
It's not really fully three-dimensional, I would say, but there's still...

940
00:55:46.443 --> 00:55:51.208
It involves complex navigation, threat detection, and all of this other very cool stuff.

941
00:55:51.548 --> 00:55:55.553
So it's a way harder problem than... Up, down, down.

942
00:55:56.798 --> 00:56:18.293
how close yes exactly right oh you're already getting there is like those are the sort of salient things that i need to code in you know before in pong i needed to code in position of my paddle and where the ball is here in this case i need to figure out where what direction the enemy is and how close they are anyways let's get to um the stem the cells we got 200 000 human

943
00:56:19.954 --> 00:56:21.095
Pluriports and stem cells.

944
00:56:21.175 --> 00:56:21.395
Yes.

945
00:56:21.655 --> 00:56:23.936
So this is a fourth of what we had in Pong.

946
00:56:24.476 --> 00:56:24.857
Oh, yeah.

947
00:56:24.877 --> 00:56:26.117
This is fewer than what we have in Pong.

948
00:56:26.137 --> 00:56:26.598
Yeah, exactly.

949
00:56:26.638 --> 00:56:26.838
Right?

950
00:56:26.858 --> 00:56:27.298
Okay, yep.

951
00:56:27.678 --> 00:56:37.984
But because of, like, the cloud computing nature and the fact that this is all in Python, the guys were able to map the engine to the tissue in just a week.

952
00:56:39.004 --> 00:56:40.265
And this is what it looks like.

953
00:56:40.545 --> 00:56:41.345
Jesus.

954
00:56:41.385 --> 00:56:48.669
So you've got your chip, and it's sensing motor neurons, or it's sensing the neurons, and then it's playing Doom.

955
00:56:48.970 --> 00:56:49.730
This is crazy.

956
00:56:49.890 --> 00:56:50.110
Right?

957
00:56:50.370 --> 00:56:51.030
Yeah, yeah, yeah.

958
00:56:51.250 --> 00:56:51.830
It's pretty cool.

959
00:56:51.910 --> 00:56:55.051
Yeah, and we're seeing this, like, the real neural activity.

960
00:56:55.071 --> 00:57:03.433
Yeah, those are the neurons that are coming in, and then the Petri dish is sort of controlling which way to point and which way to fire, right?

961
00:57:04.093 --> 00:57:06.854
So let's get into how actually they did it.

962
00:57:06.954 --> 00:57:10.195
Because in Pong, remember, we've got a sensory cortex, we've got a motor cortex.

963
00:57:10.495 --> 00:57:11.795
They're going to do very similar things.

964
00:57:13.276 --> 00:57:15.276
In this case, the...

965
00:57:17.680 --> 00:57:18.360
Proximity.

966
00:57:18.400 --> 00:57:21.583
We want to encode information about the game state, right?

967
00:57:21.963 --> 00:57:27.246
So there's proximity, which is how close I am.

968
00:57:27.486 --> 00:57:27.666
Yes.

969
00:57:28.487 --> 00:57:31.609
And there's direction, which is where I'm trying to fire.

970
00:57:32.349 --> 00:57:35.692
They're using place and rate to code that, right?

971
00:57:36.352 --> 00:57:38.153
Now, we want to take the damage.

972
00:57:38.573 --> 00:57:41.695
So taking damage and dying is going to be the chaos.

973
00:57:42.496 --> 00:57:45.738
And if we shoot someone, that's going to be the predictive.

974
00:57:45.758 --> 00:57:45.818
Ah.

975
00:57:46.518 --> 00:57:46.618
Ah...

976
00:57:46.658 --> 00:57:47.078
Reward.

977
00:57:47.098 --> 00:57:47.738
Okay, interesting.

978
00:57:47.758 --> 00:57:50.299
Because I was wondering how I was going to map from Pong to Doom.

979
00:57:50.419 --> 00:57:50.959
Yeah.

980
00:57:51.019 --> 00:57:51.759
And that's how you do it.

981
00:57:51.879 --> 00:57:57.240
And for the motor neurons, there's specific firing patterns that are going to be moved forward.

982
00:57:57.800 --> 00:58:00.140
There's specific firing patterns that are going to be turned.

983
00:58:00.541 --> 00:58:03.041
And there's specific firing patterns for fire a weapon.

984
00:58:03.321 --> 00:58:08.922
And the performance, after seven days of training, the agent's kill ratio was roughly twice that of pure chance.

985
00:58:09.282 --> 00:58:11.102
Now, I don't know what that means.

986
00:58:11.702 --> 00:58:14.403
And this is where it gets kind of annoying because there's no paper.

987
00:58:14.423 --> 00:58:14.723
Yeah.

988
00:58:14.963 --> 00:58:15.143
Right.

989
00:58:15.183 --> 00:58:15.964
So there's no methods.

990
00:58:16.004 --> 00:58:17.084
They just kind of said that.

991
00:58:17.264 --> 00:58:17.464
Right.

992
00:58:17.704 --> 00:58:20.005
I'm like, I don't I don't really know what that means.

993
00:58:20.145 --> 00:58:20.325
Yep.

994
00:58:20.466 --> 00:58:25.848
But they're explicitly admitting that the cells played like a beginner who's never seen a computer.

995
00:58:26.188 --> 00:58:27.049
So it's not very good.

996
00:58:27.169 --> 00:58:27.509
Yeah.

997
00:58:27.569 --> 00:58:27.769
Right.

998
00:58:28.049 --> 00:58:30.010
But it's still, I guess, higher than chance.

999
00:58:30.250 --> 00:58:30.470
Yes.

1000
00:58:30.730 --> 00:58:31.931
So maybe that's something.

1001
00:58:32.471 --> 00:58:38.294
One of the things that I found kind of interesting was that during the study, the agents actually generally settled settled.

1002
00:58:39.314 --> 00:58:43.657
for a strategy of survival, which is hiding rather than active combat.

1003
00:58:43.957 --> 00:58:47.279
And this is something that is well known in reinforcement learning, right?

1004
00:58:47.319 --> 00:58:51.402
You just like go for the local minima, which is I don't want to deal with anything.

1005
00:58:51.502 --> 00:59:00.808
I don't want any reward whatsoever, even the positive or the negative, because that positive or negative reward comes from a dense like chain of brittle action, right?

1006
00:59:00.828 --> 00:59:01.568
I have to do stuff.

1007
00:59:01.588 --> 00:59:02.169
I have to do stuff.

1008
00:59:02.189 --> 00:59:06.351
And at the very end, I might just get chaos and then it might just be bad.

1009
00:59:06.371 --> 00:59:08.853
So why don't I just like go to a corner and try to hide?

1010
00:59:09.213 --> 00:59:17.680
I wish they gave the nuke testing AI paper people the same wetware to go to the local minimum.

1011
00:59:17.700 --> 00:59:27.027
Because as we talked in our two episodes ago, all the major model companies, Anthropic, Gemini, OpenAI, all used nukes 90% of the time.

1012
00:59:27.047 --> 00:59:27.227
Yeah.

1013
00:59:27.467 --> 00:59:29.229
Which does not sound like what you're talking about.

1014
00:59:29.249 --> 00:59:29.449
Yeah.

1015
00:59:29.469 --> 00:59:29.689
Yeah.

1016
00:59:29.729 --> 00:59:29.889
No.

1017
00:59:29.929 --> 00:59:31.350
This one is just, they just want to.

1018
00:59:31.470 --> 00:59:32.691
They're like, we're just trying to hide.

1019
00:59:32.711 --> 00:59:32.851
Yeah.

1020
00:59:32.871 --> 00:59:33.812
I'm just going to try and hide.

1021
00:59:34.573 --> 00:59:34.773
So.

1022
00:59:35.755 --> 00:59:40.718
Now we've covered what Cortical Labs is purporting they did in their YouTube video.

1023
00:59:40.738 --> 00:59:42.399
This is where I got a lot of that information.

1024
00:59:42.599 --> 00:59:42.819
Yes.

1025
00:59:43.380 --> 00:59:45.661
Now let's ask what's actually going on.

1026
00:59:45.761 --> 00:59:46.241
Yeah, yeah.

1027
00:59:46.261 --> 00:59:51.684
Okay, because there's a lot less info about the Doom game versus the Pong game.

1028
00:59:51.744 --> 00:59:54.006
The Pong game had a full-ass...

1029
00:59:54.983 --> 00:59:56.744
You know, peer-reviewed paper in Neuron.

1030
00:59:56.884 --> 00:59:57.605
This one does not.

1031
00:59:57.665 --> 01:00:00.667
This one is just, like, blogs is all I got.

1032
01:00:00.767 --> 01:00:01.047
Okay?

1033
01:00:01.748 --> 01:00:02.668
Here's my concern.

1034
01:00:02.748 --> 01:00:08.793
And this is something that I actually read Tommy Blanchard's sub stack on.

1035
01:00:09.033 --> 01:00:12.655
So if you guys want to check out his sub stack, he's got a lot more info on it.

1036
01:00:13.196 --> 01:00:13.956
Here's the concern.

1037
01:00:14.917 --> 01:00:16.918
There are eight input channels for Pong, right?

1038
01:00:17.119 --> 01:00:18.520
But Pong is a very small game.

1039
01:00:19.200 --> 01:00:20.181
I can reasonably...

1040
01:00:21.544 --> 01:00:24.185
Put all of that information into eight electrodes.

1041
01:00:24.605 --> 01:00:28.507
How do I encode everything that's going on in Doom with only eight inputs?

1042
01:00:28.647 --> 01:00:28.927
Yeah.

1043
01:00:29.127 --> 01:00:33.089
Each of which can only have a few possible values, like either the electrode is on or off.

1044
01:00:33.369 --> 01:00:34.770
And the answer is you actually don't.

1045
01:00:35.490 --> 01:00:37.531
You use a reinforcement learning model.

1046
01:00:38.091 --> 01:00:39.912
This is where it gets a little bit fishy.

1047
01:00:40.032 --> 01:00:40.192
Okay.

1048
01:00:40.212 --> 01:00:40.553
Okay?

1049
01:00:40.653 --> 01:00:40.933
Okay.

1050
01:00:41.573 --> 01:00:44.854
Here, this is from their Doom Neuron.

1051
01:00:45.034 --> 01:00:51.015
Like, they've got a little, it's kind of like a GitHub where they have a documentation on what they did.

1052
01:00:51.735 --> 01:00:58.177
This is the architecture of what they're showing is they've got a computation unit, and that is talking to their CL1 computer.

1053
01:00:58.237 --> 01:00:58.537
Yeah, yeah.

1054
01:00:58.577 --> 01:00:58.797
Okay?

1055
01:00:59.217 --> 01:01:04.458
In that computational unit, on the left, right-hand side, what you can see is there is a PPO policy.

1056
01:01:04.498 --> 01:01:04.798
Yeah, yeah.

1057
01:01:04.918 --> 01:01:07.579
Which is a reinforcement learning paradigm.

1058
01:01:07.599 --> 01:01:07.999
Yeah, yeah.

1059
01:01:08.179 --> 01:01:12.024
That is then encoding the input of the game.

1060
01:01:12.424 --> 01:01:16.328
That is then going as stimulation to the brain, to my brain in a Petri dish.

1061
01:01:16.689 --> 01:01:18.070
And what I get back is action.

1062
01:01:18.131 --> 01:01:21.414
And that is getting fed back into my reinforcement learning algorithm.

1063
01:01:21.835 --> 01:01:25.319
So now I can very easily ask who is actually doing the learning?

1064
01:01:25.599 --> 01:01:25.920
Correct.

1065
01:01:26.100 --> 01:01:27.962
Is it my reinforcement learning algorithm?

1066
01:01:28.517 --> 01:01:31.039
Or is it the actual biological neurons that I have?

1067
01:01:31.079 --> 01:01:32.280
It doesn't count.

1068
01:01:32.360 --> 01:01:34.261
It shouldn't really count.

1069
01:01:34.301 --> 01:01:35.102
It's not pure.

1070
01:01:35.122 --> 01:01:36.483
And it's not that far away.

1071
01:01:36.523 --> 01:01:41.766
Like reinforcement learning, there's video games that have perfected reinforcement learning.

1072
01:01:41.846 --> 01:01:45.529
Dota 2, OpenAI defeats Dota 2 world champions.

1073
01:01:45.569 --> 01:01:46.710
So that's not that big of a deal.

1074
01:01:46.830 --> 01:01:47.230
Correct.

1075
01:01:47.710 --> 01:01:54.275
So the question is, was the software decoder, which is that translation layer, was that actually in...

1076
01:01:55.315 --> 01:01:56.516
The learning algorithm?

1077
01:01:56.636 --> 01:01:58.417
Or is the actual neuron doing something?

1078
01:01:58.557 --> 01:02:01.999
Because in the Pong case that we already talked about earlier in the episode.

1079
01:02:02.019 --> 01:02:03.039
Very direct.

1080
01:02:03.119 --> 01:02:06.981
It was direct, and it was maybe incremental, but it was demonstrable learning.

1081
01:02:07.021 --> 01:02:07.801
Yes.

1082
01:02:07.941 --> 01:02:12.444
And we could very visibly say the neurons are doing the computation.

1083
01:02:12.464 --> 01:02:13.484
The neurons are doing the computation.

1084
01:02:13.524 --> 01:02:19.107
The neurons are doing the learning because there's no funny business going on where, like, I've got a reinforcement learning agent.

1085
01:02:19.987 --> 01:02:20.628
Okay, fine.

1086
01:02:20.668 --> 01:02:21.889
So now it's playing a video game.

1087
01:02:21.929 --> 01:02:22.670
What's the big deal?

1088
01:02:22.730 --> 01:02:25.112
It knows all the history of Doom's lore internally.

1089
01:02:25.132 --> 01:02:27.514
So here's how they address that.

1090
01:02:27.534 --> 01:02:29.415
They actually did address it in their...

1091
01:02:30.698 --> 01:02:32.359
frequently asked questions section.

1092
01:02:32.539 --> 01:02:36.360
They said, okay, isn't the decoder doing all of the learning?

1093
01:02:37.141 --> 01:02:41.382
What they tested it was, how they tested it was through an ablation study.

1094
01:02:41.622 --> 01:02:48.345
So they've got a metric called a breakthrough rate, which is the amount of episodes that scores above a negative 600 reward threshold.

1095
01:02:48.725 --> 01:02:51.967
Now negative 600 is pretty low because negative 1000 is the floor.

1096
01:02:52.247 --> 01:02:57.589
I've never played Doom, so maybe somebody can tell me if negative 600 is like a lot or a little.

1097
01:02:57.629 --> 01:02:59.730
But here's the key metric that they said.

1098
01:02:59.750 --> 01:03:00.250
Yeah.

1099
01:03:00.310 --> 01:03:06.575
They said that if I were to use the neurons in the loop, I get 27% breakthrough rate.

1100
01:03:06.635 --> 01:03:09.577
So my score is above that negative 600.

1101
01:03:09.937 --> 01:03:17.583
If it's complete silence, so no input from the neurons, or if I put random noise from the neurons, then I get only a 7% to 8% breakthrough.

1102
01:03:17.783 --> 01:03:19.224
So the neurons are doing something.

1103
01:03:19.284 --> 01:03:19.484
Yes.

1104
01:03:19.824 --> 01:03:20.105
Again...

1105
01:03:20.956 --> 01:03:22.177
There's not a lot of trials.

1106
01:03:22.237 --> 01:03:23.779
They're just spitting out these numbers.

1107
01:03:24.259 --> 01:03:26.661
It's not, you know, I don't see data.

1108
01:03:26.962 --> 01:03:31.886
I'm just like taking their word for it that they wrote in a blog post that it's 27% rather than 8%.

1109
01:03:32.307 --> 01:03:34.008
So from baseline, okay, fine.

1110
01:03:34.269 --> 01:03:36.010
Baseline is 8%, 27%.

1111
01:03:36.391 --> 01:03:37.512
Maybe I'll take your word for it.

1112
01:03:37.892 --> 01:03:44.899
The next problem that they addressed in their frequently asked questions is that there's a, you know, is it the encoder?

1113
01:03:46.216 --> 01:03:53.281
Is it the thing that is encoding the neuron state and then feeding it into the reinforcement learning thing?

1114
01:03:53.321 --> 01:03:55.202
Because that itself is a neural network.

1115
01:03:55.243 --> 01:03:56.423
So is that doing all the learning?

1116
01:03:56.663 --> 01:03:57.584
Well, let's see.

1117
01:03:57.764 --> 01:04:00.126
So if you look at Photo30, this is their response.

1118
01:04:00.446 --> 01:04:02.968
They say that, you know, well...

1119
01:04:04.638 --> 01:04:09.619
The brain cells in the loop, they're effectively saying that assuming the brain cells are static, this is not correct.

1120
01:04:10.199 --> 01:04:17.681
The policy in the cells are dynamical systems, so the biological neurons have an internal state, and that is affecting the reinforcement learning.

1121
01:04:17.961 --> 01:04:24.083
Now, to me, all that's saying is like, well, we actually made it harder for the reinforcement learning agent.

1122
01:04:24.283 --> 01:04:24.523
Yeah.

1123
01:04:25.123 --> 01:04:33.285
You know, it's like, okay, the neurons are kind of like a noisy controller that I'm, like, using to create action in my game.

1124
01:04:33.725 --> 01:04:41.346
And maybe, you know, to me it's like, okay, well, that just means my reinforcement learning agent is, like, okay with a noisy controller.

1125
01:04:41.406 --> 01:04:41.626
Right.

1126
01:04:41.706 --> 01:04:44.147
The last sentence is what is key.

1127
01:04:44.187 --> 01:04:50.168
During testing, encoder weights were frozen and still observed improvements in reward, which means that I...

1128
01:04:50.908 --> 01:04:59.273
I froze the way that I am reading my neurons, and still there was improvement, which means that the neurons were maybe doing some type of learning.

1129
01:05:00.734 --> 01:05:04.136
But there's still the encoder in between, even if it's in a frozen state.

1130
01:05:04.256 --> 01:05:11.240
And so how do you know that it's not attributed to some base capability in the encoder, even if it's frozen?

1131
01:05:11.320 --> 01:05:11.580
Yes.

1132
01:05:11.960 --> 01:05:13.241
So that's where I'm like...

1133
01:05:13.261 --> 01:05:14.762
I mean, that's my naive take.

1134
01:05:14.782 --> 01:05:16.263
That's exactly right, right?

1135
01:05:16.503 --> 01:05:19.445
It's like in the YouTube video, they talk about, oh, it's like...

1136
01:05:20.113 --> 01:05:26.676
You know, if the neurons fire a certain way, the game is going to turn left.

1137
01:05:26.716 --> 01:05:31.039
If the neurons fire a certain way, the character in the game will move forward.

1138
01:05:31.519 --> 01:05:33.860
And if they want to fire, the neurons will fire a certain way.

1139
01:05:33.880 --> 01:05:34.921
Well, not really.

1140
01:05:35.441 --> 01:05:44.185
If the neurons fire one way or the other, it might just be that the reinforcement learning agent is just also interpreting the state of the game and just overriding everything.

1141
01:05:44.506 --> 01:05:46.587
So I don't think it's a very clean test.

1142
01:05:47.582 --> 01:05:49.224
Is what I'll be honest about, right?

1143
01:05:49.324 --> 01:05:54.690
And perhaps they're working on a paper that is going to elucidate some of these holes.

1144
01:05:54.870 --> 01:05:59.475
I certainly hope so because I do want a Petri dish game.

1145
01:05:59.655 --> 01:06:00.296
Right.

1146
01:06:00.336 --> 01:06:00.576
Sorry.

1147
01:06:00.737 --> 01:06:02.298
I do want a Petri dish game.

1148
01:06:03.079 --> 01:06:22.401
brain to play doom i think that would be hella cool yes but i'm i'm i think i think the jury is still out because one all of this is anecdotal yes so there's no hard data they're not showing actual you know box plots about the spread and so on and so forth there was another thing that i read just briefly that was saying that the the the seed

1149
01:06:23.771 --> 01:06:27.454
in my neural network actually affects how the gameplay is done, right?

1150
01:06:27.474 --> 01:06:30.235
The random seed that initializes weights and things like that.

1151
01:06:30.356 --> 01:06:35.519
It's like, well, if it's that fragile, can you really say something about whether the neurons are doing anything?

1152
01:06:35.839 --> 01:06:37.020
So I don't know, you know?

1153
01:06:37.340 --> 01:06:38.421
But that's my skepticism.

1154
01:06:38.601 --> 01:06:52.450
And this is not to take away from the foundational discovery that DishBrain, in the Pong context, which is why we took the time to talk about how we got to the Doom version, which is like there is a there there.

1155
01:06:52.490 --> 01:06:52.731
Yes.

1156
01:06:52.751 --> 01:06:53.091
Yes.

1157
01:06:53.435 --> 01:07:08.251
But the way in which they're trying to scale it up for the increased complexity of the tasks they're trying to get the wetware to do, it is not clear that it is purely from improvements at the artificial...

1158
01:07:08.792 --> 01:07:13.633
human pluripotent stem cell level that's doing the learning.

1159
01:07:13.993 --> 01:07:14.174
Exactly.

1160
01:07:14.194 --> 01:07:17.074
It could just be the infrastructure that has now been built around it.

1161
01:07:17.174 --> 01:07:17.654
Exactly.

1162
01:07:17.714 --> 01:07:20.035
That makes it kind of like an interesting thing.

1163
01:07:20.175 --> 01:07:20.555
Exactly.

1164
01:07:20.575 --> 01:07:24.296
But not brains on a chip are doing all of the heavy lifting.

1165
01:07:24.436 --> 01:07:24.836
Exactly.

1166
01:07:24.876 --> 01:07:26.237
And that's what all of the...

1167
01:07:27.533 --> 01:07:29.514
That's what all of the news articles are saying, right?

1168
01:07:29.594 --> 01:07:33.877
All of the news articles are like, brain in a Petri dish, play Doom.

1169
01:07:34.497 --> 01:07:38.039
I don't think anyone's mentioned this reinforcement learning paradigm in the middle.

1170
01:07:38.239 --> 01:07:38.399
Right.

1171
01:07:38.619 --> 01:07:40.640
That is kind of the whole game, maybe.

1172
01:07:40.761 --> 01:07:40.961
Right.

1173
01:07:41.161 --> 01:07:46.984
And because there's not a paper, again, like the Pong version, which they've previously done, the jury is going to still be out on this.

1174
01:07:47.144 --> 01:07:52.307
And that's why it's really important to have peer-reviewed journals, because a reviewer would be like, what?

1175
01:07:52.647 --> 01:07:53.827
You know, what are you doing?

1176
01:07:53.847 --> 01:07:56.248
We're doing that here on the pod, let alone in the professional politics.

1177
01:07:56.268 --> 01:07:56.928
Yeah, exactly.

1178
01:07:56.968 --> 01:07:57.728
Yeah.

1179
01:07:57.928 --> 01:08:09.892
One last thing I want to touch on is there was a lot of heat that the authors took for their 2022 paper because the 2022 paper in their title mentioned sentience.

1180
01:08:10.532 --> 01:08:12.192
Learn and exhibit sentience.

1181
01:08:12.612 --> 01:08:14.033
There was another paper that came out.

1182
01:08:14.593 --> 01:08:14.733
Yeah.

1183
01:08:15.733 --> 01:08:17.195
That was a backlash.

1184
01:08:17.615 --> 01:08:17.875
Okay.

1185
01:08:18.015 --> 01:08:19.977
Again, published in Neuron as a rebuttal.

1186
01:08:20.538 --> 01:08:23.741
And they were effectively saying that you can't use sentience, right?

1187
01:08:23.961 --> 01:08:26.323
Or the idea of perceiving.

1188
01:08:26.984 --> 01:08:35.692
Because the behavior that you're mentioning and you're observing here when the Petri dish plays Pong is one of homeostatic feedback stabilization.

1189
01:08:35.932 --> 01:08:37.974
There's no high-level cognitive intent.

1190
01:08:38.054 --> 01:08:38.495
Yeah, yeah, yeah.

1191
01:08:39.676 --> 01:08:44.965
And so you can't really say things like that, because then with sentience comes ethical concerns.

1192
01:08:45.646 --> 01:08:48.471
In the public lexicon, the term sentience means...

1193
01:08:49.522 --> 01:08:54.585
There's some morality now that we have established onto this dish.

1194
01:08:55.105 --> 01:08:58.047
And if that's the case, well, what are we actually doing?

1195
01:08:58.107 --> 01:08:58.287
Yes.

1196
01:08:58.887 --> 01:08:59.067
Yeah.

1197
01:08:59.288 --> 01:09:17.118
And I think we kind of, over the course of this episode, we've kind of landed on that same point, which is, oh, this is an interesting fundamental understanding of how you can control neural networks through electrical, like electrical read and write architecture.

1198
01:09:17.138 --> 01:09:17.258
Yeah.

1199
01:09:17.878 --> 01:09:21.119
to optimize for stability in the system.

1200
01:09:21.900 --> 01:09:31.323
That's cool, but that's effectively what it's... And then we're just connecting that to an external thing we are interpreting and giving value to as like, oh, hit it with the paddle.

1201
01:09:31.423 --> 01:09:31.664
Yeah, yeah.

1202
01:09:31.804 --> 01:09:32.224
But it...

1203
01:09:33.064 --> 01:09:55.522
fundamentally yeah fundamentally it's just trying to reduce surprise but surprise itself is a mathematical construct that is trying to reduce right just because of the way that the proteins and the right and the neurons work when they when they get together right so it's it's a far leap to say this dish brain is sentient because it can sense and respond to an environment like pong right right

1204
01:09:55.962 --> 01:09:58.125
But this, again, I don't know.

1205
01:09:58.145 --> 01:10:03.793
This is, again, this is, we're going beyond my pay grade, and I do want to be very clear about that, right?

1206
01:10:03.853 --> 01:10:08.860
Because questions of consciousness, questions of sentience is something that I grapple with all the time.

1207
01:10:08.881 --> 01:10:09.221
I...

1208
01:10:10.106 --> 01:10:13.509
have a PhD in like neural systems, right?

1209
01:10:13.549 --> 01:10:17.853
Like one of my PhDs was about how neural networks talk to one another and sleep and so on and so forth.

1210
01:10:18.274 --> 01:10:21.417
You can create very simple neural systems that sleep and mimic behavior.

1211
01:10:22.238 --> 01:10:27.683
I don't know what consciousness is, but for them to put that in the title is, is a bit weird.

1212
01:10:27.703 --> 01:10:27.843
Yeah.

1213
01:10:28.263 --> 01:10:30.744
And I think that's what the rebuttal was about.

1214
01:10:31.604 --> 01:10:33.385
Which I think is fair pushback.

1215
01:10:33.785 --> 01:10:37.386
Not to minimize the work being done and just to be clear.

1216
01:10:37.566 --> 01:10:39.746
This is not trying to be like, oh, you didn't do anything.

1217
01:10:40.006 --> 01:10:41.627
Sometimes the reviewers... No, because it's pretty cool.

1218
01:10:41.687 --> 01:10:43.287
Sometimes the reviewers are like, yeah, you didn't do anything.

1219
01:10:43.367 --> 01:10:43.707
Oh, yeah.

1220
01:10:43.767 --> 01:10:44.828
That's not what we're saying.

1221
01:10:44.948 --> 01:10:49.849
It's just that there's a nuance about the really cool thing that is being done.

1222
01:10:50.349 --> 01:10:54.390
And then the CNN news headline that says, brain on a chip is playing Doom.

1223
01:10:54.610 --> 01:10:54.850
Yeah.

1224
01:10:55.330 --> 01:10:56.811
Which is not... Yeah.

1225
01:10:56.831 --> 01:10:56.931
Yeah.

1226
01:10:57.491 --> 01:10:59.032
quite accurate.

1227
01:10:59.052 --> 01:10:59.673
Yeah, exactly.

1228
01:10:59.713 --> 01:11:04.136
And so, you know, and on this podcast, we're in the business of like just talking about what was done.

1229
01:11:04.357 --> 01:11:04.577
Yes.

1230
01:11:04.617 --> 01:11:08.820
What were the methods and what were the actual findings and the results?

1231
01:11:09.260 --> 01:11:11.042
And so, you know, it is what it is.

1232
01:11:11.162 --> 01:11:12.863
They use the reinforcement learning paradigm.

1233
01:11:13.123 --> 01:11:14.585
I don't know if that changes things.

1234
01:11:15.245 --> 01:11:25.033
It seems to me that they have a lot more work to do to convince someone like me that it's the neurons that are playing doom and not some small artificial neural network.

1235
01:11:25.393 --> 01:11:49.562
right there's not a lot of details so i'm looking forward to hopefully a peer-reviewed scientific paper you know cortical labs if you want us to come over to melbourne i think it's melbourne yeah it's melbourne uh come in and do some coverage as you prepare for release of a research paper we are more than happy to join you oh yeah and have the opportunity to learn more uh if you we will send this episode to you to get your feedback as well or corrections so

1236
01:11:50.042 --> 01:11:53.485
Please let us know if there's anything that we can do to clarify here.

1237
01:11:53.846 --> 01:11:54.767
We are fit.

1238
01:11:54.787 --> 01:11:55.988
Like this is super interesting.

1239
01:11:56.048 --> 01:11:56.248
Yeah.

1240
01:11:56.268 --> 01:11:57.849
We are just curious to learn more.

1241
01:11:58.270 --> 01:12:00.592
And so I'm sure they are already well ahead of us.

1242
01:12:00.812 --> 01:12:01.333
Yeah.

1243
01:12:01.493 --> 01:12:02.053
On that point.

1244
01:12:02.073 --> 01:12:02.914
So we'll see what happens.

1245
01:12:02.934 --> 01:12:04.295
We will cover if there's a follow up.

1246
01:12:04.596 --> 01:12:05.556
We will cover it again.

1247
01:12:06.197 --> 01:12:10.919
This was a phenomenal episode around some of these details.

1248
01:12:11.339 --> 01:12:12.400
We could talk about this forever.

1249
01:12:12.700 --> 01:12:18.302
However, we said we were going to keep our episodes shorter, so I'm going to quickly wrap us up here.

1250
01:12:19.242 --> 01:12:20.983
After that, let us know in the comments.

1251
01:12:21.023 --> 01:12:24.985
We have a comment for the community today, for folks to comment on.

1252
01:12:26.766 --> 01:12:28.007
What game do you want to see?

1253
01:12:28.447 --> 01:12:29.148
Oh, that's a good one.

1254
01:12:29.168 --> 01:12:32.510
What game would you like to see the brain on a chip play next?

1255
01:12:32.630 --> 01:12:33.530
Doom was number one.

1256
01:12:33.930 --> 01:12:34.691
What's number two?

1257
01:12:34.811 --> 01:12:35.691
I know what my thought is.

1258
01:12:35.751 --> 01:12:36.952
Maybe we'll share it next episode.

1259
01:12:37.012 --> 01:12:37.232
Yeah.

1260
01:12:37.673 --> 01:12:43.756
I'm your host, Lester Nari, joined as always by my co-host and our resident PhD, Krishna Chowdhury.

1261
01:12:44.076 --> 01:12:45.557
Again, multiple episodes a week.

1262
01:12:45.597 --> 01:12:46.638
We'll see you later this week.
