WEBVTT

1
00:00:00.120 --> 00:00:01.366
Good morning, hackers.

2
00:00:01.426 --> 00:00:04.920
You are listening to The Lowdown, the best podcast about hacker stuff

3
00:00:05.020 --> 00:00:06.505
presented by Maze.

4
00:00:06.545 --> 00:00:08.832
Here with Matt J. and I am Low Level.

5
00:00:09.112 --> 00:00:11.600
We are in the PlanetScale office here at

6
00:00:11.700 --> 00:00:17.179
the AI Engineering Conference in San Francisco, having a grand old time talking about AI,

7
00:00:17.860 --> 00:00:20.774
AI security, and a variety of other topics.

8
00:00:20.835 --> 00:00:21.960
So Matt, what are we doing today, man?

9
00:00:22.300 --> 00:00:22.521
Why?

10
00:00:22.962 --> 00:00:24.326
Who put us on a plane?

11
00:00:24.346 --> 00:00:28.660
And who let two hackers physically break into a database company?

12
00:00:28.680 --> 00:00:33.080
the office are they aware what they've done are they aware of what uh what heist we are pulling

13
00:00:33.180 --> 00:00:38.279
off here they gave us the wi-fi too that's right right i'm actually on the database company's wi-fi

14
00:00:38.560 --> 00:00:44.560
insane absolutely insane uh honestly i kind of forgot how cool silicon valley and san francisco

15
00:00:44.700 --> 00:00:50.820
specifically startup offices are i know you and i work uh from home slash little studios

16
00:00:51.021 --> 00:00:57.220
i used to live out here work out here i i forgot i forgot they've got that vc money they've got

17
00:00:57.280 --> 00:00:57.863
the swag.

18
00:00:57.883 --> 00:00:59.009
They've got the ping pong table.

19
00:00:59.029 --> 00:01:00.255
They got the podcast studio.

20
00:01:00.275 --> 00:01:01.180
They let us invade.

21
00:01:01.841 --> 00:01:02.485
It's been pretty sweet.

22
00:01:02.505 --> 00:01:04.900
They got a wall of mechanical keyboards that each of us kind of

23
00:01:04.940 --> 00:01:05.864
nabbed out of one.

24
00:01:05.884 --> 00:01:09.879
Uh, it's pretty cool for planet scale to have us out here to talk AI

25
00:01:10.161 --> 00:01:12.893
security stuff during the AI engineering world's fair.

26
00:01:13.054 --> 00:01:14.300
I guess this is the second year of this

27
00:01:14.340 --> 00:01:16.267
conference, uh, over at Moscone center.

28
00:01:16.569 --> 00:01:18.757
Those of you, uh, that come to RSA every year, uh,

29
00:01:19.620 --> 00:01:22.309
Moscone center doesn't just exist during RSA.

30
00:01:22.329 --> 00:01:25.860
Uh, it doesn't just like materialize and show up

31
00:01:25.920 --> 00:01:26.991
just when 60,000 of us show up.

32
00:01:28.141 --> 00:01:29.500
There's about 7,000 people at this one.

33
00:01:29.600 --> 00:01:32.120
It's Moscone West, this secret third Moscone

34
00:01:32.750 --> 00:01:33.320
off to the side.

35
00:01:33.440 --> 00:01:34.660
It's a little smaller and quainter,

36
00:01:34.800 --> 00:01:36.140
but it's honestly, the vibes are good.

37
00:01:36.380 --> 00:01:37.000
The vibes are high.

38
00:01:38.001 --> 00:01:39.199
Everyone's here talking about the same thing.

39
00:01:39.700 --> 00:01:39.780
Yeah.

40
00:01:40.400 --> 00:01:43.820
Yeah, the world of AI is in a weird spot right now.

41
00:01:44.303 --> 00:01:46.780
And I really acknowledge for the listener

42
00:01:46.800 --> 00:01:48.540
that we're going to talk about AI a lot.

43
00:01:48.862 --> 00:01:50.993
But specifically as a security person,

44
00:01:52.541 --> 00:01:55.160
it's becoming a topic that is so hard to ignore.

45
00:01:55.300 --> 00:02:00.760
Now, if you've watched my YouTube channel, you will know that like a year ago, I did not like AI.

46
00:02:00.880 --> 00:02:05.020
I was very anti-AI because the code that it writes was garbage and it couldn't find bugs.

47
00:02:05.120 --> 00:02:09.300
So like what use does it have for me as a security researcher or, you know, a developer?

48
00:02:09.520 --> 00:02:09.721
Right.

49
00:02:10.483 --> 00:02:17.220
But I think with with the production of like Sonnet 4.6, there's kind of like this this break where like suddenly it was very good.

50
00:02:17.300 --> 00:02:20.400
And I'm not only producing code, but finding vulnerabilities in code.

51
00:02:20.440 --> 00:02:24.660
And today, like the models are a faster reverse engineer than I am.

52
00:02:24.821 --> 00:02:28.680
So I'm trying to figure out like where I fit in the world, like with that truth.

53
00:02:29.141 --> 00:02:36.780
It's a roasting a panel that I actually at PlanetScale to talk with the panel about like, what does the world of security look like with AI as a tool?

54
00:02:36.860 --> 00:02:37.000
Right.

55
00:02:37.020 --> 00:02:44.920
What can you do as not only an engineer or security researcher, but like a developer, right, to make your your your infrastructure, your code base more and more safe?

56
00:02:45.120 --> 00:02:47.980
So, yeah, it'll it'll be could be a good panel.

57
00:02:48.200 --> 00:02:51.280
But interesting stuff, man, like like Fable recently got banned.

58
00:02:51.360 --> 00:02:52.180
What's going on with that?

59
00:02:52.301 --> 00:02:55.240
I mean, that's a whole can of worms that I'm personally very scared about.

60
00:02:55.300 --> 00:02:56.499
I'm curious to hear what you have to say.

61
00:02:57.080 --> 00:03:02.280
Yeah, I don't know if you realize you're sitting in the presence of a signatory of the free fable open letter.

62
00:03:03.062 --> 00:03:04.940
It's basically the Declaration of Independence.

63
00:03:05.100 --> 00:03:06.180
That's pretty rad.

64
00:03:06.381 --> 00:03:07.760
I put my John Hancock on it.

65
00:03:07.820 --> 00:03:19.440
So, yeah, obviously, the marketing has worked to some degree of that mythos and fable are going to escape the lab and they're going to be the super hacker.

66
00:03:19.680 --> 00:03:19.881
Right.

67
00:03:19.901 --> 00:03:29.200
And you see this crap coming out in the headlines coming from some senator talked to somebody at the NSA that said that this thing could hack anything.

68
00:03:29.280 --> 00:03:31.871
It hacked all the NSA's confidential systems within minutes.

69
00:03:31.992 --> 00:03:33.960
And it's way too dangerous to be out there.

70
00:03:34.381 --> 00:03:39.940
And it's just I mean, anyone who knows what they're talking about knows that that's like not actually what's going on with any of these tools.

71
00:03:40.141 --> 00:03:45.140
Like, yes, as you just acknowledged, they're very good at code and they're very good at finding vulnerabilities.

72
00:03:46.023 --> 00:03:51.660
and the the thing that sets mythos and fable apart from the opus four sixes that turning point that

73
00:03:51.680 --> 00:03:55.440
you're talking about is not just being able to find the vulnerabilities but actually write

74
00:03:55.721 --> 00:04:01.580
exploits for them which is the whole mythos thing and why we're talking about it so much and they've

75
00:04:01.640 --> 00:04:06.160
made this project glasswing and invite certain people to the walled garden to like get the super

76
00:04:06.341 --> 00:04:11.240
secret exploit creator thing and then fable was the public version of that with like laughable

77
00:04:11.341 --> 00:04:14.860
guardrails like we talked about it on our first episode i think right where like you couldn't even

78
00:04:14.880 --> 00:04:19.019
say the word cyber to the damn thing without it being like, you're going back to Opus.

79
00:04:19.079 --> 00:04:19.320
Like you

80
00:04:19.380 --> 00:04:22.211
can't, no, you can't say the word cyber in Fable land right now.

81
00:04:22.853 --> 00:04:24.459
And so, uh, yeah, I mean, of

82
00:04:24.840 --> 00:04:30.600
course, if you market it as this day, dangerous, basically nuclear weapon, the government's going

83
00:04:30.600 --> 00:04:32.568
to, you know, get a little spicy and get involved.

84
00:04:32.588 --> 00:04:35.720
And so they've, they've, uh, invoked an export

85
00:04:35.861 --> 00:04:37.269
control is actually what they did.

86
00:04:37.309 --> 00:04:37.450
Right.

87
00:04:37.470 --> 00:04:38.174
So they didn't ban it.

88
00:04:38.194 --> 00:04:39.260
They said, they didn't say that you

89
00:04:39.320 --> 00:04:40.585
and I couldn't do it.

90
00:04:40.886 --> 00:04:44.940
They said that people not from the U.S. couldn't have access to it, which

91
00:04:45.341 --> 00:04:49.899
effectively, Anthropic has no possible way to enforce, like, upon the public, right?

92
00:04:49.919 --> 00:04:50.080
There's

93
00:04:50.100 --> 00:04:50.842
just no way.

94
00:04:51.324 --> 00:04:56.900
A lot of their employees currently cannot use their own tool under the current

95
00:04:57.543 --> 00:04:59.089
regulation that came down.

96
00:04:59.430 --> 00:05:01.860
So this is like, you know, we're getting kind of sick of living in

97
00:05:02.061 --> 00:05:06.597
unprecedented times, but it's another unprecedented thing, right?

98
00:05:07.079 --> 00:05:07.440
And so, yeah,

99
00:05:07.480 --> 00:05:09.908
there's this free fable movement going on.

100
00:05:09.928 --> 00:05:13.440
Uh, Alex Stamos, ex CISO of, uh, Facebook and Yahoo

101
00:05:13.801 --> 00:05:19.200
and, uh, you know, just overall cyber, uh, thought leader, I guess he'll probably hate that I called

102
00:05:19.220 --> 00:05:21.911
him that, but, um, has been championing this.

103
00:05:21.991 --> 00:05:24.320
And, and a lot of the people that sign it, uh,

104
00:05:24.501 --> 00:05:30.019
the letter even says, Hey, it's not that everyone on this list agrees or disagrees about AI

105
00:05:30.340 --> 00:05:31.063
regulation.

106
00:05:31.283 --> 00:05:36.700
We all just think that the way that this all went down is kind of crazy because

107
00:05:37.201 --> 00:05:42.259
Fable and Mythos aren't unique in their ability to find and even hack vulnerabilities.

108
00:05:42.560 --> 00:05:42.700
They've

109
00:05:42.720 --> 00:05:45.513
just gotten caught up in the marketing and hype of it.

110
00:05:45.774 --> 00:05:46.598
I mean, it's still going on.

111
00:05:46.638 --> 00:05:47.060
I just read

112
00:05:47.140 --> 00:05:53.040
yesterday that another, I think it was another senator closed our session said that they saw

113
00:05:53.521 --> 00:05:55.811
Mythos drain bank accounts in front of them.

114
00:05:55.851 --> 00:05:58.160
And it's just like, no, that's not how security tools

115
00:05:58.260 --> 00:05:58.742
work, right?

116
00:05:58.763 --> 00:06:01.313
That's not like maybe it found a vulnerability.

117
00:06:01.574 --> 00:06:01.835
Sure.

118
00:06:02.016 --> 00:06:02.920
But it's not like, oh,

119
00:06:03.020 --> 00:06:07.900
this tool went out and drained the bank account or that's not like yeah yeah the secret hacking

120
00:06:07.980 --> 00:06:12.240
tool escaped the lab again what's really confusing there okay there are a bunch of confusing points

121
00:06:12.701 --> 00:06:18.818
in this whole this whole scenario the first part is like people don't realize i think that any good

122
00:06:19.680 --> 00:06:25.920
researcher that knows how exploitation works can take the primitives that another model like opus

123
00:06:26.281 --> 00:06:31.520
47 or whatever finds and turn them into an exploit that is the thing that mythos is uniquely good at

124
00:06:31.600 --> 00:06:35.400
is chaining primitives together but once you have the primitives when i say primitive i mean like

125
00:06:35.500 --> 00:06:40.100
arbitrary read arbitrary right like use after free whatever um you can turn those into a meaningful

126
00:06:40.241 --> 00:06:43.660
exploit like that is that is not a hard problem obviously the ability to do that automatically

127
00:06:43.780 --> 00:06:49.160
and chain them is scary but like you can find bugs right now with opus 48 force for whatever

128
00:06:49.280 --> 00:06:53.360
whatever whatever one is active and free you can plug it into gager mcp and find bugs and close

129
00:06:53.541 --> 00:07:00.500
software or go use it on open source software or or like sorry but shocking you could find an

130
00:07:00.560 --> 00:07:05.920
exploit these things without ai right exactly but regarding ai though what makes me nervous is

131
00:07:06.342 --> 00:07:09.960
we're banning mythos because something something something or banning fable something something

132
00:07:10.101 --> 00:07:13.900
because it's really dangerous at finding vulnerabilities right i'm concerned that

133
00:07:14.101 --> 00:07:19.900
that opinion is being held in the ineptitude of government and once somebody who's smart in

134
00:07:19.980 --> 00:07:25.580
government and trust me they do exist there's not a lot of them um says hey guys we can find bugs

135
00:07:25.620 --> 00:07:29.940
without mythos and fable you know that right it is the solution to that problem then that

136
00:07:30.281 --> 00:07:34.660
we just get all the models banned like is it there and then illegal for someone not within

137
00:07:34.740 --> 00:07:39.600
the glass being bubble the government bubble to have access to a model capable of exploitation

138
00:07:39.781 --> 00:07:45.280
right because again the the models were never designed to find bugs they were designed to

139
00:07:45.440 --> 00:07:50.340
write code so that the models could be profitable and then by being able to reason about a code base

140
00:07:50.460 --> 00:07:55.560
well oops it also is very good about reasoning about vulnerabilities and code bases yeah and

141
00:07:55.740 --> 00:08:07.420
And it's actually a really good point because one of the ways that some of the government officials are suggesting the next steps are is could you make the model less good and knowledgeable about security vulnerabilities?

142
00:08:08.102 --> 00:08:10.720
We want it to be we want you to be able to use it to code.

143
00:08:11.143 --> 00:08:12.760
We just don't want you to be able to use it to hack.

144
00:08:12.860 --> 00:08:12.995
Right.

145
00:08:13.040 --> 00:08:13.760
Because that's scary.

146
00:08:13.861 --> 00:08:14.539
We don't know about it.

147
00:08:15.142 --> 00:08:20.600
And it's like, if you follow that train of thought, it's like, then the code would be filled with security vulnerabilities.

148
00:08:20.780 --> 00:08:28.700
If you could, could even possibly manage to suck the security knowledge out of these models, which is impossible, right?

149
00:08:28.841 --> 00:08:32.459
Of like, oh, you know, use after free or memory safety or whatever it is.

150
00:08:32.901 --> 00:08:40.380
So we'll know, like, of course it needs to know how to use memory safely because good developers know how to use memory safety safely.

151
00:08:40.521 --> 00:08:44.640
right so i actually think it would probably be worth it if uh we could take a second and actually

152
00:08:44.761 --> 00:08:49.060
read some of the points on this free fable letter uh basically verbatim right because i think it's i

153
00:08:49.100 --> 00:08:53.040
think it's worded really well we could show it on the screen uh on the cut but i'll read it for the

154
00:08:53.060 --> 00:08:58.300
audio version right so there's a list of things on this letter it's a very short letter that um

155
00:08:59.122 --> 00:09:04.460
i've added my name to um and i'm probably bullying low level to do the same but um it says first we

156
00:09:04.480 --> 00:09:08.660
would like to state some things we believe ai is having significant impacts on cyber security

157
00:09:09.060 --> 00:09:13.604
Like, if you're not in that camp anymore, sorry, that ship has sailed, right?

158
00:09:13.985 --> 00:09:21.051
AI is having significant impacts on cybersecurity, including by greatly reducing the difficulty of finding flaws in software and writing exploits for those flaws.

159
00:09:21.811 --> 00:09:25.575
Mythos class models are quite good at finding flaws and weaponizing exploits.

160
00:09:26.115 --> 00:09:29.158
However, they are not uniquely good at these tasks.

161
00:09:29.198 --> 00:09:31.640
This is, I think, the point that you and I are both making right now, right?

162
00:09:32.242 --> 00:09:37.940
And many of the undersigned individuals regularly use other foundation and open source models.

163
00:09:38.681 --> 00:09:43.400
We'll get back to the open source models in a second for security audits and red teaming every single day.

164
00:09:44.120 --> 00:09:50.180
Anthropic built multiple protections of the fable to the point of it was humorous to the cyber community because you literally said the word cyber and it kicked you out.

165
00:09:50.280 --> 00:09:54.400
And this was still good enough for them to pull this off the shelves.

166
00:09:55.022 --> 00:10:00.420
it's essential to provide ai to coders and security teams so they can find and fix flaws

167
00:10:00.500 --> 00:10:04.820
in their own newly written as well as decades of legacy code i think this is the point i want to

168
00:10:04.840 --> 00:10:10.140
pause on for a second this is like my most strongly held belief on this topic we've had

169
00:10:10.220 --> 00:10:15.780
this conversation before metasploit is a tool that's open source and filled with exploits

170
00:10:17.625 --> 00:10:22.700
metasploit is this open source tool it's filled with exploits it makes exploitation easier right

171
00:10:23.103 --> 00:10:25.460
It's like lowering the skill bar of exploitation.

172
00:10:25.840 --> 00:10:28.080
You don't need to know how to write a buffer overflow.

173
00:10:28.842 --> 00:10:33.440
You need to know how to point Metasploit at a thing and say, find CVE, hack CVE.

174
00:10:34.081 --> 00:10:40.580
A generation of pen testers have made a career off of just wielding a vulnerability scanner and map and Metasploit.

175
00:10:41.481 --> 00:10:47.720
And we've had this debate before of, are we lowering the skill bar for exploitation too much?

176
00:10:48.021 --> 00:10:51.880
And the answer is no, because otherwise you're just pretending like it's not possible.

177
00:10:52.825 --> 00:10:55.900
this is the same argument by the way for the listener if maybe you're not even a cyber security

178
00:10:56.402 --> 00:11:02.160
practitioner here if you've seen the movie harry potter this is literally the defense against the

179
00:11:02.200 --> 00:11:07.360
dark arts argument right the ministry of magic decided that teaching how to cast evil spells

180
00:11:07.460 --> 00:11:11.700
was bad because you shouldn't do that and then the students didn't know how to defend against it and

181
00:11:11.760 --> 00:11:15.300
harry created this whole band of people that were learning it in secret but it's the same exact

182
00:11:15.400 --> 00:11:19.980
argument right like in order to know how to defend against these things how to have a meaningful

183
00:11:20.181 --> 00:11:22.591
security posture, you have to know how hacking works.

184
00:11:22.631 --> 00:11:24.880
You have to be able to find and exploit

185
00:11:24.900 --> 00:11:27.730
the vulnerabilities yourself so you can know how to then defend against them.

186
00:11:28.252 --> 00:11:29.356
So the thought that

187
00:11:30.861 --> 00:11:36.400
you're able to do coding and security as separate tasks and you can just plop one out of the model

188
00:11:36.500 --> 00:11:44.380
is such a uniquely government senator congressman take that it's almost hilarious to see that

189
00:11:44.480 --> 00:11:45.023
written down.

190
00:11:45.143 --> 00:11:48.920
You mean this is coming from the people who said, excuse me, TikTok CEO, can you

191
00:11:49.060 --> 00:11:50.585
access my Wi-Fi, right?

192
00:11:50.665 --> 00:11:53.614
Mr. Chute, does TikTok access the home Wi-Fi network?

193
00:11:54.758 --> 00:11:55.460
Like this is the

194
00:11:55.480 --> 00:11:56.866
kind of people writing this stuff, right?

195
00:11:57.710 --> 00:11:59.739
But yeah, like we've settled this debate.

196
00:11:59.839 --> 00:12:00.040
Giving

197
00:12:00.140 --> 00:12:03.334
defenders the best tools is the best course of action, period.

198
00:12:03.455 --> 00:12:04.359
End of discussion, right?

199
00:12:04.379 --> 00:12:04.600
Otherwise,

200
00:12:04.780 --> 00:12:08.735
you're just ostrich in the sand pretending like this capability doesn't exist.

201
00:12:09.156 --> 00:12:09.638
And it does.

202
00:12:09.658 --> 00:12:10.120
And

203
00:12:10.160 --> 00:12:15.940
the next bullet on this list is the like one-two punch to that because the Chinese open weight

204
00:12:16.020 --> 00:12:21.120
models are only months behind the best American models, not to pull the China boogeyman right

205
00:12:21.221 --> 00:12:23.293
card that a lot of people like to do in cybersecurity.

206
00:12:23.635 --> 00:12:24.359
But it's true there.

207
00:12:24.379 --> 00:12:24.560
They're

208
00:12:24.640 --> 00:12:25.002
out there.

209
00:12:25.605 --> 00:12:29.060
And not that the boogeyman is true, but but this part point is true that they have

210
00:12:29.100 --> 00:12:35.320
these open weight models, the deep seeks, the Kimmy K2s, the GLM 5.2s that are like on the

211
00:12:35.440 --> 00:12:38.589
heels of the frontier models capabilities in this space.

212
00:12:39.994 --> 00:12:42.160
Alex, who wrote this, gave a talk recently,

213
00:12:42.200 --> 00:12:58.500
And he said that they're at most, at most 12 months behind the Frontier models in terms of, and when we say open weight models, we mean models that you can literally download and run on your own hardware and just circumvent any like regulation or guardrail or whatever it is that might be in place.

214
00:12:58.902 --> 00:13:03.480
So and these are whatever models that the Chinese government is even telling us about.

215
00:13:03.600 --> 00:13:03.737
Right.

216
00:13:03.860 --> 00:13:08.880
So it seems likely that the PRC government has access to private capabilities beyond

217
00:13:08.962 --> 00:13:09.540
what it has published.

218
00:13:09.640 --> 00:13:12.900
And I think that's just very, very safe to assume.

219
00:13:13.000 --> 00:13:13.160
Right.

220
00:13:13.380 --> 00:13:13.492
Yeah.

221
00:13:13.660 --> 00:13:17.740
And to put the pull the best capabilities away from defenders without a good reason

222
00:13:17.840 --> 00:13:19.993
when our adversaries are rapidly advancing is dangerous.

223
00:13:21.740 --> 00:13:21.894
Right.

224
00:13:22.723 --> 00:13:24.440
So I just love this list.

225
00:13:24.642 --> 00:13:26.240
I think it was worth reading verbatim.

226
00:13:26.501 --> 00:13:33.479
I think the other point that I started to make, right, was not everyone on this list thinks that like, hey, hands off government, don't touch my AI.

227
00:13:34.040 --> 00:13:34.201
Right.

228
00:13:34.221 --> 00:13:39.220
I think some people probably think that, no, this is a scary paced technology.

229
00:13:40.260 --> 00:13:40.482
Right.

230
00:13:40.563 --> 00:13:42.620
The pace of change is very rapid.

231
00:13:43.461 --> 00:13:48.280
And the capabilities that it is launching are sensitive.

232
00:13:49.303 --> 00:13:54.180
but there's four things on this letter that they say hey if you're going to regulate models then

233
00:13:54.200 --> 00:14:00.560
the regulation should be one grounded in scientific evaluations developed with input from the industry

234
00:14:00.821 --> 00:14:07.240
and academia to created through a democratic rulemaking process three enforced transparently

235
00:14:07.501 --> 00:14:12.240
and fairly with appropriate time given to remediate it right that did not happen at all

236
00:14:12.341 --> 00:14:15.680
this was just a letter that popped out anthropic had to yank this stuff off the shelf because there

237
00:14:15.680 --> 00:14:18.260
There's no way to limit this whole export control thing.

238
00:14:18.682 --> 00:14:22.700
And then four, used only to the minimal extent necessary to ensure the safety of American

239
00:14:22.808 --> 00:14:22.980
public.

240
00:14:23.060 --> 00:14:26.440
And I think these are all pretty reasonable asks out of this.

241
00:14:27.506 --> 00:14:27.873
Yeah, 100%.

242
00:14:28.340 --> 00:14:32.960
Again, it just goes down a scary path that if the government, not really knowing much

243
00:14:33.020 --> 00:14:37.840
about anything, decides to pull one model, what says they're not going to pull the rest

244
00:14:37.860 --> 00:14:38.280
of the models?

245
00:14:38.460 --> 00:14:40.400
Oh, and also this model is Chinese and open source.

246
00:14:40.500 --> 00:14:41.320
That's also dangerous.

247
00:14:41.820 --> 00:14:45.559
And we can go into talking about the Dario comments about open source models, about how

248
00:14:45.660 --> 00:14:53.400
Now, Dario, you know, the CEO of Anthropic says, yeah, open source models, because, you know, there's no control over them, just like, you know, open source hacking tools, etc.

249
00:14:54.422 --> 00:14:58.720
are uniquely dangerous and should also be heavily regulated if not banned.

250
00:14:59.160 --> 00:15:05.560
The meme being maker of closed source models says not closed source models are dangerous and should be illegal.

251
00:15:05.621 --> 00:15:07.740
Yeah, entering entering very dangerous territory.

252
00:15:08.023 --> 00:15:08.699
I have the same opinion.

253
00:15:08.800 --> 00:15:17.260
It's like I acknowledge that this is a huge shift in the tectonic plates of like the foundation of security, of cybersecurity, of security research.

254
00:15:18.241 --> 00:15:25.780
But to just remove the average person's access to it is very, very knee jerk and definitely not democratic.

255
00:15:26.102 --> 00:15:26.323
Right.

256
00:15:27.149 --> 00:15:28.980
Yeah, I think the Dario thing is super funny.

257
00:15:29.101 --> 00:15:29.342
Right.

258
00:15:29.844 --> 00:15:33.180
It's like, of course, the head of Anthropic is like, whoa.

259
00:15:33.982 --> 00:15:39.200
And I'm also I'm seeing a lot of even cybersecurity leaders hop on that comment as well.

260
00:15:39.921 --> 00:15:49.060
I'm not sure I fully understand, you know, especially when the open weight models you can run on your own hardware, your own cloud hardware or anything like that.

261
00:15:49.821 --> 00:15:57.660
It's not like you're necessarily just, oh, I'm going to now use a Chinese frontier model on their hardware.

262
00:15:58.081 --> 00:15:58.283
Right.

263
00:15:58.303 --> 00:15:59.539
These are open weight models.

264
00:16:00.303 --> 00:16:05.140
um there's obviously some concerns about they're not exactly free from their government regulations

265
00:16:05.401 --> 00:16:10.220
either over there so there are some things to be concerned about when you're putting those kinds of

266
00:16:10.280 --> 00:16:16.720
models in decision making loops uh for your organization because you do have to take in

267
00:16:17.181 --> 00:16:21.860
to account like the biases and regulations of the chinese government that's putting that stuff out

268
00:16:21.960 --> 00:16:26.300
yeah when uh deep seek first came out it was like a meme to ask it about tiananmen square and like

269
00:16:26.400 --> 00:16:31.740
the deep seek model would uh not comment so you know go any deeper if you want to on that about

270
00:16:31.820 --> 00:16:36.320
like you know making wartime decisions or making some kind of vulnerability research decision on

271
00:16:36.340 --> 00:16:41.320
maybe a chinese platform right it gets um very scary very quick what you know the the amount of

272
00:16:41.460 --> 00:16:46.540
power that a a model developer has over over people in general this applies not only to china

273
00:16:46.620 --> 00:16:51.620
but like to anthropic to open ai anyone who's in charge of a frontier model wields unprecedented

274
00:16:51.821 --> 00:16:58.260
control over literally humanity very scary stuff yeah i think i mean i think it's kind of impossible

275
00:16:58.641 --> 00:17:06.540
to take specifically frontier american models leadership's word on the dangers of open

276
00:17:07.382 --> 00:17:13.200
weight models i mean you know there are some concerns but uh i don't think that a lot of

277
00:17:13.200 --> 00:17:18.780
those concerns are necessarily uh handled by using the frontier models instead right like

278
00:17:18.840 --> 00:17:20.247
oh, you can't see inside this.

279
00:17:20.307 --> 00:17:20.830
Well, I can't.

280
00:17:20.930 --> 00:17:22.900
I don't know what's going on inside Claude,

281
00:17:23.080 --> 00:17:23.321
right?

282
00:17:23.341 --> 00:17:28.580
Like, you know, it's like, oh, a lot of these risks are just risks of using LLMs in

283
00:17:28.660 --> 00:17:32.090
general, not necessarily just the open weight models.

284
00:17:32.271 --> 00:17:32.973
So I don't know.

285
00:17:34.257 --> 00:17:35.100
Ironically, and kind

286
00:17:35.260 --> 00:17:37.267
of the similar funny fashion.

287
00:17:37.287 --> 00:17:41.400
I don't know if you've seen this, but Meta recently restricted

288
00:17:41.561 --> 00:17:42.645
the use internally.

289
00:17:42.665 --> 00:17:46.880
This is came in from a leaked internal memo, restricted the use of quote unquote

290
00:17:47.081 --> 00:17:53.460
competitor model data, referring to the models that are made by OpenAI and Anthropic, because,

291
00:17:53.960 --> 00:17:59.480
quote, there could be legal implications if this data is leaked into our models or potentially

292
00:17:59.741 --> 00:18:01.306
poisons our models.

293
00:18:01.326 --> 00:18:02.389
Yeah.

294
00:18:02.449 --> 00:18:05.940
So the funny part about that is what are the legal implications of all

295
00:18:05.980 --> 00:18:12.160
these companies having stolen quite literally in some cases, literally all of the planet's data

296
00:18:12.561 --> 00:18:18.600
to train their models why why is it only illegal if you get caught as meta doing it but if other

297
00:18:18.680 --> 00:18:23.980
companies do it to make their models like function it's not illegal there's this whole open-ended

298
00:18:24.100 --> 00:18:29.160
question about like is it even legal for anthropic to have the models in the current way that they

299
00:18:29.240 --> 00:18:33.600
argue and there's like literally i think we have receipts of maybe it was open ai like like taking

300
00:18:33.761 --> 00:18:39.300
books that were like pirated online and using them to train their models i think i saw somebody demo

301
00:18:39.461 --> 00:18:44.140
how you could literally get one of the models to recite almost word for word like harry potter and

302
00:18:44.140 --> 00:18:48.371
the sorcerer or something like that second harry potter reference of the podcast yeah so it's uh

303
00:18:52.482 --> 00:18:56.180
very interesting that suddenly we're really worried about the legal implications of models

304
00:18:56.260 --> 00:19:00.520
with if ai gets caught with the bag anybody else is like now you're fine or sorry if if meta does

305
00:19:00.600 --> 00:19:05.060
rather yeah this this brings up some interesting points and thoughts for me you know we're seeing

306
00:19:05.120 --> 00:19:11.760
a lot of internal enterprise restrictions popping up like about uh ai usage in general because it

307
00:19:11.760 --> 00:19:15.060
was like a free-for-all for a while right it's like hey no one you're not allowed to write your

308
00:19:15.080 --> 00:19:21.060
own code anymore like you must be ai super pilled 100x engineers and then all of a sudden everyone

309
00:19:21.060 --> 00:19:25.220
got the bills in the mail at the end of the quarter and they were like oh hey slow down on

310
00:19:25.240 --> 00:19:30.260
that token stuff everybody real quick uh and a lot of these restrictions are coming out um the

311
00:19:30.260 --> 00:19:36.500
The other bit about this is just like it's the meme of the surge projector like plugged into itself, this whole industry, right?

312
00:19:36.580 --> 00:19:47.680
Like I saw, you know, I was talking to some people here that run some Neo clouds, which is even like a new word for me of these, you know, little mini clouds that are purpose built.

313
00:19:47.840 --> 00:19:54.020
And some of these is some of the AI companies are using because GPUs to that scale were hard to get for so long.

314
00:19:54.120 --> 00:19:58.120
So a bunch of people started hoarding them and there's all sorts of companies popping up for it.

315
00:19:58.401 --> 00:20:03.100
And you go and read some of these Neo cloud testimonials and it's like Google and Microsoft

316
00:20:03.222 --> 00:20:04.100
and whatever.

317
00:20:04.180 --> 00:20:05.860
It's like, oh wait, those are cloud companies.

318
00:20:06.081 --> 00:20:11.140
Like they sell compute in mass and they're using other cloud company.

319
00:20:11.604 --> 00:20:13.320
Like I said, search projector plugged into itself.

320
00:20:13.860 --> 00:20:14.027
100%.

321
00:20:14.501 --> 00:20:18.480
So now you get it not just in compute, but in AI model usage.

322
00:20:18.580 --> 00:20:25.020
So like, oh yeah, what is Google and meta and Microsoft and whatever?

323
00:20:25.221 --> 00:20:29.580
like what is their stance internally about using other models they're all making their own models

324
00:20:29.781 --> 00:20:33.820
right amazon made their own model like everyone's making their if you're a company of a certain size

325
00:20:34.362 --> 00:20:39.280
everyone was like we got to make our own model right and now like yeah you look at meta is is

326
00:20:39.501 --> 00:20:43.960
at the point of training their own model that they're actually worried about their employees

327
00:20:44.201 --> 00:20:49.120
using the frontier models not just for like legal exposure but they're also worried about

328
00:20:49.622 --> 00:20:53.700
the quality they're in because they're like wait now we're like training on training output

329
00:20:54.402 --> 00:20:59.480
and it's like starts to become you know the watered down it like incest meme of like oh this

330
00:20:59.540 --> 00:21:03.560
is not actually high quality stuff anymore yeah this is the argument about like the overall

331
00:21:03.861 --> 00:21:10.000
in slopification of the internet right as more and more user generated content comes out of llms

332
00:21:10.401 --> 00:21:15.260
and then those llms are trained on user generated content you literally like like by definition have

333
00:21:15.300 --> 00:21:20.600
a circular process uh the question becomes like what does that boil down to right what how how

334
00:21:20.760 --> 00:21:27.280
low can the quality bar get before we realize, OK, there is a structural issue with the way

335
00:21:27.300 --> 00:21:28.759
that we're producing information?

336
00:21:29.782 --> 00:21:33.780
And yeah, to your point, I think companies, maybe the smaller ones that have less blindfolds

337
00:21:33.880 --> 00:21:36.780
on, less blinders on, are starting to see that, right?

338
00:21:36.880 --> 00:21:40.700
There are some companies that are being like, hey, these tokens are extremely expensive.

339
00:21:41.060 --> 00:21:41.900
Oh, not even small companies.

340
00:21:42.140 --> 00:21:47.160
AWS recently or Amazon pulled down their internal company leaderboard for token maxing because,

341
00:21:47.381 --> 00:21:54.140
oh shocker uh incentivizing your employees to use tokens for the sake of using tokens is a expensive

342
00:21:54.341 --> 00:22:01.000
and b not productive at all and aws again i ran by amazon for a long time with basically zero

343
00:22:01.080 --> 00:22:06.400
outages has had their two or three biggest outages in the last couple of years uh you know correlated

344
00:22:06.681 --> 00:22:13.580
but not maybe causated by ai so yeah have you seen did you see the meme it's like uh the hall

345
00:22:13.640 --> 00:22:17.880
of fame of useless statistics and it was like lines of code written in a day story points

346
00:22:17.960 --> 00:22:22.660
closed in a day now it's token yeah now the lines of tokens used right i mean it's totally yeah

347
00:22:22.660 --> 00:22:26.880
it's totally true right it's you can't be like oh look how many tokens they spent that's how

348
00:22:26.940 --> 00:22:30.560
productive they were right it's like oh no there's tons of ways to just like waste tokens i think

349
00:22:30.640 --> 00:22:34.420
recently added to that leaderboard was also like git commits because we have people that are just

350
00:22:34.440 --> 00:22:38.800
like using claude in a loop and it's like literally the guy that created open claw or

351
00:22:38.920 --> 00:22:44.320
or whatever and then a levels io or like the number two like one and two on the github uh

352
00:22:44.420 --> 00:22:47.799
commits leaderboard like obviously they're ai people you know what i mean that's how it's going

353
00:22:47.900 --> 00:22:53.520
to play out now friend of the show zach corman uh was tweeting about kind of dunking on open

354
00:22:53.560 --> 00:22:57.260
claw and the maker responded to him like hey can we stop dunking on open claw like

355
00:22:57.823 --> 00:23:01.320
yeah there was some security issues but like it's been four months and like microsoft is

356
00:23:01.380 --> 00:23:06.240
rolling us out now and then the replies were like microsoft famous for never rolling out security

357
00:23:06.621 --> 00:23:12.260
vulnerabilities like yeah oh in four months wow you must have solved it like all security solved

358
00:23:12.742 --> 00:23:17.120
yeah when uh when uh satya was at the conference where he said like they were going to integrate

359
00:23:17.341 --> 00:23:24.040
open claw into windows i saw so many sub tweets on that of literally like five months ago microsoft

360
00:23:24.200 --> 00:23:29.280
being like open claw plenty of vulnerabilities unsecure like don't use it so yeah oh how the

361
00:23:29.340 --> 00:23:34.120
turntables with open ai is involved there's there's one more interesting point on this meta bit um i

362
00:23:34.200 --> 00:23:35.306
pulled up this article, right?

363
00:23:35.346 --> 00:23:37.980
That right at the end, and it's kind of like snuck in here,

364
00:23:38.080 --> 00:23:39.327
buried the lead a little bit, right?

365
00:23:39.367 --> 00:23:41.860
But it's like, it says the forward-looking question is

366
00:23:41.900 --> 00:23:48.220
whether Anthropic and OpenAI can build enterprise-grade deployment options that satisfy

367
00:23:48.401 --> 00:23:51.414
the data residency requirements of AI-native companies.

368
00:23:51.474 --> 00:23:52.940
And at the end, they kind of squirrel

369
00:23:53.040 --> 00:23:58.080
an editor's note that says, watch whether Anthropic and OpenAI respond with on-premise

370
00:23:58.302 --> 00:24:00.080
or air-gapped deployment options.

371
00:24:01.800 --> 00:24:04.180
This is, I mean, this like my spidey senses went ding

372
00:24:04.381 --> 00:24:07.800
because I used to work for a large bank, Bank of America.

373
00:24:08.220 --> 00:24:09.279
I don't care, I can say it, it's on my LinkedIn.

374
00:24:10.121 --> 00:24:13.580
But, and this was like a major, major thing there,

375
00:24:13.720 --> 00:24:14.599
pre-AI, right?

376
00:24:15.284 --> 00:24:16.599
I got hired there to be like

377
00:24:17.064 --> 00:24:18.459
head of cloud security architecture.

378
00:24:18.700 --> 00:24:19.879
And then I walk in the door and I was like,

379
00:24:20.561 --> 00:24:23.060
oh, basically every policy document here

380
00:24:23.182 --> 00:24:24.380
says don't use the cloud.

381
00:24:24.661 --> 00:24:25.880
Like, it's like all data,

382
00:24:26.584 --> 00:24:29.760
all the bank's data need to stay in the bank you can't send data over there that's not the bank

383
00:24:30.282 --> 00:24:35.280
right and so a lot of large enterprises have these like massive massive like policies compliance

384
00:24:35.681 --> 00:24:40.780
regulations government regulations whatever it is that say like your data and your customers data

385
00:24:41.181 --> 00:24:47.840
cannot go touch that other system without xyz like million controls in place i know some

386
00:24:47.860 --> 00:24:52.780
cybersecurity startups that i've been talking to that are really kind of behind and they feel

387
00:24:52.860 --> 00:24:56.280
they feel scared that they're falling behind on the whole AI curve because

388
00:24:56.320 --> 00:25:01.100
they have contractual obligations with government contracts that say that none

389
00:25:01.120 --> 00:25:04.020
of that government department data can touch an AI thing.

390
00:25:04.680 --> 00:25:05.480
And now the company's like,

391
00:25:05.580 --> 00:25:05.697
well,

392
00:25:05.881 --> 00:25:07.000
we can't like buy for,

393
00:25:07.080 --> 00:25:10.740
we're not big enough to like have gov cloud at our company that like,

394
00:25:11.348 --> 00:25:11.480
Oh,

395
00:25:12.140 --> 00:25:15.240
those contracts only touch this copy of our system.

396
00:25:15.305 --> 00:25:15.520
It's like,

397
00:25:15.580 --> 00:25:15.655
no,

398
00:25:15.740 --> 00:25:16.840
their system is their system.

399
00:25:17.060 --> 00:25:18.870
We're going to be nearly acquire like an H 200 rack for $250,000.

400
00:25:20.820 --> 00:25:20.840
Right.

401
00:25:20.884 --> 00:25:21.080
Or like,

402
00:25:21.120 --> 00:25:21.879
we don't have that kind of money.

403
00:25:22.040 --> 00:25:22.180
Right.

404
00:25:22.361 --> 00:25:25.360
And so their option right now is to just not.

405
00:25:25.980 --> 00:25:27.239
And so now they're like, well, crap.

406
00:25:27.520 --> 00:25:30.560
Like, are we, you know, how many years behind the curve now?

407
00:25:30.640 --> 00:25:36.740
Because we are like contractually obligated to not use these kinds of coding systems because of these contractual obligations.

408
00:25:37.081 --> 00:25:39.800
So this is why my Spidey senses went up is like, oh, yeah.

409
00:25:39.940 --> 00:25:47.460
Like, you know, that snuck in at the end of this article that like Meta is not going to be the first one to publicly say, hey, we can't do this.

410
00:25:47.580 --> 00:25:53.980
I'm sure the big financials have like whole little research teams off doing cool AI stuff.

411
00:25:54.201 --> 00:26:02.140
And then the rest of the employees, like multiple hundreds of thousands of employees at the U.S.'s biggest banks are not allowed to use these tools.

412
00:26:02.260 --> 00:26:02.763
I'm sure.

413
00:26:02.984 --> 00:26:03.265
Right.

414
00:26:03.547 --> 00:26:06.260
Unless they have like their own internal copy or something like that.

415
00:26:06.400 --> 00:26:10.560
And so, yeah, it's gonna be super interesting to see what they spit out that the enterprises could actually use.

416
00:26:10.860 --> 00:26:11.001
Yeah.

417
00:26:11.603 --> 00:26:11.724
Yeah.

418
00:26:11.744 --> 00:26:13.512
Take away for the for the viewer, for the for the listener.

419
00:26:13.532 --> 00:26:15.280
I mean, it's it's an interesting world out there.

420
00:26:15.542 --> 00:26:17.900
I don't think there's too much dooming to be done yet.

421
00:26:18.040 --> 00:26:21.000
I think the AI is still in a very like transitionary state.

422
00:26:21.060 --> 00:26:23.440
We're trying to figure out what to actually do with this technology.

423
00:26:23.642 --> 00:26:23.820
Right.

424
00:26:24.021 --> 00:26:28.540
In the world of finding bugs, it is scary where the regulation may go on AI.

425
00:26:28.720 --> 00:26:30.840
But I think the best thing you can do is like maybe a practitioner.

426
00:26:31.380 --> 00:26:31.839
Go play with them.

427
00:26:32.020 --> 00:26:33.080
Go, go, go and use the models.

428
00:26:33.402 --> 00:26:35.280
Well, first of all, go download one before they're illegal.

429
00:26:36.342 --> 00:26:38.680
But after that, go, go use the model to see if you can find some bugs.

430
00:26:38.740 --> 00:26:40.060
Go, go see what they're capable of doing.

431
00:26:40.100 --> 00:26:42.160
It's really neat to see them operate.

432
00:26:42.341 --> 00:26:47.560
whatever you think about like the the efficacy or the ethics behind ai watching them operate and

433
00:26:47.580 --> 00:26:52.960
like find bugs or like like reason about a code base is honestly like magical to watch it operate

434
00:26:53.060 --> 00:26:57.620
so i highly recommend everyone at least go go give it a shot and see what you know what uh what

435
00:26:57.761 --> 00:27:00.900
messes you can find yeah that's right i mean this is what i said about the free fable thing right

436
00:27:01.041 --> 00:27:04.940
is like you know you can't ostrich your head in the ground and pretend that this isn't happening

437
00:27:05.101 --> 00:27:10.520
just because you have like feelings about some of the stuff going on which was me by the way like i

438
00:27:10.620 --> 00:27:14.940
literally was that way up until like opus four six and i was like okay maybe it's time to try it

439
00:27:15.020 --> 00:27:18.780
out and i did and i was like wow it's actually really impressive i had a lot of similar opinions

440
00:27:19.001 --> 00:27:22.560
to you but i was still kicking the tires on this stuff all the time just not on anything i cared

441
00:27:22.620 --> 00:27:27.400
about right i was just like trying to keep up with because like all the harnesses that are coming out

442
00:27:27.440 --> 00:27:31.500
right now like there's like the new hot word because a lot of the capabilities at the most

443
00:27:31.620 --> 00:27:36.320
most frontier models that are super expensive or banned by the government you can replicate a lot

444
00:27:36.340 --> 00:27:43.020
those capabilities with the right harnesses around some of the still available and cheaper models

445
00:27:43.762 --> 00:27:48.619
and these these kinds of harnesses have existed for a while to try to especially pre-opus 4.6

446
00:27:49.000 --> 00:27:53.180
when the quality sucked and that's what you were seeing and a lot of us that were like really

447
00:27:53.280 --> 00:27:58.940
really staying at the tip of the spear were like yeah i get it but at the we're not using the out

448
00:27:58.960 --> 00:28:03.260
of the box models we're using the out of the box with like three different tools layered on top of

449
00:28:03.421 --> 00:28:06.000
to make sure that, you know, it was before loops were cool.

450
00:28:06.221 --> 00:28:07.500
Like we had stuff that was looping.

451
00:28:07.640 --> 00:28:09.160
We, you know, all sorts of stuff.

452
00:28:09.260 --> 00:28:12.400
There's, you know, been this community of kind of hackers doing a lot of this.

453
00:28:12.880 --> 00:28:15.320
And so, you know, that's all still going on.

454
00:28:15.380 --> 00:28:18.820
And so, yeah, I just like, we can't pretend that it's not.

455
00:28:18.920 --> 00:28:24.020
We have to get our hands on it to just like see what the capabilities are actually materializing.

456
00:28:24.364 --> 00:28:24.599
100%.

457
00:28:24.962 --> 00:28:27.680
Hey, hackers, before we keep going, I want to take a quick break and say a quick thank

458
00:28:27.721 --> 00:28:28.359
you to our sponsors.

459
00:28:28.760 --> 00:28:32.240
Today's Lowdown episode is sponsored by our friends over at Maze.

460
00:28:32.780 --> 00:28:37.980
Maize has spent the last couple of years using AI to find and remediate vulnerabilities in the cloud.

461
00:28:38.140 --> 00:28:44.860
And now they've released Maize Code, AI agents that find and deeply investigate vulnerabilities in your code.

462
00:28:45.122 --> 00:28:45.920
Now, we all know the deal.

463
00:28:46.340 --> 00:28:52.520
Software composition analysis tools and SaaS toolings are all historically really bad at just making more noise.

464
00:28:52.620 --> 00:28:59.300
They find issues that may not actually be real and just create a lot of problems for your engineers to deal with that don't solve real security vulnerabilities.

465
00:28:59.561 --> 00:28:59.989
I get it.

466
00:29:00.029 --> 00:29:00.579
We've been there.

467
00:29:01.300 --> 00:29:06.640
MaizeCode is a really cool tool because MaizeCode actually knows how your code works.

468
00:29:06.980 --> 00:29:11.920
MaizeCode investigates every finding with the context of how your code is actually deployed.

469
00:29:12.400 --> 00:29:15.020
Sure, you're using a version of libxml2 that's vulnerable.

470
00:29:15.341 --> 00:29:19.980
The actual code path to hit that vulnerability is not exposed in your Docker image.

471
00:29:20.400 --> 00:29:23.740
MaizeCode can find that and won't show you that that vulnerability is a big deal.

472
00:29:23.940 --> 00:29:28.500
Every vulnerability that MaizeCode reports comes with evidence that it's real and a way to fix it.

473
00:29:28.841 --> 00:29:32.920
Check out maze at go.lowdownpod.com slash maze.

474
00:29:33.060 --> 00:29:36.480
That's go.lowdownpod.com slash maze.

475
00:29:36.641 --> 00:29:37.180
Thanks again, maze.

476
00:29:37.381 --> 00:29:38.180
Let's keep going.

477
00:29:38.581 --> 00:29:40.079
All right, we promise we'll stop talking about AI.

478
00:29:40.360 --> 00:29:42.239
We'll move on to a couple of bugs here and there.

479
00:29:42.560 --> 00:29:48.960
So interesting bugs found actually in the Apple Beats by Dre, a vulnerability in the

480
00:29:49.020 --> 00:29:51.800
Bluetooth stack that allows you to, what is this?

481
00:29:51.961 --> 00:29:55.040
Get remote code execution on a pair of headphones.

482
00:29:55.460 --> 00:29:57.480
Interesting sentence I just said there.

483
00:29:57.581 --> 00:29:58.319
How is that possible, Matt?

484
00:29:58.340 --> 00:29:58.802
what are we doing?

485
00:29:58.903 --> 00:30:02.800
Yeah, I covered this one in a video last week because anytime I've seen these

486
00:30:02.840 --> 00:30:08.320
bugs before, like on AirPods or other headphones and the public does not like these because guess

487
00:30:08.380 --> 00:30:08.581
what?

488
00:30:08.681 --> 00:30:12.436
Like we're all like sticking speakers and microphones to our head.

489
00:30:12.497 --> 00:30:13.340
Most of the day I walk

490
00:30:13.360 --> 00:30:17.336
around my, I mean my, my AirPods are like my most used piece of technology.

491
00:30:17.617 --> 00:30:18.280
I walk around with an

492
00:30:18.300 --> 00:30:19.965
AirPod in a lot of my day.

493
00:30:20.748 --> 00:30:24.400
And when you have a security vulnerability in a microphone and speaker

494
00:30:24.520 --> 00:30:27.913
attached to most people, they tend to not like it, right?

495
00:30:27.933 --> 00:30:29.640
They're not like, oh man, can't wait for

496
00:30:29.640 --> 00:30:31.470
this hacker to listen into the microphone.

497
00:30:31.490 --> 00:30:33.340
And that's what this vulnerability would have allowed,

498
00:30:33.761 --> 00:30:35.647
which is kind of crazy.

499
00:30:36.449 --> 00:30:39.800
The caveat being, of course, with a lot of these Bluetooth

500
00:30:40.382 --> 00:30:45.220
vulnerabilities is you did need to be within a physical range of the device that you were

501
00:30:45.220 --> 00:30:45.602
hacking.

502
00:30:46.063 --> 00:30:50.240
That's not really like a warm and cozy, you know, if someone goes to a conference like

503
00:30:50.280 --> 00:30:54.380
this, we're filled with 7,000 people and you can walk around with a nice antenna in your backpack

504
00:30:54.400 --> 00:30:59.260
I mean, these were the DEF CON of old vulnerability, like hacks back in the day, right?

505
00:30:59.280 --> 00:31:05.640
Or even like RFID, early RFID things that would ping off of random antennas and stuff like this.

506
00:31:05.920 --> 00:31:08.738
But yeah, you do need to be within range of it.

507
00:31:09.160 --> 00:31:11.080
You're going to be way better at talking about the actual vulnerability.

508
00:31:11.400 --> 00:31:16.538
The one point that I will bring up before I toss it back to you is that this vulnerability was discovered a year ago.

509
00:31:17.680 --> 00:31:17.800
Yeah.

510
00:31:18.101 --> 00:31:20.960
So we can talk about kind of the nature of the bug and like the disclosure timeline.

511
00:31:21.160 --> 00:31:33.100
So like I said before, this bug, it gives you remote code execution on any device that uses a specific Bluetooth chip and Bluetooth SDK in particular, the software development kit.

512
00:31:33.481 --> 00:31:39.440
It's called the I'm going to pronounce this wrong, potentially AirOA, the word Air O-H-A, Bluetooth Audio SDK.

513
00:31:40.141 --> 00:31:46.760
The way that it works is literally the researchers that found it found a protocol that is implemented in the firmware of this controller.

514
00:31:47.205 --> 00:31:48.420
They called it the RACE protocol.

515
00:31:48.440 --> 00:31:49.801
I'm not sure what that acronym stands for.

516
00:31:49.821 --> 00:31:51.383
And it's also not like the real acronym.

517
00:31:51.423 --> 00:31:53.104
It's what they made up about this protocol.

518
00:31:53.525 --> 00:32:01.552
But effectively, it is a hidden, non advertised service on the Bluetooth chip that is meant to be for debugging the device.

519
00:32:01.892 --> 00:32:10.300
It gives you the ability to do arbitrary memory reads to RAM, arbitrary memory writes to RAM and the ability to do over the air firmware updates.

520
00:32:10.380 --> 00:32:15.000
And you can use that information to leak the Bluetooth key about the device.

521
00:32:15.140 --> 00:32:20.660
You can use that to connect to the device without pairing to it properly and ultimately use it to initiate phone calls.

522
00:32:21.080 --> 00:32:24.540
Listen to the microphone data on the headphones while they're paired to another device.

523
00:32:24.620 --> 00:32:27.220
You basically get total control over the headphones.

524
00:32:27.320 --> 00:32:27.840
Kind of a fun fact.

525
00:32:27.880 --> 00:32:30.880
You also could push a bad firmware update and brick the device.

526
00:32:31.040 --> 00:32:33.640
You know what you get from bricking headphones of a friendly neighbor.

527
00:32:33.762 --> 00:32:34.500
And I really couldn't tell you.

528
00:32:34.580 --> 00:32:36.460
But yeah, it's a scary attack.

529
00:32:36.520 --> 00:32:41.180
and this is kind of what you said before like the only attack nowadays that really matters when it

530
00:32:41.180 --> 00:32:44.140
comes to like proximal stuff like if you're at defcon and you're afraid of getting popped or

531
00:32:44.220 --> 00:32:49.940
things like this it's the low level it's the name of my youtube channel um the the low level

532
00:32:50.000 --> 00:32:55.580
vulnerabilities in the firmware of controllers like your wi-fi controller your baseband controller

533
00:32:55.640 --> 00:33:00.760
your bluetooth controller these are the areas that get the least attention from people because

534
00:33:00.840 --> 00:33:05.960
the density of the skill set of people on planet earth that know how to like extract this stuff

535
00:33:06.101 --> 00:33:08.308
find bugs exploit them is increasingly low.

536
00:33:08.730 --> 00:33:10.697
Again, to mention AI again, I apologize.

537
00:33:10.717 --> 00:33:11.500
This is why

538
00:33:12.142 --> 00:33:18.260
AI is being touted as like this vulnerability research tools because they the models not

539
00:33:18.561 --> 00:33:22.000
trying to personify them, but like are really good at like learning about a specific technology,

540
00:33:22.121 --> 00:33:26.460
you give it a binary and say, hey, this lives inside this microchip, it can deduce bugs like

541
00:33:26.520 --> 00:33:26.761
this.

542
00:33:27.304 --> 00:33:30.617
So yeah, there have been patches that have been fixed that have fixed this.

543
00:33:30.838 --> 00:33:31.260
The reason this

544
00:33:31.340 --> 00:33:36.660
broke the news by the way is um you know one of the line of apple headphones and not airpods but

545
00:33:36.700 --> 00:33:43.440
the the beat studio buds that apple now produces uses this uh this this chip and uses the sdk and

546
00:33:43.500 --> 00:33:47.717
therefore was vulnerable to it the patch just came out a about a week ago at this point june 19th

547
00:33:48.621 --> 00:33:53.219
the interesting part though going back to the the disclosure timeline the researchers of the

548
00:33:53.701 --> 00:33:58.937
the researchers that found this vulnerability found it in march of 2025 they didn't get contact

549
00:34:00.301 --> 00:34:03.638
from the SDK developer until May of 2025.

550
00:34:04.701 --> 00:34:06.560
And then they worked with various manufacturers

551
00:34:06.802 --> 00:34:08.659
that make headphones that use this SDK.

552
00:34:08.980 --> 00:34:11.220
So it was like Sony, Marshall,

553
00:34:11.864 --> 00:34:12.720
a couple of other companies

554
00:34:13.163 --> 00:34:15.260
to figure out how they can quickly put out updates

555
00:34:15.340 --> 00:34:18.054
before going into closing this live at 3.9.C.3,

556
00:34:19.683 --> 00:34:20.760
something chaos convention.

557
00:34:21.301 --> 00:34:22.519
It's a conference that happens.

558
00:34:22.680 --> 00:34:23.360
It's a hacker conference.

559
00:34:24.160 --> 00:34:25.420
And so, you know, that went live

560
00:34:25.460 --> 00:34:26.760
and then Apple kind of kicked it off

561
00:34:26.820 --> 00:34:28.980
and made it more public by not pushing out a patch

562
00:34:29.021 --> 00:34:29.760
to fix this vulnerability.

563
00:34:30.441 --> 00:34:35.640
in these beats so you know take away for the viewer if you use a um a headphone that uses

564
00:34:35.941 --> 00:34:40.420
sdk and you've been using them since i mean basically since this vulnerability came out the

565
00:34:41.384 --> 00:34:44.740
the timeline for when the vulnerability got created no one really knows but we definitely

566
00:34:44.820 --> 00:34:49.780
know someone knew about it the researchers in march um there is a world where your headphones

567
00:34:50.041 --> 00:34:54.460
could have bad firmware in them and could be doing evil stuff to you what does that mean maybe maybe

568
00:34:54.480 --> 00:34:58.560
you get new headphones maybe you make sure your firmware is patched um a couple courses back from

569
00:34:58.580 --> 00:34:58.841
there.

570
00:34:58.861 --> 00:35:04.860
But yeah, these kinds of firmware updates, I always hate because it's like, we can't we can't

571
00:35:04.860 --> 00:35:07.728
get people to hit the restart my browser button, right?

572
00:35:07.909 --> 00:35:12.000
Like, these kinds, they should get updated

573
00:35:12.120 --> 00:35:12.783
automatically.

574
00:35:13.003 --> 00:35:17.760
Like if you're connected to power and within Bluetooth range of your phone or your

575
00:35:17.820 --> 00:35:22.800
computer that it's paired with, I've definitely I remember a few AirPods vulnerabilities very

576
00:35:22.900 --> 00:35:23.623
similar to this.

577
00:35:23.985 --> 00:35:27.940
And I remember being pretty frustrated trying to get the damn thing to catch

578
00:35:28.000 --> 00:35:33.760
a firmware update and there's like no interface you know it's not like you can push buttons on

579
00:35:33.880 --> 00:35:40.600
your airpods and like get it to like update right it's like okay the thing said plug it in and leave

580
00:35:40.640 --> 00:35:45.240
it near my mac and i remember taking me like a few days to like i was like okay i gotta like leave

581
00:35:45.280 --> 00:35:50.260
this plugged in next to this like okay let me unplug it let me restart my phone or restart like

582
00:35:50.441 --> 00:35:53.620
just a lot of just weird troubleshooting and checking the firmware settings and i wasn't

583
00:35:53.720 --> 00:35:54.464
catching the update.

584
00:35:54.624 --> 00:35:58.180
I think even that scenario, like you were lucky in that you were trying to

585
00:35:58.340 --> 00:36:02.194
update a piece of hardware that lived within the Apple ecosystem.

586
00:36:02.655 --> 00:36:03.920
I would imagine if you're in the

587
00:36:03.980 --> 00:36:07.434
Sony ecosystem or like I think there was one line of Bose headphones.

588
00:36:07.735 --> 00:36:08.860
If you don't have like the

589
00:36:09.041 --> 00:36:13.780
Sony app on your phone or the Bose app on your phone, you're probably not getting an update from

590
00:36:13.800 --> 00:36:16.696
these companies unless you go and seek it out by installing the app and running it.

591
00:36:16.978 --> 00:36:17.320
And again,

592
00:36:17.862 --> 00:36:20.996
if you're listening to this podcast, you're a security conscious person.

593
00:36:21.397 --> 00:36:21.980
A lot of people on

594
00:36:22.020 --> 00:36:25.560
planet earth aren't and they're not going to update the firmware on their headphones right

595
00:36:25.600 --> 00:36:28.880
so there's going to be a lot of people walking around planet earth that uh will still be open

596
00:36:28.940 --> 00:36:33.700
to this right so kind of a scary thing and yeah we'll never get stats on this one because you

597
00:36:33.740 --> 00:36:38.780
can't scan the internet for this is a proximity thing so we'll never really fully understand like

598
00:36:39.522 --> 00:36:44.400
well was that patch like successful and rolling out automatically to people like at a at a large

599
00:36:44.440 --> 00:36:51.540
enough clip like this is like a decent one to you know keep in the arsenal uh you know if you're a

600
00:36:51.540 --> 00:36:55.700
pen tester or something like that to see what's what's out there and what's in people's pockets.

601
00:36:57.221 --> 00:37:01.680
Yeah, I this is a frustrating what we can't get people to update the firmware on their Wi-Fi

602
00:37:01.800 --> 00:37:05.373
routers that are like very sensitive to over the Internet attacks.

603
00:37:05.755 --> 00:37:07.220
Never mind like, oh, let me

604
00:37:07.441 --> 00:37:12.398
check my beats firmware settings somewhere in some, you know, hidden menu.

605
00:37:12.518 --> 00:37:13.060
It's just not going

606
00:37:13.060 --> 00:37:13.441
to happen.

607
00:37:13.461 --> 00:37:13.662
Right.

608
00:37:13.863 --> 00:37:14.124
Yeah.

609
00:37:14.164 --> 00:37:18.540
The you know, the adage of, you know, the S in IOT stands for security

610
00:37:19.203 --> 00:37:22.300
continues to live on not necessarily the headphones or iot but it seems that like

611
00:37:22.501 --> 00:37:26.620
anything embedded tends to have bugs again i think the reason being that they tend to have

612
00:37:26.680 --> 00:37:31.120
the least amount of eyes on them and so the least amount of scrutiny unfortunately yeah i mean these

613
00:37:31.160 --> 00:37:35.179
are these are this one was also kind of a cool poc that they released because it was also

614
00:37:36.103 --> 00:37:41.140
um able to initiate and receive calls and also read off of the memory of the device which

615
00:37:41.220 --> 00:37:47.160
apparently actually did include some like call history uh type information on your head like

616
00:37:47.180 --> 00:37:51.980
you just wouldn't think right that my bluetooth headphones might have a copy of my call history

617
00:37:52.201 --> 00:37:55.900
or the ability to initiate phone calls and stuff like that but of course they do right if you hit

618
00:37:55.960 --> 00:37:59.800
the you know they have tap functionality or whatever it is anyway kind of a spicy one in

619
00:37:59.900 --> 00:38:04.099
my opinion and especially it's been out for a year before the patch even came out and this poc

620
00:38:04.341 --> 00:38:07.440
dropped a while ago too i'm shocked this didn't get more attention i feel like this would be a

621
00:38:07.480 --> 00:38:11.560
thing you'd hear more about i mean it's proximity because it's proximity yeah but still go to a

622
00:38:11.600 --> 00:38:15.180
conference like defcon my point being there is like that's kind of the only bug that matters

623
00:38:15.240 --> 00:38:20.120
anymore is like this kind of attack like this if you connect the defcon wi-fi you're actually fine

624
00:38:20.200 --> 00:38:24.800
like literally unless you have like an unpatched windows 10 box you're gonna be a-okay but a bug

625
00:38:24.880 --> 00:38:30.140
like this is like actually a big deal it's probably the most watched network right like the knock

626
00:38:30.220 --> 00:38:33.920
there have you ever taken a tour of it i mean then it's probably got the most eyeballs on it

627
00:38:33.960 --> 00:38:40.420
with the most talent density of spotting shitty behavior like on any network at any point so yeah

628
00:38:40.500 --> 00:38:44.400
you're good people bring burners or whatever no you're fine right but yeah this is kind of spicy

629
00:38:44.600 --> 00:38:56.040
I mean, I'm not even thinking about DEF CON or like a crowded environment like that, super targeted towards like government officials or people that might bring their headphones in a pocket into a room with insider information that might move the stock market.

630
00:38:56.281 --> 00:39:04.420
Like, I mean, like microphone attacks are spicy and especially ones that would not really throw a lot of detections like that.

631
00:39:04.520 --> 00:39:07.680
You know, if you have to pop someone's laptop to get access to their microphone.

632
00:39:08.346 --> 00:39:08.752
Good luck.

633
00:39:08.854 --> 00:39:08.997
Right.

634
00:39:09.037 --> 00:39:09.200
Right.

635
00:39:09.801 --> 00:39:18.320
uh but what kind of edr is going to fire on your beats rce right uh ed's headphones are beaconing

636
00:39:18.420 --> 00:39:22.700
to china what's happening right now like yeah kind of spicy the psc was actually pretty gnarly too

637
00:39:22.780 --> 00:39:26.800
and in in like exfiltrating the info and eavesdropping on conversations and call history

638
00:39:26.840 --> 00:39:32.120
and all that so uh kudos to the researchers to to even release like a polished psc like that too

639
00:39:32.140 --> 00:39:36.660
yeah 100 all right what's next speaking of researchers uh nightmare eclipse is making

640
00:39:36.921 --> 00:39:37.624
is making waves.

641
00:39:37.665 --> 00:39:38.348
You can talk about that.

642
00:39:38.931 --> 00:39:39.433
What, yeah.

643
00:39:39.493 --> 00:39:40.800
What don't we want to talk about

644
00:39:41.502 --> 00:39:43.167
on nightmare eclipse?

645
00:39:43.528 --> 00:39:47.360
So this one, I think this is the story that just keeps on giving.

646
00:39:47.541 --> 00:39:49.648
And I think there's more to it than the public knows.

647
00:39:50.310 --> 00:39:53.440
Um, I've talked to a fair few people that

648
00:39:53.480 --> 00:39:57.980
might have some inside info on this one, uh, that basically have given me the, like,

649
00:39:59.083 --> 00:40:02.417
I can't tell you anymore that you're not, you don't have all the info.

650
00:40:02.598 --> 00:40:03.220
The news doesn't have

651
00:40:03.220 --> 00:40:03.903
all the info.

652
00:40:04.425 --> 00:40:05.932
There's lots of stuff going on.

653
00:40:06.012 --> 00:40:07.920
And if you read between the lines in Nightmare Eclipses,

654
00:40:08.020 --> 00:40:12.758
like comms and blogs that he's putting out, it's obviously like in turmoil, right?

655
00:40:13.059 --> 00:40:13.360
In like,

656
00:40:13.621 --> 00:40:17.140
there's much more going on between him and Microsoft than we know about in the blogs.

657
00:40:17.240 --> 00:40:20.000
And Microsoft being a big corporation can't exactly come out and be like,

658
00:40:20.901 --> 00:40:23.490
well, you see our lawsuit that's going on over here.

659
00:40:24.012 --> 00:40:26.420
There's some rumblings that they're actually

660
00:40:26.460 --> 00:40:27.946
an insider or they were an insider.

661
00:40:29.071 --> 00:40:31.420
And so I think that I've heard from a few credible sources that

662
00:40:31.420 --> 00:40:35.460
that might actually be a possibility but there's be missing context by the way nightmare eclipse

663
00:40:36.202 --> 00:40:42.120
is a researcher that was on github and for fun they dropped at this point like six or seven

664
00:40:42.862 --> 00:40:48.660
windows defender or some kind of windows zero day that give you a local account to system escalation

665
00:40:49.322 --> 00:40:53.980
the reason they're dropping these zero days is apparently they tried to submit a couple of these

666
00:40:54.040 --> 00:40:58.840
to msrc the microsoft security response center which runs the microsoft bug bounty program right

667
00:40:58.880 --> 00:41:01.640
So a bug like that where you can escalate from one account to the other.

668
00:41:02.461 --> 00:41:06.640
Actually, technically, MSRC doesn't respect the the admin to system boundary.

669
00:41:06.720 --> 00:41:08.880
But if you're like a normal user, they will pay you for an LPE.

670
00:41:09.480 --> 00:41:12.860
Apparently, they submitted these bugs or at least one of them to MSRC.

671
00:41:13.460 --> 00:41:15.260
MSRC said, nope, does not meet the bug bar.

672
00:41:15.400 --> 00:41:17.440
Sorry, you know, they don't give the money to the for the bug.

673
00:41:17.460 --> 00:41:18.500
And they said, OK, F you.

674
00:41:18.600 --> 00:41:20.479
And they they kick it off to the rest of the world.

675
00:41:21.000 --> 00:41:21.137
Yeah.

676
00:41:21.401 --> 00:41:26.017
As Matt was alluding to before, there's some rumors that that person may be a fairly young

677
00:41:26.800 --> 00:41:32.160
because you can kind of read in the way that they write be in some kind of turmoil, like financial

678
00:41:32.281 --> 00:41:35.640
or otherwise, because they said like they're living out of their car because of Microsoft

679
00:41:35.720 --> 00:41:38.211
and Tom Gallagher, who was like the lead of MSRC, right?

680
00:41:38.231 --> 00:41:39.176
You know, calling him out.

681
00:41:39.196 --> 00:41:39.919
So,

682
00:41:40.220 --> 00:41:42.691
yeah, definitely not the most stable person.

683
00:41:42.772 --> 00:41:44.600
You know, you can read that from not only their text,

684
00:41:44.680 --> 00:41:47.530
but also dropping seven zero days publicly for fun.

685
00:41:48.594 --> 00:41:50.440
But yeah, so the most recent one is Rogue

686
00:41:50.540 --> 00:41:52.267
Planet, which is a pretty interesting exploit.

687
00:41:52.428 --> 00:41:55.380
If you are a defender and you're listening to this,

688
00:41:55.400 --> 00:41:59.020
There actually is no patch for Rogue Planet right now at the time of this recording.

689
00:41:59.421 --> 00:42:03.200
So if you want to look for the IOCs, we'll put a link to maybe an article that has them.

690
00:42:03.700 --> 00:42:08.580
But basically, it's a vulnerability in a race condition in Defender, right?

691
00:42:08.600 --> 00:42:13.640
So Microsoft Windows Defender is the called EDR antivirus that lives on Windows.

692
00:42:14.720 --> 00:42:20.200
And so when Windows is scanning for viruses, if it sees a file that is potentially malicious,

693
00:42:20.460 --> 00:42:23.920
what it's going to do is scan the file, put it somewhere for quarantine, do more research

694
00:42:24.000 --> 00:42:29.219
on it do more more uh scanning on it and then go put it back okay well the time in between it

695
00:42:29.400 --> 00:42:33.340
taking the file and putting it back there's a race condition where you can put what's called

696
00:42:33.400 --> 00:42:37.800
an op lock and windows operational lock so that windows defender gets blocked on its operation

697
00:42:37.840 --> 00:42:44.160
and effectively the exploit replaces the now considered safe file with a piece of malware

698
00:42:44.321 --> 00:42:48.760
and then uses what effectively in windows is a sim link to point the temporary directory

699
00:42:49.242 --> 00:42:53.680
back to system 32 and then bada bing bada boom microsoft defender is putting malware

700
00:42:54.426 --> 00:42:55.980
over a legitimate Windows service.

701
00:42:56.120 --> 00:42:58.020
So you use the Windows Event Manager in the example,

702
00:42:58.140 --> 00:43:01.260
but you could use any system level service.

703
00:43:01.842 --> 00:43:03.880
So race conditions are alive and well

704
00:43:04.021 --> 00:43:05.695
in the year of our Lord, 2026.

705
00:43:07.224 --> 00:43:08.640
And yeah, there's no patch for this right now.

706
00:43:08.720 --> 00:43:10.440
The IOCs, companies like, for example,

707
00:43:10.500 --> 00:43:11.238
ThreatLocker put out,

708
00:43:11.900 --> 00:43:13.031
or you can look for programs

709
00:43:13.960 --> 00:43:15.309
that are not Windows Defender

710
00:43:17.565 --> 00:43:18.939
using the Windows Defender DLL

711
00:43:19.060 --> 00:43:20.040
is like the main one

712
00:43:20.101 --> 00:43:21.358
because they use that API

713
00:43:21.841 --> 00:43:27.720
to do the scanning and the replacement of the file but a crazy exploit not only just to exist

714
00:43:27.780 --> 00:43:32.340
but like to be dropping publicly for free yeah you you explain this to one to me before we hit

715
00:43:32.400 --> 00:43:39.000
record and it just blew my mind right that that the exploit actually just holds the file in place

716
00:43:39.241 --> 00:43:44.820
before defender can like do the defender thing on it i mean yeah that's that's sick that's just

717
00:43:44.840 --> 00:43:51.560
like really creative uh stuff yeah the the nightmare eclipse saga it's i've heard it described

718
00:43:51.701 --> 00:43:56.520
as painful disclosure uh you know you've got responsible disclosure and then what's going on

719
00:43:56.560 --> 00:44:02.960
here is painful it's just like one after the other of these public pocs dropping i also read this one

720
00:44:03.461 --> 00:44:08.759
uh is this the same one i don't have it on my screen where uh the poc that they put out was for

721
00:44:09.662 --> 00:44:16.500
windows uh 10 and 11 but it actually does work on windows server they the nightmare eclipse said

722
00:44:17.122 --> 00:44:21.920
this poc will not work on windows server but this vulnerability does exist you just need to go tweak

723
00:44:22.000 --> 00:44:28.100
the poc yeah so the way that the poc that's public works is they give you an iso file that is mounted

724
00:44:28.341 --> 00:44:34.900
as read only and you trigger the uh the actual vulnerability on the iso but then you make like

725
00:44:34.940 --> 00:44:39.620
a non email like a writable copy locally and use that to do the junction exploit right so

726
00:44:39.981 --> 00:44:44.840
the reason it's not vulnerable on server is you can't mount isos as a non-privileged user

727
00:44:44.900 --> 00:44:50.780
on server but i imagine through some other file system voodoo you can make a file that is read

728
00:44:50.840 --> 00:44:55.680
only somewhere copy it somewhere else effectively what i'm saying is you don't need the iso i believe

729
00:44:55.780 --> 00:44:59.880
to make this vulnerability work because at the end of the day all the real vulnerability has

730
00:44:59.940 --> 00:45:04.140
nothing to do with the iso it has to do with the time of check time of use race condition

731
00:45:04.180 --> 00:45:08.840
in defender checking the file and putting it back where it found it um and so if you can figure out

732
00:45:08.860 --> 00:45:14.599
a way to execute that without the need for an iso file the server is fair game too yeah i think uh

733
00:45:15.301 --> 00:45:19.540
the the larger picture here too besides this i mean a bunch of these exploits have been super

734
00:45:19.600 --> 00:45:24.980
cool the psc i mean yellow key was crazy right the bitlocker stuff um these this is obviously

735
00:45:25.060 --> 00:45:31.000
a talented researcher and like you said obviously you know going through something right um legally

736
00:45:31.301 --> 00:45:33.268
going through something from what it seems like as well.

737
00:45:34.533 --> 00:45:36.340
But what it's brought to light,

738
00:45:36.500 --> 00:45:40.220
and I think we've talked about this a fair bit on camera, both of us already, but like the

739
00:45:40.420 --> 00:45:46.298
reputation of your bug handling team, it should be considered sacred for security teams.

740
00:45:46.819 --> 00:45:47.060
So yeah,

741
00:45:47.200 --> 00:45:50.360
obviously, Nightmare Eclipse isn't the only game in town when it comes to being able to find

742
00:45:50.460 --> 00:45:53.290
exploits in large enterprise products.

743
00:45:53.491 --> 00:45:56.140
And I think the larger picture here is that the reputation

744
00:45:56.642 --> 00:46:00.400
of your bug handling system and team like MSRC

745
00:46:00.902 --> 00:46:04.180
should be considered sacred for security teams out there

746
00:46:04.301 --> 00:46:07.080
because what this has brought to light,

747
00:46:07.220 --> 00:46:11.720
I know we both have kind of beat this to a pulp on camera before,

748
00:46:11.920 --> 00:46:17.560
but the community has collectively experienced a lot of pain

749
00:46:17.881 --> 00:46:21.220
interacting with a lot of these large enterprise bug bounty programs

750
00:46:21.543 --> 00:46:23.520
or even just disclosure processes.

751
00:46:23.741 --> 00:46:27.580
and Microsoft being one of the bigger ones, right?

752
00:46:27.681 --> 00:46:30.640
They have a very large code footprint out there

753
00:46:30.862 --> 00:46:32.620
on the internet and on hardware and everything

754
00:46:33.985 --> 00:46:38.160
full of code that is, let's say, target rich, right?

755
00:46:39.245 --> 00:46:40.680
In terms of the kind of vulnerabilities

756
00:46:40.762 --> 00:46:41.480
that we're talking about.

757
00:46:42.304 --> 00:46:43.640
And over and over again,

758
00:46:43.700 --> 00:46:47.380
the researchers seem to be running into a few issues.

759
00:46:47.823 --> 00:46:50.040
And I think the security community is pretty used

760
00:46:50.181 --> 00:46:51.760
to this being a difficult process

761
00:46:51.760 --> 00:46:53.020
because it's a hard one to get right.

762
00:46:53.600 --> 00:46:56.560
is at scale having to go through this triage process.

763
00:46:56.680 --> 00:46:57.279
Have you ever lived it?

764
00:46:57.460 --> 00:47:00.019
Have you ever had to triage a bunch of output from tools?

765
00:47:00.482 --> 00:47:00.878
Oh, yeah.

766
00:47:01.381 --> 00:47:03.859
That was my job before I quit my last job exclusively.

767
00:47:04.687 --> 00:47:05.280
It's horrendous.

768
00:47:05.441 --> 00:47:06.080
It's painful, right?

769
00:47:06.341 --> 00:47:10.580
And so I think we all understand that I've been on both sides of this.

770
00:47:11.082 --> 00:47:13.420
I've disclosed bugs to bug bounties.

771
00:47:13.600 --> 00:47:16.380
I'm an early Google bug bounty receiver, right?

772
00:47:16.521 --> 00:47:20.320
And I've been on the other side triaging findings of both scanners and humans.

773
00:47:21.263 --> 00:47:22.920
The quality varies greatly, right?

774
00:47:23.140 --> 00:47:28.345
So I think there's a certain tolerance that the community has to understanding what the other side of this is like.

775
00:47:28.906 --> 00:47:42.960
But the thing that keeps happening that I think people are starting to revolt against due to the, you know, maybe Nightmare Eclipse is a match being thrown into a powder keg of like pent up frustration around this issue, not even just Microsoft, right?

776
00:47:43.020 --> 00:48:01.060
We're seeing very similar stuff come out about Apple and their handling of things and some other programs, other big programs is when a bug gets disclosed, the process is hard, slow, painful, not very communicative, any combination of the above.

777
00:48:01.983 --> 00:48:08.440
then the response is lackluster so either you get marked as a duplicate with very little information

778
00:48:08.661 --> 00:48:14.100
or you get marked as not actually a severe security issue and then the one-two punch that

779
00:48:14.120 --> 00:48:19.920
i think is what's getting specifically msrc in a lot of trouble is and then fixing the vulnerability

780
00:48:20.121 --> 00:48:25.240
anyway right and then not crediting the researcher uh for it right and that's what i think is going

781
00:48:25.300 --> 00:48:29.359
on here yeah that is the the difficult balance of this yeah so from a timeline perspective i

782
00:48:29.620 --> 00:48:33.820
understand the timelines because yeah the amount of reports they probably get and the amount of

783
00:48:33.920 --> 00:48:39.000
people that they don't have to triage these varying quality vulnerability reports of a bug

784
00:48:39.040 --> 00:48:45.500
that may or may not actually be real especially with ai creates you know an internal uh inertia

785
00:48:45.761 --> 00:48:50.560
to solve the problem so i heard on that 100 but it does get very hairy where it's like okay

786
00:48:50.881 --> 00:48:56.520
like this is a real bug in the software our company is incentivized to have a better product

787
00:48:56.881 --> 00:49:01.200
but also like because of what the bug bounty contract legally says we don't have to give you

788
00:49:01.260 --> 00:49:05.380
any money so what do we do with that and yeah microsoft's got caught with their pants down

789
00:49:05.560 --> 00:49:11.960
multiple times on not not acknowledging the researcher in the cve 60 days will go by the

790
00:49:12.000 --> 00:49:15.220
cve comes out the bug is fixed and they don't hear anything they'll get an email back right

791
00:49:15.461 --> 00:49:19.740
which obviously is not the right answer um so it's it's a difficult balance and uh microsoft

792
00:49:19.840 --> 00:49:23.860
has not been very good at at finding this the center point yeah and i think i zoom out here

793
00:49:23.961 --> 00:49:27.040
not just to like dunk on MSRC or anything like that.

794
00:49:27.100 --> 00:49:29.400
It's more just to say like, this isn't going to be it.

795
00:49:29.621 --> 00:49:32.260
Like it's not going to be nightmare eclipse is going to be the only person out

796
00:49:32.320 --> 00:49:37.840
here publicly disclosing POC is for stuff because the responsible,

797
00:49:38.461 --> 00:49:41.740
which I'm putting in air quotes because the responsible part is a two way

798
00:49:41.820 --> 00:49:42.162
street.

799
00:49:42.403 --> 00:49:45.800
And, and there's like a social contract there of like, Hey,

800
00:49:46.261 --> 00:49:50.380
the agreed upon path now is you've broken your side of the social contract.

801
00:49:51.063 --> 00:49:53.600
The agreed upon path is the community is public disclosure.

802
00:49:53.961 --> 00:49:56.500
to like get you to act right.

803
00:49:56.961 --> 00:49:59.360
Like, or just to acknowledge for the security

804
00:49:59.623 --> 00:50:01.160
of the community that the bug exists, right?

805
00:50:01.200 --> 00:50:02.800
That's kind of the whole point of disclosing like,

806
00:50:02.860 --> 00:50:05.140
hey man, your laptop has a vulnerability in it,

807
00:50:05.381 --> 00:50:06.720
figure it out so you don't get hacked, right?

808
00:50:06.720 --> 00:50:07.459
That's kind of the whole point.

809
00:50:07.480 --> 00:50:10.000
Yeah, the goal here is for us all

810
00:50:10.081 --> 00:50:12.500
to have a more secure internet and devices,

811
00:50:13.383 --> 00:50:16.640
not for like the multi-trillion dollar corporation

812
00:50:16.762 --> 00:50:17.839
to have like a good day

813
00:50:18.061 --> 00:50:20.160
and like the cheapest, quickest path possible

814
00:50:20.322 --> 00:50:21.940
to like whatever it is that they're doing, right?

815
00:50:22.000 --> 00:50:23.880
That's like not our incentive, right?

816
00:50:24.201 --> 00:50:28.460
Our incentive is, hey, I found a bug that can like cause some serious pain on the internet.

817
00:50:29.627 --> 00:50:30.178
You wrote it.

818
00:50:30.803 --> 00:50:32.039
I'm now telling you about it.

819
00:50:32.280 --> 00:50:35.700
Like, let's do this whole thing and like get this fixed.

820
00:50:35.780 --> 00:50:40.480
And instead, it's like, like I said, slow, painful, no credit, bug gets fixed anyway.

821
00:50:40.700 --> 00:50:41.840
It's like, okay, what am I doing?

822
00:50:41.920 --> 00:50:43.260
Why am I on this side doing this?

823
00:50:43.441 --> 00:50:43.918
You a favor.

824
00:50:44.340 --> 00:50:44.440
Yeah.

825
00:50:44.620 --> 00:50:49.040
There is an incentives problem too, where I think with, again, AI, sorry, with AI being

826
00:50:49.542 --> 00:50:55.060
as prevalent as it is as good if i find any bugs it is um the market value of these bugs is going

827
00:50:55.140 --> 00:50:59.320
to shift dramatically right like part again like you should be doing it for the love of the game

828
00:50:59.360 --> 00:51:02.520
for the love of the internet right but like people have to pay their bills and so a lot of people do

829
00:51:02.540 --> 00:51:07.640
this for money and if you can find a vulnerability in hyper-v or in windows defender or whatever and

830
00:51:07.640 --> 00:51:11.780
you get paid a hundred thousand dollars for it of course you're going to go and do that but if

831
00:51:11.920 --> 00:51:16.560
people now get paid maybe a thousand dollars because like everyone's doing it now like where

832
00:51:16.620 --> 00:51:17.365
does the incentive go?

833
00:51:17.386 --> 00:51:19.540
Do they now go to the black market and sell it for the same value?

834
00:51:19.600 --> 00:51:22.700
Do they now just go public and become an influencer because now they can get that

835
00:51:22.720 --> 00:51:26.620
kind of clout with a bug like that, you know, and then does that make the internet less secure over

836
00:51:26.720 --> 00:51:26.941
time?

837
00:51:26.981 --> 00:51:27.182
Right?

838
00:51:27.724 --> 00:51:32.040
Yeah, I think I mean, and even the black market or whatever, like we're seeing,

839
00:51:32.742 --> 00:51:36.880
I recently covered and talked to the researchers who found Karuna and Dark Sword, these two iOS

840
00:51:37.040 --> 00:51:43.800
exploits, and they called it a $40 million exploit kit that was just on a watering hole website,

841
00:51:43.981 --> 00:51:49.180
not even targeted on like a Chinese crypto scam website for one of them and like a Ukrainian

842
00:51:49.441 --> 00:51:50.907
like dating site on another.

843
00:51:50.927 --> 00:51:54.300
And, you know, these are the kinds of things that usually

844
00:51:54.400 --> 00:51:58.700
governments point at specific individuals like political opponents, dissidents, journalists,

845
00:51:58.840 --> 00:52:00.206
unfortunately, like stuff like this.

846
00:52:01.150 --> 00:52:03.118
And they just like put it out on a website.

847
00:52:03.279 --> 00:52:03.540
And like,

848
00:52:03.580 --> 00:52:07.959
that's why these researchers were able to just find it and like, you know, Ari it and do all

849
00:52:08.140 --> 00:52:09.525
the stuff that they did with it.

850
00:52:10.208 --> 00:52:13.480
And I asked them pointedly, I said, did they not know what they had

851
00:52:13.561 --> 00:52:14.660
or did they not care?

852
00:52:14.901 --> 00:52:16.520
And the researchers that I verified said,

853
00:52:17.007 --> 00:52:17.719
I don't think they cared.

854
00:52:18.284 --> 00:52:19.300
And they're like, what does that tell us?

855
00:52:19.340 --> 00:52:20.420
Does that tell us the value

856
00:52:20.782 --> 00:52:23.357
of a $40 million exploit kit is approaching zero

857
00:52:24.144 --> 00:52:25.520
for them to be able to just do that?

858
00:52:26.142 --> 00:52:30.320
Do they think that they can dial up another iOS zero day

859
00:52:30.401 --> 00:52:32.480
or 23 of them that were in this exploit kit?

860
00:52:32.600 --> 00:52:35.737
Like there was 23 exploits under Karuna's umbrella

861
00:52:36.584 --> 00:52:37.700
that like chained it all together

862
00:52:37.922 --> 00:52:39.600
to do these zero click things, right?

863
00:52:39.640 --> 00:52:43.320
You go to a website in Safari that had Karuna on it

864
00:52:43.521 --> 00:52:47.460
they had rce that's good that's not one vault right that's a chain of bones yeah by the way

865
00:52:47.480 --> 00:52:53.000
guys gray market value for one of those exploits is like four to eight million dollars yeah so the

866
00:52:53.040 --> 00:52:57.040
whole thing they said it was i i've never heard people speak about an exploit kit the way i've

867
00:52:57.080 --> 00:53:03.020
heard really smart people talk about karuna uh the words like elegant and beautiful and like i mean

868
00:53:03.080 --> 00:53:06.800
they were just fawning over this exploit kit and if you guys don't know the background it was written

869
00:53:06.981 --> 00:53:12.560
by government u.s government contractors l3 harris and trenchant and then leaked and the guy who

870
00:53:12.660 --> 00:53:13.705
leaked it is in jail.

871
00:53:13.806 --> 00:53:16.900
Yeah, this is the one, the L3 Harris guy that sold it to a Russian, right?

872
00:53:17.100 --> 00:53:17.521
Yeah, yeah.

873
00:53:17.561 --> 00:53:23.540
So a Russian exploit broker, who then sold it to both Russian buyers and Chinese buyers.

874
00:53:23.600 --> 00:53:25.950
So it wound up on this Chinese crypto scam website.

875
00:53:26.332 --> 00:53:28.180
Like I said, I mean, they couldn't

876
00:53:28.300 --> 00:53:31.069
have been brokered for that much money then at that point.

877
00:53:32.293 --> 00:53:34.600
Because of how it was then used,

878
00:53:34.720 --> 00:53:34.841
right?

879
00:53:34.861 --> 00:53:38.960
There's no way you're spending $40 million to use it on like a crypto scam website, right?

880
00:53:39.020 --> 00:53:43.340
that's like not going to return that and so yeah are we starting to see the tips of like

881
00:53:43.962 --> 00:53:48.780
the exploit even dark market crash right and value is super interesting it's like oh yeah

882
00:53:48.840 --> 00:53:53.180
where's the where's the economics of this all go if ai gets really good at writing this stuff

883
00:53:53.561 --> 00:53:58.520
yeah 100 i'm sorry i'm not going to apologize every time i say yeah i like you yeah yeah i know

884
00:53:59.303 --> 00:54:03.260
it's just the nature of the beast right it's a reflex from youtube because uh my youtube audience

885
00:54:03.380 --> 00:54:08.640
is not a not a huge ai fan you know i get it and i do get the revolt right everyone's like

886
00:54:08.740 --> 00:54:13.220
permanent underclass scared and like everyone's like oh the other thing that i'll just say like

887
00:54:13.260 --> 00:54:18.680
can we go on a tangent here really quick because we keep apologizing for it so like i i am of the

888
00:54:18.720 --> 00:54:23.179
camp that all these companies that are doing tens of thousands of layoffs under the guise of ai are

889
00:54:23.520 --> 00:54:28.280
full of shit right like so everyone who's like anti-ai because it thinks it's gonna like eat

890
00:54:28.400 --> 00:54:32.340
everyone's jobs and like all the water on the planet i think all of these things are actually

891
00:54:32.701 --> 00:54:33.524
not happening.

892
00:54:34.025 --> 00:54:39.080
I think these giant mega corps just have to do those layoffs so often or not

893
00:54:39.140 --> 00:54:41.109
have to, but like do those layoffs so often.

894
00:54:41.551 --> 00:54:43.760
And they just, they, the reason they come in waves

895
00:54:43.840 --> 00:54:45.949
is not actually like a macro economic thing.

896
00:54:45.989 --> 00:54:46.973
It's a PR thing.

897
00:54:47.496 --> 00:54:48.540
It's like, they're all peers

898
00:54:48.680 --> 00:54:49.182
of each other.

899
00:54:49.443 --> 00:54:53.738
And like, if Oracle can point to Microsoft and go, well, yeah, look, AI and the

900
00:54:54.440 --> 00:54:57.030
economy, that's why we had to cut 30,000 people.

901
00:54:57.230 --> 00:54:59.920
And not that like, Hey, by the way, like if we

902
00:54:59.960 --> 00:55:06.080
can keep revenue here and like corporate ozempic ourself down to like here and get rid of like

903
00:55:06.681 --> 00:55:13.760
15 of our most expensive thing which is you know our capex right on like all of these people like

904
00:55:13.820 --> 00:55:19.400
why wouldn't we do that and the pr case is well yeah of course ai right because they want to seem

905
00:55:19.460 --> 00:55:23.860
like they're at the forefront there's no effing way i know the people at the like tip of the spear

906
00:55:24.463 --> 00:55:27.700
of ai and especially at those kinds of organizations there's no way they're like

907
00:55:27.780 --> 00:55:32.474
taking 30,000 jobs worth of output and, and like justifying AI.

908
00:55:32.915 --> 00:55:34.440
I will say at like a small scale,

909
00:55:35.341 --> 00:55:39.260
companies that like can't afford to like startups and like smaller companies are definitely running

910
00:55:39.320 --> 00:55:44.020
a little leaner these days and maybe not hiring that like social media person because they're

911
00:55:44.180 --> 00:55:49.200
just, well, we'll have AI generate our LinkedIn posts until we can afford a real person because

912
00:55:49.200 --> 00:55:52.596
they realize the quality is lower or like, Oh, we're not hiring that role.

913
00:55:52.817 --> 00:55:53.500
Like, you know,

914
00:55:53.761 --> 00:55:58.360
they're pinching pennies in the smaller side i just think i don't know i maybe it's tinfoil

915
00:55:58.380 --> 00:56:04.120
hat of me but i i think on the large enterprise side all right i agree bananas i run tinfoil hat

916
00:56:04.160 --> 00:56:08.380
on most things whenever you think you have the answer for why a large entity is behaving a

917
00:56:08.420 --> 00:56:12.040
certain way you probably don't know the full story yeah so like i don't know if that helps

918
00:56:12.321 --> 00:56:16.340
the like anti-ai crowd for like looking at the thing that they think is going to like eat all

919
00:56:16.380 --> 00:56:20.820
the jobs i just don't think that that that's like coming i think that's all like i don't think that

920
00:56:20.981 --> 00:56:26.300
all of the ai stuff is all marketing right i get i get yelled at a lot by like people on the internet

921
00:56:26.441 --> 00:56:29.980
because i am talking about mythos capabilities and exploit development and whatever and they're

922
00:56:30.000 --> 00:56:34.239
like you're falling for the marketing and i'm like hey listen i'm not going off of dario's

923
00:56:34.681 --> 00:56:41.160
right press release i'm going out off like project glasswing members like who i know and respect in

924
00:56:41.200 --> 00:56:45.720
this industry are like oh this thing found rce like while i slept right yeah you can acknowledge

925
00:56:45.941 --> 00:56:50.400
that like it could be marketing and it also could be good or it could be somewhere in the middle

926
00:56:50.581 --> 00:56:54.840
where like it's just a yes they are go they're obviously using this to like generate a little

927
00:56:54.900 --> 00:56:59.640
bit of fear and then at the same time it is a better model objectively but like and like what

928
00:56:59.720 --> 00:57:03.120
happened they got banned because their marketing was so good right so like you know yeah people

929
00:57:03.200 --> 00:57:08.020
bought their marketing and they marketed themselves into a regulation situation anyway i don't know

930
00:57:08.020 --> 00:57:12.160
if that like calms ai nerves or whatever as we talk about it but like probably not but

931
00:57:14.066 --> 00:57:17.900
and on that note we'll wrap it for those of you listening we do appreciate it follow us on

932
00:57:18.121 --> 00:57:22.960
Spotify hit subscribe on YouTube and what are our closing remarks Matt what should we be doing

933
00:57:23.422 --> 00:57:28.920
what's the next play um I think just in general just want to say like we we started this I don't

934
00:57:28.940 --> 00:57:32.680
know should we should we like pull the curtain back a little bit like stuff going behind the

935
00:57:32.720 --> 00:57:38.460
scenes we recorded a few episodes of this by the way but we we caught some attention of of some

936
00:57:38.500 --> 00:57:43.078
sponsors and stuff like this and we had to pause and like get everything together we're gonna we're

937
00:57:43.921 --> 00:57:47.560
we're doing this like we're here in person to like do this we're gonna do this at least weekly

938
00:57:47.741 --> 00:57:52.280
uh we've committed to it we're like looking to grow the team to help us behind the scenes

939
00:57:52.581 --> 00:57:56.280
i mean yeah so if you're here and if you've lasted this long you're you've at least heard of

940
00:57:56.681 --> 00:58:00.980
us and you're here and you're chilling um we want to hear what you want to see out of this we're

941
00:58:00.980 --> 00:58:04.820
gonna have we have we have already like two or three exciting guests lined up we're going to

942
00:58:04.820 --> 00:58:08.320
start doing some interviews uh those you go to black hat and defcon we're going to do some cool

943
00:58:08.380 --> 00:58:12.700
sit uh you know stuff in vegas live shows and interviews and stuff like that so if you're here

944
00:58:12.780 --> 00:58:18.700
be here it's going to be cool um we're we're kind of joining forces low level and matt jay

945
00:58:19.602 --> 00:58:25.540
right like the the youtube conglomeration here um and and yeah let let us know where you're

946
00:58:25.600 --> 00:58:30.040
listening to this i want to know are we we're obviously doing youtube first but we're it's a

947
00:58:30.100 --> 00:58:35.580
podcast it's our first like official podcast as a bunch of youtubers and instagrammers um so i want

948
00:58:35.580 --> 00:58:40.120
to know if people are listening on like podcast apps stuff like this reach out where we we read

949
00:58:40.280 --> 00:58:45.100
the stuff that you guys send our way both good and bad this is a very personal like an interaction

950
00:58:45.120 --> 00:58:48.360
right you're listening to us talk to each other for over an hour and i want to know what your

951
00:58:48.400 --> 00:58:52.300
take is like what you took away from this what you liked and didn't like we love feedback so

952
00:58:52.340 --> 00:58:56.280
please give it to us and we'll uh see you all next week all right