Security for AI-Native Developers

This transcript was generated by AI and may contain mistakes, including in names and technical terms. It has not been manually verified. Please refer to the recording for accuracy.

Source: YouTube automatic transcript. Select a timestamp to watch that moment on YouTube.

1. 0:00

   All right. So, my name is Ismael. Um, and I'm going to talk to you about uh

2. 0:06

   security in the age of AI. And, uh, I'm I'm one of those guys who actually think

3. 0:14

   this is a problem. Uh, and I think that it's just going to get worse. And um so

4. 0:22

   I'm going to talk to you today about how attackers, you know, how they actually go about attacking a software using AI.

5. 0:32

   what has changed from five years ago when we didn't have any AI that could uh

6. 0:38

   attack software and then also uh what you guys can do about it when you're building software when you are building

7. 0:46

   apps with or without AI because that's the thing um cyber security has been around for quite some time so this is

8. 0:55

   not something new but something very amplified um so this is me I'm the co-founder and

9. 1:02

   CTO of Super Agent. Super Agent is a autonomous security factory. We work with a bunch of companies uh which you

10. 1:11

    might or might not know of. Um I'm an open- source contributor at heart. So I love open source. I work with open

11. 1:19

    source basically every day. Most of the tools that we build at super agent is open-source.

12. 1:26

    and I have been one of the core contributors to many popular projects uh that you might recognize on the right

13. 1:35

    hand side. Um my latest bounty which is always something that a hacker likes to

14. 1:41

    brag about is a bug pretty serious bug in Vext which is Cloudflare's copy of Nex.js.

15. 1:51

    Um so that just got published uh a week ago. Um, yeah, but that's me.

16. 1:58

    Uh, I'm going to talk to you, uh, start off by talking about what the anatomy of

17. 2:06

    a successful attack looks like. Many of us or many people that I talk to think that, you know, there's a bug somewhere in the system. An attacker has an AI.

18. 2:19

    They do a scan of a repository or they do a scan of a web page or an app. they fi find that bug they attack that bug

19. 2:28

    and that bug leads to like the hugging face incident right uh that's a very common misconception I

20. 2:36

    would say and that's not actually how it works u this picture is more uh I would say relevant to how a uh attacker

21. 2:46

    actually works and what they are very good at is finding stuff in our software

22. 2:54

    in your software that might look benign just by looking at it and is benign often. So one it could be one bug in one

23. 3:04

    part of the system that's just you know something that hey I'll I'll probably manage this later. I have other things

24. 3:10

    in my task list or on my to-do that I need to fix. And then there's some credential lying around on the internet

25. 3:18

    that somebody forgot to revoke. In the case of hugging face, that was one of the u you know attack chains that the uh

26. 3:28

    that the agent used to compromise that system. It could be other bugs in other

27. 3:35

    systems that are you know seamlessly unrelated to to the actual uh target

28. 3:42

    that the uh uh attacker is trying to compromise. uh and what attackers what were really good attackers and I'm not

29. 3:51

    talking about you know a bug bounty guy like myself I'm talking about state actors people from countries that have

30. 4:00

    in their doctrine to attack you know western countries or western companies in order to exfiltrate data and then

31. 4:08

    either blackmail that company or actually sell that data to a third party that might want to do malign things with

32. 4:14

    that data. They are very good at finding these, you know, crumbles, security

33. 4:22

    crumbles, you could call it, all over the place and then chain that together into a successful attack, right? That's

34. 4:30

    what they are very good at. That's what they have been good at and that's what makes them um, you know, in lack of a

35. 4:38

    better word, a great attacker or hacker, right? Um what has happened now is that

36. 4:45

    most of this work was done using quite rudimentary tools. Previously you had to

37. 4:53

    there was a there's a bunch of software that is also open- source that you can use to try to find these you know uh

38. 5:01

    different vulnerabilities and try to chain that together. The problem is that they are deterministic uh which means that it takes a lot of

39. 5:10

    time uh to find those bugs and vulnerabilities and chain them together.

40. 5:16

    uh and this has been the way it has been you know ever since I would say the launch of GPT4 which was the real first

41. 5:27

    powerful reasoning model that you could use on you know fairly long horizon

42. 5:33

    tasks. So what attackers do today is the exact it's the exact same recipe they

43. 5:40

    use but instead of doing it manually or doing it through you know deterministic software they use probabilistic software

44. 5:48

    and they use the same models we use to do our daily work you know but they use

45. 5:55

    it in a um in order to attack us more or less. So they can do you know 10,000

46. 6:02

    times more of this work than they could do previously. The same you know uh I would say the same uh increase in

47. 6:11

    productivity that we see they see in their in their work. Um, and one other

48. 6:18

    thing that is a common misconception is that, you know, security when when you're using AI to to penetrate or to

49. 6:26

    attack another system, it's not just, hey, I'm opening up chat GPT or my own, you know, a model and I'm prompting it

50. 6:35

    one time and it goes out and finds all of this stuff and an hour later I have a recipe for attacking something. That's very rarely the case.

51. 6:45

    Sophisticated attackers use thousands of agents that run in parallel, that work in uh together, and

52. 6:54

    that can run over days or weeks in order to find vulnerabilities. And that usually costs a lot of money. Uh so it

53. 7:02

    could be, you know, hundreds to a million dollars to depending on what target you have, how much money you

54. 7:09

    have, etc., etc. Uh the hugging face attack which is very interesting uh was you know thousands of agents

55. 7:17

    collaborating uh in a in a very long period of time and and that costs a lot of money and

56. 7:25

    and so we as defenders usually don't have the same budget. What I'm trying to say is that we as defenders don't have the same

57. 7:34

    budget to use AI as attackers do. So they have, you know, they have a competitive advantage. Uh and and one

58. 7:43

    other thing is that, you know, an attacker only needs to be right once in order to to actually attack a system. We

59. 7:51

    as defenders need to be right all the time. So there there is a huge competitive disadvantage for for defenders contra um an attacker.

60. 8:04

    But this is more or less the way they work.

61. 8:09

    Um and there are a bunch of different types of attacks as well. And I think that you know uh one thing that I agree

62. 8:16

    with with the previous speaker is that we focus a lot on you know code bugs in code and how that can uh then you know

63. 8:25

    end up being a successful attack. But if you look at how um sophisticated

64. 8:32

    attackers work, it's very rarely that they find an actual code bug and then are able to use that in order to attack

65. 8:41

    you. What usually happens or at least in 90% of cases is that you get fished

66. 8:47

    somehow. There's some social engineering that starts the whole chain. So it could be you, it could be one of your

67. 8:55

    colleagues, it could be an outside contri contributor, it could be a consultant you have, whomever has access

68. 9:03

    to that system. Um, so that's that's actually the first the first um point there is actually um I would say it

69. 9:13

    would stand for around 90% of successful attacks is social engineering, fishing emails, you know, uh DMs on LinkedIn.

70. 9:21

    Hey, do you want to I get them all the time. Do you want to be uh featured on TechCrunch? And then you go and look at the guy and he's working out of some

71. 9:30

    weird country and has nothing to do with Techrunch, right? I click that link, I schedule a meeting and boom, he has access to all my passwords, all my

72. 9:39

    credentials, my GitHub account, and can, you know, start publishing new updates to my packages on MPM. And then we have

73. 9:48

    a worm. That's usually how it works. 90% of the cases bugs in code is also something that they look for but that

74. 9:56

    would I would say be around 5% of all the cases that are actually that we actually see

75. 10:03

    uh one other thing that's uh interesting is bugs in your configuration that could be let's say your repository there's a

76. 10:10

    bunch of settings in your repository on GitHub that you need to set in a way where if an attacker actually gets

77. 10:18

    access to a credential that they shouldn't have access to that they still can't like manipulate that system. So there's a bunch of

78. 10:27

    configuration bugs as well that these attackers are very good at finding, right? And then pun credentials, there's

79. 10:36

    a bunch of them. Somebody should build some kind of service that finds all of them. But you know almost any company,

80. 10:43

    every company that we work with have some kind of credentials that are lying around that are still active. Could be

81. 10:50

    API keys, could be secrets, could be something they pasted in somewhere, forgot about you know the in some cases

82. 10:58

    it's even you know visible in the code itself. Um so that's also a uh a huge issue.

83. 11:08

    Um so

84. 11:12

    [clears throat]

85. 11:13

    if we look at you know uh the amount of attacks that has been uh since since the launch of GPT4 we can we can uh almost

86. 11:22

    see that it has doubled. Uh and I would I would think that this will just as as models get stronger this will just keep

87. 11:32

    on increasing. Um and and again one would think that hey we also have strong models as defenders but I as I said

88. 11:41

    earlier we are at a competitive disadvantage. Uh we don't have the same resources. We don't have the same um um

89. 11:51

    skill sets that these guys do and so we are at a disadvantage. So I would I would think that the 2026 number is

90. 11:59

    actually pretty low and we would see uh a a bigger number there but it is

91. 12:06

    increasing. So the question is uh what what can we do about it and what does good look like? uh and I've I've chosen

92. 12:15

    three things that I think that uh I would like to highlight and that's important for anyone that's working to

93. 12:24

    secure a system. The first thing is time from vulnerability discovery to patch.

94. 12:31

    If we look at, you know, uh, how long it takes for a company that, you know, has a vulnerability reported and and the

95. 12:40

    time that it's actually patched in code, usually that process takes months. My uh, Cloudflare bounty took three months

96. 12:49

    from the time that I reported it to the time that it got pushed as a PR. It's not even merged yet. Uh, but it's there

97. 12:57

    as a PR. Uh so being able to compress that time delta

98. 13:04

    is super important if you're working with a security and in an ideal world you would have a patch ready for a

99. 13:12

    vulnerability within a day or within a couple of hours that would drastically reduce the risk that you have uh when it

100. 13:22

     comes to to vulnerabilities in code. So that's the first thing uh that that uh

101. 13:29

     one needs to look at the KPI that that one needs to look at. How fast does it take for me to patch a vulnerability?

102. 13:37

     The faster the better.

103. 13:39

     Uh the second thing is how often I run security checks. Um and security checks, you know, there are a bunch of ways of

104. 13:48

     doing it. It could be security checks on your supply chain or your dependencies.

105. 13:53

     is it could be on the code. It could be monitoring your infrastructure. It could be a bunch of different stuff even

106. 14:00

     triggering pen tests with like codeex or cloud code if you have access to their uh cyber program which nobody does but

107. 14:08

     yeah anyways um um so so how often you run these checks, the more often you run

108. 14:15

     them and the the shorter the time delta is from you finding a a vulnerability to patching it. uh the better off you are.

109. 14:25

     Uh and then the last point is how much of your stack is covered. And this is something that we see a lot. People are,

110. 14:33

     you know, very good at security at certain parts of their uh stack but then really bad at other parts of the stack.

111. 14:42

     And as I mentioned earlier, attackers look at the whole stack. So if you don't if you only secure one part of your

112. 14:49

     stack, you're still vulnerable. So uh like like it's more or less like writing tests for your code. The more coverage

113. 14:57

     you have for your test, the better the the more you can trust that uh test testing system, right? Same thing with

114. 15:05

     uh security. The more coverage you have, the better it is. So three things diminish the time from vulnerability to

115. 15:13

     patch do code scanning of different kinds all the time

116. 15:20

     247 if if I'm uh you know if I would do it and we do it we do it 24/7

117. 15:27

     uh and then cover as much as of your stack as possible

118. 15:34

     um so this is an ideal world a picture that I've painted on how uh like an

119. 15:41

     ideal uh security setup for any given company should look like. Um so you have on the left hand side you have your

120. 15:50

     agents which is a uh is is a place where you can um get hacked. Of course agents

121. 15:58

     can read an email. You know I have instinct. How many have instinct? You guys know what it is? It's like a new

122. 16:06

     agent that you can have on SMS that has access to your bank account. It has access to your email, you know, etc.,

123. 16:13

     etc. So, that's a target for attackers, agents, repositories, infrastructure,

124. 16:21

     application, uh, all of these need to get protected.

125. 16:25

     That's that's more or less the whole stack that you have. Um and all of them

126. 16:32

     need to to uh go into uh what I call a security factory more or less. And the

127. 16:38

     security factory is different type of u security workers in a factory that do

128. 16:46

     different stuff depending on what part of your stack uh we're talking about. So pen tests is one thing that you know uh

129. 16:55

     previously a company did maybe once a year and usually you do those pen tests because you have some kind of compliance

130. 17:04

     or regulatory thing that you need to pass. You don't even care about security. It's just to pass that compliance thing so you can go and sell

131. 17:11

     to your customers, right? Usually those pentest suck and everybody knows it. It's it's sloppy pentest more or less.

132. 17:20

     Uh what would be ideal there is to run pentests every week and do good pentests. Pentests that actually have

133. 17:28

     the context of your stack and your application that can find vulnerabilities and that can give your

134. 17:35

     agents on the right hand side you know the the context needed to do the patch.

135. 17:43

     Um infrastructure monitoring is something very interesting again. Um I think that yesterday data dog released a

136. 17:52

     model actually to do exactly this. They released a model that can be ran 24/7 on

137. 18:00

     top of your infrastructure looking at logs doing all of the S sur work that a human would do uh usually and that can

138. 18:09

     escalate more or less things findings or incidents to to either a human or

139. 18:16

     another agent that can then triage patch or do whatever work that you have in your workflow. So that's also very

140. 18:23

     important and I would say that um dependency updates which is something that has been a couple of months ago

141. 18:32

     there were a lot of attacks coming in from dependency updates uh and that usually those attacks are usually some

142. 18:40

     kind of dependency that you have in your system. It could be let's say Nex.js JS.

143. 18:46

     Uh, one of the, uh, employees at Verscell got socially engineered,

144. 18:54

     [clears throat] their credentials got leaked, an attacker updated the Nex.js package to the to a new version with a

145. 19:01

     worm inside it, which basically exposes your your credentials to that attacker

146. 19:07

     and they then it spreads. Um, and so there are ways of securing yourself from

147. 19:14

     that kind of attack. The simplest way is to just not update dependencies at all.

148. 19:20

     But that's uh usually um usually a bad bad thing to do because dependencies

149. 19:28

     have vulnerabilities and these companies patch these vulnerabilities on a weekly basis. So you actually need to update them. But then there are other uh

150. 19:36

     software which I will get to later uh that can do security checks on your dependencies even though um they are new

151. 19:45

     like one day old or two day old so that you can uh merge them into your systems.

152. 19:51

     Uh PR scans is also something that's uh um very valuable especially if you're

153. 19:58

     open source which we are. uh that means that we get contributions from all over the place from people we don't know who

154. 20:06

     they are right so it's very uh important for us and I would say for any company

155. 20:13

     to scan any PR human written or agent written for war vulnerabilities uh and a lot of people use like code

156. 20:22

     rabbit or codeex or something to do regular PR reviews the problem with those are that they are tuned to just do

157. 20:30

     regular PR reviews. They are not tuned to be to think as an attacker thinks.

158. 20:37

     So, uh I think that that's a huge plus uh for anyone that's running a critical system, especially if that is open source uh to run those type of tests.

159. 20:48

     And it's also something that's fairly easy to set up yourself uh if you want to and if you know what you're doing.

160. 20:55

     Um, so all of this needs to run 24/7 and the whole idea behind the security factory

161. 21:05

     is to automate it as much as possible without human intervention. Um, connect your applications, your infrastructure,

162. 21:13

     your repository, your agents to a system.

163. 21:17

     that can run 24/7 and that can alert your agents or your S surres or your

164. 21:25

     humans when something when it finds something of value, right? And this doesn't mean that you should like kick

165. 21:33

     out your security team or your uh compliance team or your SRRES and just run agents. That's not the idea. uh the

166. 21:42

     the power of agents is that you can have many of them and they can run 24/7.

167. 21:48

     Uh so it's an advantage compared to humans because humans cost a lot of money, agents do as well but uh they are

168. 21:56

     harder to maintain humans and they don't run 24/7.

169. 22:01

     Um so having this as a uh complement to your existing security um team stack

170. 22:10

     whatever is I believe a must in the future. And I also believe that if you

171. 22:17

     are going to defend against AI powered attacks you need to actually use AI for the defense as well. uh it will give you

172. 22:26

     leverage even though you are at a disadvantage from the get-go.

173. 22:32

     Uh so here are some uh products that uh are mostly open source that I would

174. 22:40

     recommend you guys to to check out. Sam Grep is like you know u a bastion of security and has been for many many

175. 22:49

     years. They do a bunch of stuff. They look at your dependencies. They do uh pentests. They do a bunch of stuff. So,

176. 22:57

     I definitely recommend uh checking those out uh those guys. Socket.dev is for

177. 23:05

     dependency updates. So, you can just plug in socket to your to your repository and it will check every

178. 23:12

     dependency for fishing attacks, for all of the different type of worms and stuff that's out there. uh so that you can

179. 23:20

     safely bump those dependencies. It's also more or less free. So it it doesn't make any sense not to use it really. Um

180. 23:30

     renovate is more like dependabot you know that does something similar to to socket but does not only focus on

181. 23:38

     security it also focus on you know uh depend dependency updates that are not vulnerabilities vulnerability updates.

182. 23:47

     So that's also open source.

183. 23:50

     Uh, Zap is like a pentesting framework or a pentesting product you could use.

184. 23:55

     Prompt fu is for red teaming agents. So if you're deploying agents to your customer, you want to make sure that they don't, you know, help the customer

185. 24:04

     commit suicide, you should use prompt fu. Uh, which I think is now a day is a part of open AI. If I'm not mistaken,

186. 24:12

     they got acquired. Um I want to mention this company as well

187. 24:18

     uh obliteration AI. Um and I think that most of you guys have noticed that um

188. 24:26

     the more capable the models get the more they refuse to do work that you want them to do as well. Right? So you get

189. 24:33

     basically guardrailed. Uh I get guardrailed all of the time. Thus I don't use anthropic or open AI when I code even because I do security work. So

190. 24:42

     I have to use open source models uh or open weight models. So there are two types of open weight models. There are

191. 24:50

     open weight models that the um labs release like uh Neotron from Nvidia or

192. 24:56

     you know Kim K3 or GLM or whatever you want to use depending on the task. And then there are models where people have

193. 25:04

     actually taken these open weights and removed all of the guardrails from them.

194. 25:09

     So these models are very good at generating data uh that might be uh good to train let's

195. 25:17

     say an agent on to protect your systems because they have no guardrails at all

196. 25:24

     uh or even use as pentesters. So you can use these obliterated models which means basically no guardrails. It could do anything. It could cook drugs for you.

197. 25:33

     we could, you know, uh, help you create a weapon or even secure your system depending on what you want to use it

198. 25:40

     for. This company has basically a, uh, inference API that you can use, but there's also a bunch of them open source

199. 25:49

     on hugging face if you rather would use them. Uh, but we use these models all of the time in the in the these types of

200. 25:57

     models all of the time in the security work that we do. uh and also in my regular daily like working with uh

201. 26:05

     Copilot or Codeex or whatever you guys use and then of course super agent that does all of this in a package more or

202. 26:12

     less. So that's a plug from me. Um thanks

203. 26:20

     [applause]