We’ve been mired in endless discussions on how Adversaries and Defenders are (or could be) taking advantage of AI. Does one side have the upper hand? Or is this just a continuation of the endless “cat and mouse” game adversaries and security professionals play?
Check out this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Geoff Belknap (@geoffbelknap). Joining us is our sponsored guest, Rob Allen, chief product officer, ThreatLocker.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, ThreatLocker
Full Transcript
Intro
0:00.000
[David Spark] We’ve been mired in an endless discussion on how adversaries and defenders are, or could be, taking advantage of AI. Does one side have the upper hand, or is this just a continuation of the endless cat-and-mouse game adversaries and security professionals play?
[Voiceover] You’re listening to Defense in Depth.
[David Spark] Welcome to Defense in Depth. My name is David Spark, I’m the producer of the CISO Series. And joining me as my co-host, I’ve so thoroughly enjoyed having him on board, and I’m sure he enjoys it too, and if he says otherwise, don’t believe him. It’s Geoff Belknap. Geoff, say hello to the audience.
[Geoff Belknap] Hello, David, and hello, audience. Thanks for letting me be here.
[David Spark] We’re thrilled you’re here. By the way, our sponsor for today’s episode, a spectacular sponsor of the CISO Series, thoroughly supporting every effort across all of our programming. It is ThreatLocker, zero trust endpoint protection platform. And in fact, they’re responsible for our guest today, who has been very entertaining just before we started this recording, so I’m counting on him to deliver, but I’ll introduce him in just a moment.
First, Geoff, I want to bring up today’s topic. Historically, we’ve seen all new technologies be both a benefit to drive business, but also a tool that can be used against us. Now this is becoming ever more prevalent with the meteoric rise of AI. But what seems different here is we are seeing new capabilities daily, and we don’t see an end in sight.
And like most tech, it both excites us and scares us. But now these extremes are being cranked way up. What’s also unique is this technology is clearly a tool being used by both adversaries and defenders. So, on LinkedIn, Geoff, you asked the security community what are they excited about and fearful of, and what they were seeing today.
Just want to get your first reaction to those takes. What do you think?
[Geoff Belknap] I think my first reaction was, thankfully, people still see that there is value to AI outside of just it being used against them in an attack mechanism, and so I’m optimistic. But I think the other side of this is everyone clearly sees what’s coming around the corner, and we’re all talking about how to get ready for it.
[David Spark] Yes. And there were a lot of really, really interesting takes on this, and this is going to be an awesome discussion, and I have a very, very opinionated person to join us for this discussion. Thrilled to have him on. We’ve had him on before. It’s our sponsor guest, the chief product officer from ThreatLocker, none other than Rob Allen.
Rob, thank you so much for joining us.
[Rob Allen] You’re very welcome. I’m not sure whether to be pleased or displeased about being described as opinionated, but I’ll take it as a positive. So, thank you, David.
[David Spark] Take it very much as a positive.
We’ve seen this one before
2:50.487
[David Spark] David Houchin of CohnReznick said, “AI helps us act out decisions with alacrity, and to that end, the risks are the same but with greater speed. Data leakage now happens at massive scale within the blink of an eye.” And Vaughan Shanks from Cydarm Technologies said, “We have been using machine learning for many years now to classify threats automatically with mixed results.
LLMs take the complexity up a level, providing more capability, but greater risk. LLMs don’t think and should not be trusted for important decisions.” So, again, we’re just touching the surface here of people’s concerns, if you will, Geoff. I think really kind of like what I said in the opening, it’s what we’ve seen before, but the extremes on both ends are pretty darn severe.
Do you feel the same?
[Geoff Belknap] The extreme potential for abuse here is ever present, and I think not just potential for abuse, but Vaughan makes a really great point here. People are forgetting, because interacting with a large language model can be very engaging and sort of make you think you’re talking to another human or a thinking intelligence, but it’s not thinking, right?
This is not something that is putting a lot of thought into the outcomes or to the decision that it’s making. It’s doing analysis and it’s making some decisions about what to present in front of you, but you have to do the thinking, you have to think about the result that you’re getting here, and you have to decide how to trust what’s presented to you, even though it can be very humanized and very comforting.
[David Spark] Can I just shoot a little devil’s advocate back at you? Because I’ve heard this line is it’s not thinking, but when you come right back to it, it’s kind of the way we think in that we take the knowledge that we have, what we’ve absorbed, use that to make a judgment call on whatever we’re seeing now.
Yes, a human has better reasoning capabilities, but at the same time, the LLM is absorbing far more knowledge than we are able to do. So, can it make a better decision than us?
[Geoff Belknap] I think depending on how it’s trained, and it’s not always clear when you’re engaging with that LLM, what it’s been trained on and what its data sources and reference points are and how it’s weighting the information that it’s giving you. But yeah, depending on how it’s been trained, what it’s been trained on, it absolutely could give you insights that you otherwise would not have come to.
Should you let it make every decision for you in your trade or in your day? I probably wouldn’t go there yet.
[David Spark] All right. I’m throwing this to you, Rob. Your first take on sort of the concerns of the extremes – extreme good and extreme bad.
[Rob Allen] I think there is, I suppose, one of my pet peeves with the cybersecurity industry as a whole is the AI, obsession with AI, as in AI is going to save you. I think one of the things that we try to educate people about is the fact that it’s just as likely to be used against you. I mean, there’s millions of different reasons.
Everybody’s seen the improvement in spam emails over the last while. It used to be that spam emails were very easily spotted because of spelling and grammar mistakes. That’s not such a big thing anymore, and I think we largely have the likes of ChatGPT to thank for that.
Another example is the fact that you can technically trick or use most of the LLMs to fundamentally give you malware. I mean, we’ve got examples of reverse shells provided to us by ChatGPT, beautifully formatted and amazingly functional code. So, it’s pretty much reduced the barrier of entry for malicious actors.
I mean, there was a time when there was a relatively small number of people worldwide who had the necessary skills to create malware. Now, pretty much anyone can do it by asking an LLM the right question in the right way. So, the dangers are very much real, and they are present.
What problem is this solving?
7:00.315
[David Spark] Ahsan Mir of Rapticore said, “Response time is critical, and AI is shifting the advantage toward defenders. Traditionally, adversaries had the upper hand in timing, able to choose the moment to strike after gathering ample information. Meanwhile, defenders often needed more context, skill, and time struggling to process everything rapidly.
With AI, however, defenders gain a substantial edge. AI could even act as an always-on purple team, supporting the blue team in proactive threat detection and response. I’m optimistic that this is a net gain for defenders.”
And Sean Cassidy, CISO for Asana, said, “It’s a big-time friend and I think it gives defenders the upper hand. With AI, we can now triage both alerts and vulnerabilities faster and more accurately. AI also helps us answer security questionnaires, reducing the cost of responding to them, and we can reply a scalable way to questions asked of our security team by having an AI take a first pass.” So, both Ahsan and Sean are really bullish on AI, essentially, helping them just be better security professionals.
What do you think, Rob?
[Rob Allen] I admire their optimism. I share it to some extent. It is a useful tool, but I mean, fundamentally, it is a tool. As I said, it’s a tool that can be used for good and it is a tool that can be used for bad. I still think that even with the descriptions that they gave in terms of how they think it can help them, fundamentally it still comes down to something or someone making a decision about good or bad behavior.
So, looking at behavior, is this behavior good or is this behavior bad? Now, obviously, in some cases, it’s done by a human being. In some cases, it’s done by AI. AI is arguably, I’m not going to say it’s just as likely to make an incorrect decision about good or bad, but what I can pretty much guarantee you is that AI is going to make a decision quicker, whether it be correct or incorrect, so it may just be helping you make incorrect decisions faster.
[David Spark] That is a fear, but I must say that the percentage of incorrect decisions being made is being reduced over time as AI gets smarter. Would you agree or disagree with that, Rob?
[Rob Allen] I would not disagree with that, but any percentage that is greater than zero is a problem.
[David Spark] That’s a good point. Geoff is nodding his head sort of in a quizzical way to Rob’s last comment. Geoff, your take on this? And again, I am also impressed with Ahsan’s and Sean’s optimism here.
[Geoff Belknap] I think the key here is not necessarily to think about this as a decision-making tool, although you definitely can, but where the advantage for AI for defenders right now is is leveraging it for scale and volume, right? So, when you are working on, let’s say, an active breach and one of the things you want to look at is, “Okay, well, I have all this log data, and I need to figure out what the attacker took or what they looked at or where they might be going next.” That’s not a yes-or-no decision.
That’s not an, “Am I being attacked or not?” It’s like I’m being attacked, but now I need to figure out how I’m going to organize my defense because I have limited resources and the attacker’s ahead of me.
If I want to close that gap and be able to catch up and get ahead of the attacker, one significant advantage is be able to use AI to understand what you’re looking at, where they’re going, where they’ve been, what they’ve taken, what passwords and secrets you need to reset, what has been impacted by that.
That’s an incredibly difficult thing for individual humans to do, and it’s something that AI can really take a chunk out of. So, I think when we think about how do we compress the scale or volume that are sometimes involved in responding to mass attacks, I think AI can give us an enormous lift. And I think as long as we think about AI as giving us a benefit and helping the humans attack a problem instead of just replacing the like “find bad guy” button that we all hope to find someplace, I think then we can really see the advantage.
Sponsor – Threatlocker
11:09.026
[David Spark] Who’s our sponsor this week? Well, it’s none other than the fantastic ThreatLocker. So, let me ask you a question. Do zero-day exploits and supply chain attacks keep you up at night? Well, worry no more. You can actually harden your security with ThreatLocker. So, imagine taking a proactive – this is interesting – a proactive deny-by-default approach to cybersecurity.
Oh, you’re thinking that’s too limiting. Well, hold on. It is blocking every action, process, and user unless specifically authorized by your team. You would be surprised how permissive your environment could be if you’re not doing something like that.
So, ThreatLocker helps you do this and provides a full audit of every action, allowed or blocked, for risk management and compliance. Onboarding and operation is fully supported by their US-based support team. Stop the exploitation of trusted applications within your organization to keep you running efficiently and secure, protected from ransomware.
So, worldwide, companies like JetBlue trust ThreatLocker to secure their data and keep their business operations flying high. To learn more about how ThreatLocker can mitigate unknown threats and ensure compliance for your organization, go to their website, visit threatlocker.com.
Where does the solution fall short?
12:35.492
[David Spark] Jeremiah O., That Cloud Group said, “AI is definitely more in favor of a malicious actor.” All right, so the opposite side of the coin of what we discussed in the last segment. “The downsides to AI like hallucinations, bias, bad data, etc., all it does is mean they are a little bit less likely on their attack working.
But on an AI unintended defense, having a hallucination means missed event.” Ah, what you were referring to, Rob. “So, one missed event has the potential to do serious damage.” As you were saying, anything over zero is a problem.
But let me also throw out here, Felix Matenaar of Asana said, “Attackers have the upper hand because AI makes it much easier to exhaust the attacker search space, therefore significantly increasing the level of security fundamentals needed to be present in a system in order for it to remain uncompromised on the internet.” So, that is a really interesting take, the very last quote here, Geoff.
What Felix said is you really need your entire fundamentals locked down, and we talk about this all the time on this show, but if there was anything screaming, “Get your fundamentals in place!” Felix is saying, “Oh, AI is telling you to do that.” Yes?
[Geoff Belknap] Absolutely. This is the thing I always beat the drum on. If you were relying on obscurity or the attacker just not noticing or not having the time or wherewithal to sort of address things that were more complicated in your environment, this is the thing that AI sort of ruins for you.
You now have to be good at the fundamentals because low-hanging fruit at scale is much, much easier for attackers now. I think to Rob’s point, where you might have had to spend time writing lures and sort of thinking of things, AI can do all that for you, and agentic AI now makes it really easy. You can add bad guys to your bad guy team very easily by scaling out with AI.
So, you have to be even more on your game as a defender than you ever had to be before, and you need to be thinking about how you’re going to leverage AI to make your job easier to defend against the bad guys that are also using AI to make your life harder.
[David Spark] Okay, you bring up essentially the entire theme of this episode of who’s got the edge here. Definitely in this segment, both Jeremiah and Felix said the attackers have the edge. And they referenced exactly what you were talking about, Rob, in the previous segment, but I got to just double down really on both of what they said.
It only takes one to make your life miserable, and two, what Felix said is they’re going to exhaust everything. Like every avenue that could be attacked, it’s much easier for them to do that. Does that scare the crap out of you, Rob?
[Rob Allen] Does it scare the crap out of me? No, I wouldn’t go so far as to say it does because again, we are very much advocates of minimizing the attack surface, of making your environment as unfriendly and as difficult as possible.
[David Spark] What I just discussed about when I talked about the ad read there, it’s like you have a very much a deny-by-default attitude. So, what you’re doing is that attack surface becomes extremely limited at that point. Go on.
[Rob Allen] Absolutely, absolutely. Look, we’ve discussed in depth before the merits of the zero-trust approach. As I said, default deny, as we see it, is a very fundamental part of that. As I said, it’s one mistake is all it takes, one port open to the internet, one bad decision. And I completely agree with Geoff’s points about there being value to AI and the fact that it can be used in certain circumstances to accelerate, I suppose, defenses and responses.
But I’m very much of the opinion to try and stop there being things that require responding to is a better approach than waiting for something to happen that you then need to respond to, if that makes sense.
[David Spark] Let me ask this. We’ve seen in the history of security that things like regulations, like GDPR, all of a sudden move the needle significantly for privacy in that case. Could AI be that thing? Because we beat the drum endlessly on this show about get your fundamentals straight, get your fundamentals straight.
Could AI be that trigger that finally gets more companies to get their fundamentals straight? I want both your takes. I’ll start with you, Geoff.
[Geoff Belknap] Will it be the trigger? No, probably not. I think there’s regulatory motion coming, especially the CRA coming out of Europe. I think that will drive a lot of people to either get their fundamentals straight or, frankly, get out of the business. I think, though, AI is going to increase the pain when you don’t.
Today, I think to Rob’s point, absolutely the right strategy. You need to make sure the surface area you have that is risk, that is exposed to the internet, is as locked down as possible at all times, and that’s time consuming and challenging. But today, there are a lot of scenarios where if you don’t have a lockdown and you have a breach, sometimes that’s not so bad.
I’m not a big fan, and that’s certainly not the way I approach it, but there’s not always consequences like there should be from the market or from regulators.
That’s changing. Right? People have really, really started to notice that it matters to them, that it matters to global economies, supply chains, all kinds of things. And so I think we’re finally pivoting out of a time where tech companies can take a laissez-faire approach to this thing, and AI is going to accelerate the pain that you are going to feel when you make that mistake or when you just don’t invest in an area that an attacker is incented to take action against you.
[David Spark] All right. I’ll let you have the closing comment here, Rob. What do you think? Is AI going to push better fundamentals or not?
[Rob Allen] AI in itself, I don’t think is going to push better fundamentals. As Geoff said, and it’s a really good point, is it’s more likely to be legislation. It’s more likely to be companies and individuals realizing that there is a big stick that is coming for them if they don’t do things correctly.
I mean, Geoff did mention about reducing the attack surface being a case of minimizing what’s exposed to the internet, but realistically, reducing the attack surface means minimizing what’s available internally for bad actors to use as well. It comes back to this idea of assume breach. So, assume that they’re in already, they’re on your network right now.
Having defenses against the internet is not going to help you in that scenario, that situation. So, again, it’s about minimizing the attack surface everywhere there is a surface. But as I said, AI is just an accelerant. It’s an accelerant for good and it’s an accelerant for bad. But which wins out in the end?
Who’s to say? And I suppose time will tell.
What’s the issue here?
19:29.042
[David Spark] Conner Biolsi of Lewis County said, “I am most worried about truth and trust in our everyday experience in reality. We are rapidly heading towards a reality that will be shaped by context – text, picture, video, audio, etc. – that has been generated artificially. How are we as humans going to adjust to this?
There’s an inevitable degradation of trust coming where we won’t necessarily know the true source of content and whether it is truth.”
Immanuel Chavoya of RiskHorizon.ai said, “The real concern I have lies in the unchecked visibility foundation model companies could gain akin to nation-state intelligence over populations and economies.” So, that’s pretty big here. “We’re just scratching the surface of what this AI footprint means for operational security, privacy, and identity.
Twenty years from now, we may see this era more clearly, but the bigger danger might be human over-reliance on AI eroding our capacity for critical thinking and disconnecting us from cultural wisdom built over generations.” Ah, so this refers to what you were talking about, Rob, and that we do need humans, and could we run into a situation where we become far too dependent on AI?
Like, “Let’s just let AI write it. Let’s just let AI do it.” What do you think?
[Rob Allen] [Laughter] I do have to say, I love the quote, “Eroding our capacity for critical thinking.” I mean, you could argue, and I often describe Google as a tool that has helped me to not know things for the last 15 years. Because I don’t need to know things. All I need to know is how to ask on Google.
So, you could argue that AI is broadly speaking similar. It’s just going to present it in a more nicer fashion.
[David Spark] I want to cut in with just a quote from an old video editor of mine, Diana Brody, who used to say to her 16-year-old nephew at the time said, “Do you know, back in my day when I was your age, when we didn’t know something, we didn’t know it.”
[Laughter]
[Rob Allen] It’s absolutely true. It’s absolutely true. But yeah, as I said, that’s my own sort of opinion, and that is it’s like Google, but more. I mean, I’m not going to go so far as to say AI is just Google on steroids. Let me rephrase that. Not AI in general. I’d say the LLMs are basically Google on steroids because they’re ingesting information and they’re just giving it back to you in a more presentable format.
The other one that’s actually quite interesting because it reminds me of probably my favorite little project that I undertook this year, which was I made a AI voice version of Danny Jenkins, our CEO.
[David Spark] Oh, yeah.
[Rob Allen] You’ve obviously spoken to Danny, Danny has a very distinctive English accent, and I recorded him without his knowledge. I was standing behind him just with my phone and I recorded him for about three or four minutes, and then I put it into a AI voice generator. So, I made an AI Danny, and it was phenomenally good.
He will say that you couldn’t notice the difference, and there were certain words that it would say in a particular way that didn’t match what he would say.
So, for example, it said anti-virus instead of antivirus. So, there were certain things that were tells, but in general, I mean, I got it to say that Rob is amazing and he’s always right and he’s better at golf than anyone in the company and things like this, and it sounded like Danny. So, it’s not a big jump from having an AI voice Danny to having the AI voice Danny call people in your company and say, “Hey, I need a password reset on this account.
Can you give it to me?” and things like that. So, it is absolutely possible and quite a scary thought to think that there may be AI versions of us out there talking to people without our knowledge.
[David Spark] Yeah. It doesn’t take much, and also I’ve heard the audio versions of this. It’s astonishingly good, and I could be easily fooled by. There’s endless amount of audio of me out there that could easily fool me, for that matter.
[Rob Allen] I think once we get off this call, I am going to make an AI you.
[David Spark] You don’t need to.
[Rob Allen] I kind of want to, David. I kind of want to.
[David Spark] Geoff, I’m going to quote this again, I’ve quoted it multiple times. I talked to a CISO at a major pharmaceutical. Again, they have hundreds of thousands of employees. They were seeing – I know this is a small number given how many employees – but they were seeing 10 deep fakes via WhatsApp every month.
And I was like, “Oh, that’s only going to increase. That’s only going to get worse.” That’s a pretty strong fear right there. That’s a new paradigm we haven’t had to deal with.
[Geoff Belknap] Yeah. I think we, until recently, only had to worry about attackers attacking computers, largely. In the last, let’s call it 12 to 18 months, we’ve certainly seen a couple of specific threat actor groups target individuals by saying via text or voicemail, like, “Hey, I need your password,” or even just threatening people, “You’re going to give me this information or something bad’s going to happen.” Well, now AI and all the generative tools that are available have progressed to the point where you can just completely trick people into thinking they’re on a video call with somebody, you’re interviewing somebody.
You might be able to convince them to hire you or a bad actor to the company, and you’re going to just give them a log-in and password as part of that hiring process.
I think it’s terrifying to me, and the thing that keeps me up at night is thinking that attackers are getting the tools to completely bypass all the cyber defenses and the technology and the infrastructure and the sort of surface area that we build for detection there by just tricking the human into either believing something that’s against their best interest or taking an action that is against the best interest of their organization.
Or nation-states now don’t necessarily need to cause chaos and drop banking systems, they can just convince a third of the population to believe something that’s wrong or something that’s in their political best interest for the adversary. So, I think that’s the kind of thing that’s even scarier than finding 0-day via LLMs.
[David Spark] It also gets to the whole issue of, for our entire existence, we’ve relied on our eyes and our ears to work and accept what we see and hear, but that’s the big question is, that may no longer be a verification tool at all.
[Geoff Belknap] Well, I think it certainly makes it more difficult because while most people have been taught to be critical thinkers and to question at least what they’re seeing and hearing and reading, we certainly haven’t upgraded our ability to critically think or to judge what we’re seeing or reading now that AI exists.
And I think this is certainly a whole new area that I as a cybersecurity professional never thought I would have to get into. Like, how do I train people to be even more critical of what they’re reading if the email looks perfect and it was even sent by an employee, but maybe it’s just complete bunkus?
Like, how do I teach people to think even more critically than they might have before?
[David Spark] It’s a new skill you’re going to have to have as a security leader. Very good point. All right.
Closing
26:28.860
[David Spark] That’s a good point to conclude, and now I’m going to throw this to you, Rob. We’ve come to the point in the show where I ask you to pick your favorite quote and tell me why it’s your favorite quote.
[Rob Allen] I think I may have already mentioned it, which is the over-reliance on AI eroding our capacity for critical thinking.
[David Spark] That would be Immanuel Chavoya. You like that. Why do you like that quote so much?
[Rob Allen] As I said, it’s basically a modern-day version of my Google helping me not know things for the last 15 years. How long has Google been around? Is it 15 years?
[David Spark] They’ve been around for a good long time.
[Rob Allen] Yeah, but as I said, it’s basically just a modern-day version of that.
[David Spark] All right, Geoff, your favorite quote and why.
[Geoff Belknap] I love Immanuel’s quote, but since I can’t compete with Rob, I’m going to go with Sean Cassidy, the CISO at Asana, who points out that AI can really help us triage alerts and vulnerabilities faster and more accurately, and now, and this is my favorite, we can reply in a more scalable way to questions asked of our security team by having AI help us kind of take a first pass and assist the human that’s doing the investigation.
[David Spark] Well, you heard it here. You heard people who are bullish and fearful, and we question what the next level of AI is. This was a great, great episode. Thank you so much, Rob and Geoff. I want to thank your company, Rob, ThreatLocker, zero trust endpoint protection platform. Go to their website, threatlocker.com.
In fact, they will help you greatly reduce, and I mean dramatically, it’s kind of astonishing what they do, your attack surface for an AI type environment. Rob, anything last to say? I know you guys are hiring like mad. Your company’s growing like crazy. Any other last thoughts?
[Rob Allen] Anybody who’s interested, we have an event here in Florida, 19th to the 22nd of February.
[David Spark] I will be there doing a show with you on stage.
[Rob Allen] I think you mentioned that to me, and I’d completely forgotten, but yay! We’re doing a show together. No, that will be fun. That will be fun. So, yeah, you’re going to be here. Realistically, I think everybody else should be here as well. February in Florida is very much the time to be in Florida.
It is cold everywhere else. It’s nice and warm here, and it is a really, really great learning event, hands-on, just educational. People love it when they come to it. So, ztw.com if anyone wants to check it out, and we would love to have as many people as possible.
[David Spark] By the way, if you would like to see how Rob abuses me in person, you will, I’m sure, be able to get a chance to see that at Zero Trust World. [Laughter]
[Rob Allen] I resemble that comment, David.
[David Spark] Thank you very much, Rob. Thank you very much, Geoff. And thank you to our audience. We greatly appreciate your contributions and for listening to Defense in Depth.
[Voiceover] We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site CISOseries.com where you’ll also see plenty of ways to participate, including recording a question or a comment for the show.
If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com. Thank you for listening to Defense in Depth.