Large language models can do a lot of things. But they are hardly infallible. We continue to see stories emerge of organizations relying on the output of LLMs without any human review, resulting in consequences ranging from humorous to disastrous. So, how do we build in human-in-the-loop verification when the scale of LLM output can be so daunting?
This week’s episode is hosted by me, David Spark, producer of CISO Series and Andy Ellis (@csoandy), principal of Duha. Joining us is our sponsored guest, Kevin Tian, co-founder and CEO, Doppel.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, Doppel
Full Transcript
Intro
0:00.000
[Voiceover] What I love about cybersecurity. Go!
[Kevin Tian] What I love about cybersecurity is that it seems like all of us vendors are trying to solve some of the same technical problems around false positives versus false negatives, precision versus recall. How do we make sure that we’re showing teams the right information without making sure we’re missing anything as well? As a software engineer, that’s always the fun part about cybersecurity here.
[Voiceover] It’s time to begin the CISO Series Podcast.
[David Spark] Welcome to the CISO Series Podcast. My name is David Spark, I’m the producer of the CISO Series. And joining me for this very episode, it’s Andy Ellis. Now, you may know him as the editor of How to CISO. If you’re not following it, do that. And he’s also the author of 1% Leadership, a book that I’ve read that I strongly recommend. Andy, say hello to the audience.
[Andy Ellis] Good afternoon, folks, or depending on when you are in the world, good morning, good evening, or good night.
[David Spark] Now, Andy is saying that in English because he was too lazy to find another language to translate it to. Am I correct, Andy?
[Andy Ellis] That is exactly true. That and I recognized I hadn’t done English in quite a while, so I’m going to do English for a little bit.
[David Spark] You did English the last two episodes! [Laughter]
[Andy Ellis] The last two episodes? I’m going to do English for a little while as I think about what my call sign should be, if this is the right thing or not.
[David Spark] Yeah, well, he’s still working that out. By the way, we’re available at CISOseries.com, where we have four other programs on our network, which you are welcome to listen to. We invite you to listen to our other programming. Our sponsor for today’s episode is Doppel. Defend what’s real, disrupt what’s not. More about that later in the show. But first, Andy, we were showing off mousepads just moments ago.
[Andy Ellis] Yep. And that’s not a euphemism, folks.
[David Spark] Yes, it’s not. [Laughter] And the mousepad is no longer a needed item for any stretch, hasn’t been needed for quite some time or necessarily wasn’t ever really needed.
[Andy Ellis] It was needed to try to keep the dust out of the ball of your mouse.
[David Spark] Yeah, but the mousepad collected plenty of dust anyways. But years and years ago, I worked at an ad agency. I was an IT support guy, and I worked with another guy named Manny, who was an IT support guy. And one of the women complained that she believed her mousepad was broken. That was the request, [Laughter] a broken mousepad. And honest to God, he grabbed the mousepad, beat it against the ground, and handed it back to her. [Laughter]
[Andy Ellis] Well, it’s better than when people complain that the drink tray on their tower doesn’t work correctly.
[David Spark] Oh, yes, yes. It’s a classic, the drink tray. For those of you who haven’t purchased a computer or got their first computer recently, they used to come with DVD or CD drives.
[Andy Ellis] Yep.
[David Spark] So, besides that drink tray, have you ever had a request that was so far afield from reality like that one?
[Andy Ellis] No, actually, I have one that’s actually funnier in the other direction. We actually failed an audit for our sensitive compartmented information facility because the auditor came through and we had both a classified and an unclassified and actually ended top secret network as well. So, we had all three layers. And part of the security control that you do is the networks are all different media. So, the top-secret network was, actually, it was on the other side of the building. They didn’t even complain about that. The secret network was fiber, and the unclassified network was Cat 4, I think, at the time. And the day before they’d come in, the conference room that had a drop for each of them on opposite ends of the front of the room, the cleaners had come in and they’d rolled over the whiteboard that was in the room, and they’d hung the cables on each end of the whiteboard to vacuum. So, the cables didn’t get caught. One fiber, one Cat 4. The auditor complained and failed us because we had connected an unclassified and a classified network together.
[David Spark] There you go.
[Andy Ellis] What are you going to do?
[David Spark] I’d actually like to hear from our audience, too, if they’ve ever had a tech request that was so far afield of reality, like someone should have known better.
[Andy Ellis] Right. And then it’s what do you do to mitigate it? Like that or some of the fun things.
[David Spark] Well, that’s what I found was very funny, is that my colleague, the way he “mitigated” was beating the mousepad on the ground, [Laughter] which all fixed now. Let’s get our show on the road. Thrilled to have our guest on, with our sponsor, that would be Doppel. In fact, the co-founder and CEO of Doppel, our sponsor guest, it’s none other than Kevin Tian. Kevin, thank you so much for joining us.
[Kevin Tian] Thank you so much for having me.
What about this AI security challenge?
4:57.150
[David Spark] The rise of generative AI has created “ad tech fraud on steroids.” AI content farms are now stealing publishers’ ads.txt files to hijack ad revenue, mass producing fake news sites that mimic legitimate publishers to siphon advertising dollars. As noted, all this in a recent Forbes article by Ashish Bhardwaj of Google. Now, we’re seeing sophisticated deepfake celebrity endorsements, AI-generated articles that perfectly mimic legitimate content, and entire networks of fake websites created at unprecedented scale. The tools being deployed to combat the fraud? Well, also AI powered. Now, this raises a fundamental question. Is the AI-powered fraud detection arms race simply the latest chapter in the seemingly eternal cat-and-mouse game in cybersecurity? Andy, are we looking at a new battlefield or just the same old techniques with shinier tools?
[Andy Ellis] So, I think the answer is yes to both of them. Like the idea that somebody’s going to use shiny tools to do a new thing, always go for that. But I think the ad fraud is a fascinating place to take a look at. It’s a battlefield people don’t often consider at this scale. Historically, a lot of the fraud has focused on like click-through rates, creating doppelganger websites. It was less about stealing ad revenue and usually had different sorts of attacks, watering hole, trying to get people’s credit cards, etc. So, I think it’s combining these two things. And I think it’s really fascinating, but I also wonder how much of this is going to be less of an issue as more and more of the search engines are not even showing web pages anymore. Like you search for a thing and you’re now getting an AI-generated summary. They’re trying really hard not to send clicks to websites. And so, I do really wonder how much ad revenue will really get stolen from just cloning somebody’s website in the future. As somebody who runs a website, it’s sort of distressing that people are going to clone my content and make money off it.
[David Spark] Yeah, that’s actually a really good point. The summaries at the top where people are often typing in just to get answers, if I can get the answer, not click anything, well.
[Andy Ellis] Right.
[David Spark] And I’d be interested to know if site visits are now globally going down.
[Andy Ellis] So, talking with CMOs within companies that I’ve worked with in the past, I will tell you that more and more of them are looking at they’re not getting as much search traffic as they used to because it’s really hard for people to click through off of search engine pages.
[David Spark] But what’s very interesting in those AI summaries is once you start clicking the links, you see where they pull all that content.
[Andy Ellis] Yep. But you just have to chase it down.
[David Spark] Yeah. Yeah. All right. I’m throwing this one to you, Kevin. This seems like an endless cat-and-mouse game. What makes this one different?
[Kevin Tian] It’s a good question. I mean, I think what makes it different is this is really a new paradigm in terms of technology, right? Like we’re talking about intelligence, right? That’s typically we always think about it as human intelligence, things like that. But we’re really seeing the precipice of machine intelligence and machines being able to operate autonomously without pre-programming, without specific training or anything like that. But you’re now getting technology that can work out of the box to reason through things and solve things. I know that’s a bit of a controversial statement. Are LLMs really reasoning or not? But in terms of the output and the outcomes that we’re seeing, right, from what these agents can do. And I see Andy shaking his head there. But in terms of like, well, what does it mean for us, right? In terms of the technology that we build and what we think about as vendors, it does mean it’s a significantly different ballgame. And so, that’s why I think it’s not just a standard cat-and-mouse thing, but we’re really seeing a revolution across technology, not just in security either. Right? But really across your software stack, your SaaS stack, things like that.
How is AI going to solve this problem?
9:03.177
[David Spark] “If we don’t rethink our role now from blockers to enablers, from rule makers to copilots, we risk becoming the bottleneck in a machine speed world.” Now, that’s Rinki Sethi, CSO over at Upwind Security, warning of agentic AI. It’s coming fast. CISOs need to be ready for when AI tools don’t just support our mission, they act on our behalf. That’s an enormous leap. Many brands have suffered the brand-damaging consequences of letting GenAI publish on their behalf. Agentic AI has already begun. I’ll start with you, Kevin. How are you swallowing that “acting on our behalf” pill? I mean, you have to balance a challenge without falling behind. I mean, I know there’s a desire to do this, but at some point, that seems a little scary, doesn’t it?
[Kevin Tian] It’s funny because as a startup, right, founded by two engineers who are super passionate about AI and technology, it’s less so fear, more so excitement.
[David Spark] Right, right. I’m with you on this. I’m totally with you on it.
[Kevin Tian] Right.
[David Spark] But there’s got to be a balance here. I mean, as bullish and excited as you are, do you see that there needs to be a balance?
[Kevin Tian] Well, I do think, for example, in terms of balance, right, there are new risks that get brought in, right? Like for example, if we’re using this AI coding assistant tool, if we’re using this five-coding tool, etc. Of course, we want to make sure that it’s not accidentally leaking our source code out to other people using the tool and not accidentally indexing our GitHub repos and enabling other people to now suddenly get access to our source code. So, from that sense, absolutely, you got to be thinking about what does it mean from a risk perspective and what are the implications of it, for sure.
[David Spark] All right, Andy.
[Andy Ellis] I think the important thing when people talk about agentic AI is to recognize that we’re just talking about automation, where you’ve handed the API to call automation to some LLM, and you probably had the LLM also write the automation at some point in the past. And that’s like the really key structure, which is, do you actually know what you want your automation to do? And if the answer is no, you’re going to have a problem whether you let an AI write it or you let an engineer write it.
Like, no offense to the engineers in the room, but engineers are often really bad at automation because if it’s not their process, then they don’t understand the process. Like, engineers are great at software automation because they’re automating their own job. It’s when they go automate somebody else’s job. They’re like, “Oh, I can automate our content management system and auto publish things.” And then some editor is like, “No, no, no, no, just because you wrote it doesn’t mean it’s ready to be published,” because it has to get edited. It has to be hyperlinked. It has to go part of a campaign. Like, there’s a whole bunch of work here. The LLM will be better than an engineer because it’s a generalist, right? It sort of knows how to do a lot of different things, but you’re still going to suffer the same problem of you handed an API to get work done to somebody who didn’t actually understand the work and AIs never actually understand the work.
[David Spark] But the AI can be trained to take actions based on a rule set that you give it. The question is rule sets change. This is a very simplistic way of looking at it. And it’s exciting to have it do the work for you, per what Kevin said. I’m all on board, especially engineers. Very, very exciting. But at the same point, there’s never not a time of outliers, right?
[Andy Ellis] Right. And so, I think that if you’re going to be deploying something agentic, the way that you should probably be thinking about this is at what point is it doing a thing that’s irreversible, right? If I’ve got an agentic platform that takes in an alert and has a bunch of agents to go collect information and do diagnostics, that it’s not actually changing the environment, but it’s collecting data and now says, “Hey, by the way, here’s what this alert really means. Do you want to go fix this? Press here to do something.” Like, that’s fine. The problem is when it goes and does the research and then presses a button for you. On the bright side, for most of our listeners who are security practitioners, note that you’ve had that button for a long time, and most companies won’t let you, the human, push the button because we don’t trust the tools already. And the engineers who actually own the systems never want to have security people affect their systems directly. So, I don’t think on the security side, we’re likely to make this problem worse. But the rest of the business might be in for some eye-opening surprises when bad things happen.
[David Spark] So, Kevin, I also want to throw out like getting it right 99% of the time really still stinks, sadly.
[Kevin Tian] Right.
[David Spark] In a situation like this. Do you agree or disagree?
[Kevin Tian] I mean, I agree that 1% of the time you don’t get it right. Like sometimes that 1% of the time, they’re going really wrong, right? [Laughter] So.
[David Spark] Right. No, but like that means 1 out of 100 actions is not what you want it to be doing.
[Kevin Tian] Right.
[Andy Ellis] Right.
[Kevin Tian] Right.
[Andy Ellis] And people think that 1 out of 100 is small but think about how many actions that AI is going to take on your behalf every single day. It’s not going to number in the hundreds. It’s going to number in the thousands or millions. So, 1 in 100 is really bad.
[David Spark] Yeah, that’s what I was saying.
[Kevin Tian] Yeah. I think that’s why the frameworks are so key in how you deploy AI. Exactly what Andy said, right? If AI is just helping you make the decision to click the button, that’s a little safer than AI can click the button. Or how do we just make sure it can’t click a certain set of buttons? That’s certainly super critical.
Sponsor – Doppel
14:42.116
[David Spark] Before I go on any further, I do want to tell you about our spectacular sponsor, and that is Doppel. Now, Doppel, if you don’t know, is the first social engineering defense platform purpose-built to dismantle impersonation threats before they cause harm. Now, while legacy tools focus on detection and alerting, which is good in its part, Doppel goes further using AI and infrastructure correlation to link phishing emails, fake domains, deepfakes, and impersonation campaigns across channels. So, from executive protection to brand impersonation takedowns, Doppel doesn’t just flag threats, it disrupts them at the source. Every attack fuels their shared threat grid, giving every customer the benefit of collective intelligence. The result? You got faster disruption, stronger resilience, and fewer opportunities for adversaries to profit. Doppel makes digital deception unprofitable – and this is kind of the model for a lot of cybersecurity professionals – makes it unprofitable, protecting your people, your reputation, and your revenue in a world where social engineering is now the biggest threat to enterprise security. So, you want to learn more about this, you got to go to their website. It’s all at doppel.com. Go to doppel.com.
It’s time to play “What’s Worse?”
16:11.006
[David Spark] Kevin, are you familiar with this game?
[Kevin Tian] I have an idea of what’s coming.
[David Spark] All right, it’s going to make perfect sense to you.
[Andy Ellis] Just agree with me and everything will be fine.
[Kevin Tian] [Laughter]
[David Spark] Do not listen to him whatsoever. So, what happens is our fans, they send in scenarios, and they’re usually very bad scenarios. Neither you will like. And ideally, they’re sort of equally weighted in their badness. And so, what has to happen is you have to determine which of these two scenarios is a worse scenario. It’s kind of a risk management exercise. I always ask the co-host to answer first, so Andy will answer first, and you will either agree or disagree with Andy and give your reasoning with that. All right. So, let’s jump into this, Andy. Comes from Louis Zhang of AIA Australia. Two scenarios we have here. Your organization never conducts cyber awareness training for your staff. The employees are flying blind. They reuse passwords, think MFA is just extra hassle. You can only mandate technical controls. However, you cannot stop your curious staff clicking malicious links day in and day out.
[Andy Ellis] Okay. Although I would have liked if they’d said, if that would have had as part of the scenario, that your employees think that MFA is the Museum of Fine Arts.
[David Spark] They could think that as well. Both are true.
[Andy Ellis] Okay. So, I’ve got no security awareness training, and we have, as expected, like all of the practices that we rely on humans to do things because we don’t have good tools. The humans do the things that we don’t want them to do. Okay. Got that.
[David Spark] Yes. I mean, yeah. And they’ll also kind of go out of their way to screw things up.
[Andy Ellis] I didn’t hear it says they’ll go out of their way. I just hear that they…
[David Spark] That nothing’s going to stop them.
[Andy Ellis] Nothing’s going to stop them. Okay.
[David Spark] Nothing’s going to stop.
[Andy Ellis] Nothing is trying to stop them.
[David Spark] There you go.
[Laughter]
[Andy Ellis] There’s nothing trying to stop them. So, it’s not that they’re determined to do the wrong thing, but like…
[David Spark] They’re not determined. No, they don’t want to act maliciously.
[Andy Ellis] Right. They don’t know what the right thing is.
[David Spark] But they don’t seem to care.
[Andy Ellis] Okay.
[David Spark] Scenario two, your organization never conducts executive or board level cyber crisis simulation. So, your leadership team has never rehearsed a breach. When the inevitable happens, the CEO panics, the chief legal officer misses regulatory deadlines, and your chief marketing officer sends wrong information to the media, and the board is blindsided. Which one is worse?
[Andy Ellis] I like this one because they’re a little bit different from each other.
[David Spark] Mm-hmm. I have a sense I know which way you’re going to go on this but go ahead.
[Andy Ellis] Actually, I’ll be honest. I had an inclination of which way I was going to go, but now I’m actually, I’m second guessing. So, normally I’m always going to go like, look, I don’t blame the humans.
[David Spark] Right.
[Andy Ellis] So, the problem in the first scenario is you have bad systems, not bad humans. But that second one, that’s a kind of a fascinating… And the challenge that I have with that second one is that’s not a failure of cyber crisis training. That’s a failure of your executive management at doing their core jobs. The CMO ought to know better than that, even if you’ve done training or not, but apparently the CMO doesn’t.
[David Spark] Well, you could also say the same thing for the first scenario. People should know better than to expose the company secrets.
[Andy Ellis] No, no, it is not the job of individual employees to know the details of what length my password ought to be or whatever, but it is the job of the CMO to know not to publish things that you shouldn’t. Like, that is the CMO’s job.
[David Spark] Mm-hmm.
[Andy Ellis] So, I’m actually going to go with the second one is worse because…
[David Spark] This is the way I thought you’d go. Yeah.
[Andy Ellis] But for a different reason than I was originally intending, which is that the failure is people who have core jobs to do don’t know how to actually do their core job, that have nothing to do with security but now has a security implication. The first one, people who don’t have a security job, don’t know how to do the security job that isn’t their job to do, and you didn’t teach them. Eh, that’s just it is what it is. So, second one is worse.
[David Spark] All right. I like the answer. Kevin, I throw this to you. You know the two scenarios here. The first is your staff could care less what’s going on in the world of security. And the second one is your executive team is as ill-prepared as ill-prepared comes. Which scenario is worse?
[Kevin Tian] Yeah. I mean, that’s a tough one. Yeah, I was swinging back and forth as well. Right?
[David Spark] The bottom line is which would cause more damage?
[Kevin Tian] Yeah. I mean, look, with the first one, it’s going to be so easy for me to get into your organization, right?
[David Spark] Yes.
[Kevin Tian] Yes, of course, never blame the human. And of course, you’ve got to set up certain technical systems. But there’s just so many sources, at least from what we’ve seen now in terms of social engineering campaigns that are borderline, right, in terms of from a pure systems perspective, like, hey, if everyone on the team is authorized, this sort of click and this sort of transaction, things like that, at a certain point, a threat actor can get in. Right? And so, certainly the first one is pretty…
[Andy Ellis] But is training people to rotate their passwords a little faster actually going to solve that problem?
[David Spark] No, but here’s the thing. In scenario number one, the speed you’re getting breached is going to be very quick. In scenario two, the systems may be working, and it may never come to that point. May.
[Andy Ellis] Oh.
[David Spark] May. I got to say may.
[Laughter]
[David Spark] But the first case, something’s going to happen. It’s going to be bad.
[Andy Ellis] Right. But the problem in system one is bigger than security awareness, right, and that’s one of the reasons why I’m putting system to it. Scenario two is worse, even though scenario one is really bad. The problem is not in scenario one that you don’t have security awareness training. The problem is that your systems are awful, and you don’t have security awareness training, because if your systems were fine, the lack of awareness training wouldn’t matter. But if you threw in awareness training, you’re not going to actually be better. Like, you cannot make people so aware that they will compensate for bad systems and system one posited bad systems.
[Kevin Tian] Right.
[Andy Ellis] Scenario one.
[Kevin Tian] I guess it depends on how exactly these scenarios are getting defined. But let’s say even if you had perfect systems in scenario one, right, and maybe it’s less so about, hey, security awareness training would solve it. But kind of like what David said, though, the time to breach, though, I think would still be very, very short. And even if you had perfect systems, I would say you’d still be able to get a breach to happen or at least some really negative business impact to happen, even if it’s not formally a breach.
[David Spark] The thing is, even with perfect technical systems, the human can bypass it.
[Kevin Tian] Exactly. And if you got multiple humans’ approval, right.
[David Spark] Right. Okay, so which one are you? Kevin, are you landing on the first one is worse then?
[Kevin Tian] Again, obviously, both pretty bad.
[Laughter]
[Kevin Tian] But I’m landing on the first one because if you’re defining the second one as, hey, it’s the executive. They just haven’t had the training yet to run this sort of exercise and things like that. I mean, again, depends exactly how you define that second scenario, but the first one just seems like, yeah, a breach or significant loss of money is going to happen so quickly.
[David Spark] Good point. All right. You disagreed with Andy. You played well. That’s what I appreciate.
[Laughter]
[Andy Ellis] Pretty much all the sponsors do, partly because I try to see if I can pull them off of the problem that they fix.
[Kevin Tian] Right. Fair enough. Fair enough.
Please, enough! No, more!
23:44.393
[David Spark] So, our subject today is social engineering defenses. We actually don’t talk too much about the defenses around social engineering, so I’m glad we’re bringing this up. I’ll start with you, Andy. What have you heard enough about with social engineering defenses? Mostly we talk about awareness training, but what would you like to hear a lot more?
[Andy Ellis] So, I bet a lot of people think I’m going to say stop blaming the human because that’s like my pet soapbox.
[David Spark] You know what? I say we don’t blame the human enough.
[Andy Ellis] Yeah. Well, you’re wrong on that one. I blame you for that. Oh, dang it.
[David Spark] I’m not going to win a lot of fans that way. [Laughter]
[Andy Ellis] No. And I actually recently went through a related awareness training thing around physical security, and I had this realization that we spend all of our time talking about awareness. And for those of you who are fans of Colonel Boyd’s OODA loop – observe, orient, decide, act – everybody keeps talking about observation. How do you know when something’s happening? How do you orient? And honestly, I’m tired of that. We’re doing it backwards. We need to start from action. If we want to talk about defense, the first conversation is not noticing you’re under attack. It’s talking about what you would do once you realize you’re under attack. Figure out what all of your moves are. What are your plays? What are your options? Let’s talk about what our defensive strategies are. And then let’s talk about how we can be aware that there’s an attack so we can funnel it into a defense. Because I’m sorry, if we spend all of our time trying to get people to be aware of attacks, we’re burning their energy and then we don’t give them anything meaningful to do. They need to have a meaningful action if we want them to detect things.
[David Spark] Very good point. What can they do? All right. I throw this now to you, Kevin, please answer in the same way. What have you heard enough about when it comes to social engineering defenses? And what would you like to hear a lot more? And then I want to know what you’re doing with Doppel. So, let’s hear it.
[Kevin Tian] Well, I think that was a good answer by Andy. I think from what we’ve seen at Doppel in our perspective, certainly there’s a lot of focus on, like Andy said, security awareness. There’s focus on, hey, how do we prevent email attacks, right, BEC, things like that. The things that we want to hear more about and talk more about is the fact that how these threat actors are really taking advantage of your organization and your insiders is they’re going through channels that traditional corporate security solutions don’t cover as much, right?
So, for example, if I want to get access to your Salesforce instance or whatever, right, or if I want to get an email to go past your email gateways today, it’s not just going to be a [Inaudible 00:26:23] URL anymore, but I’m going to try to start doing phone calls. I’m going to maybe set up certain personas, maybe run certain campaigns on different platforms like LinkedIn, etc., and target your more vulnerable populations in your company. So, naturally, of course, we don’t have a security solution that lives on your LinkedIn account, and we don’t necessarily have a security solution that lives on your personal devices and things like that, but just because they’re not corporate communication channels doesn’t mean that a threat actor’s not going to take advantage of them.
[David Spark] Right. And this goes to the process of social engineering that it can have multiple touch points. Depending on how big the prize is for the criminal, they will spend the time developing that dossier on you. So, my question to you because we talked about it in my sponsor read in the middle of the show, how are you sort of connecting the dots here? Because it seems that’s what you’re doing with Doppel. Yes?
[Kevin Tian] Yeah.
[David Spark] Walk us through.
[Kevin Tian] So, the biggest thing about Doppel and what makes us different is that multi-channel perspective, right? We’re not just collecting traditional threat intelligence around domains, IP addresses, URLs, and dark web collections, things like that. But we’re pulling in the phone numbers that are getting past your email filtration systems. We are collecting the LinkedIn accounts that look suspicious that are maybe targeting your offshore team, right, pretending to be part of HR, pretending to be part of some help desk team, things like that. And so, because we collect that data, that’s how we can pivot and correlate and find stuff that a lot of traditional security solutions can’t.
So, for example, we got one phishing email address and because the database lookup in our end basically actually tied that email address to a LinkedIn account that had been registered with that email address. And from that LinkedIn account, we found adjacent sock puppet accounts. We found adjacent phone numbers, the emails that they were registered with, things like that. And basically now we’re mapping out an entire network of accounts, telephone numbers, and email addresses that have been targeting a Philippines team for this company. And as a result, what’s great about Doppel too is exactly what Andy just described. It’s not just about the detection awareness, but how do we respond to that as well and actually shut down that threat actor activity.
[David Spark] All right. So, explain how you do that. How do you respond?
[Kevin Tian] So, we basically can shut down these campaigns across any channel. So, historically, there’s always been great domain takedown solutions, right? Anytime someone does a type of squatted website or something like that, of course, you can issue a takedown request to the registrar or web host. But we can do that. We can do the telecom companies. We can do the social media platforms. We can do the advertiser platforms. We can even do some of the private encrypted chat channels like Telegram, WhatsApp, etc. And so, we can do takedowns that a lot of other vendors don’t even support. And so, for us, if we’re going to shut something down, we’re going to shut it down across that whole kill chain and across that entire network of activity.
[David Spark] One of the things we talk about is that they can be either dormant or do a lot of investigation over a period of time, a criminal. Can you see sort of the investigation period? Or maybe the question is, what’s the longest period of time you’ve seen an attacker operate for?
[Kevin Tian] For us, what we see, right, is we see the age of the accounts. We see the age of their activities. We see when they toggle things on and off, things like that. So, to give you a more concrete example, just yesterday was we were analyzing a campaign, and someone had spun up a fake Facebook page of a very high-profile executive, and it had an entire friend connection list of a whole bunch of other fake Facebook accounts, right? So, basically like mapping out that, hey, this is a fake executive page that’s connected to all these other fake executives, and those accounts have been around for X number of months, right? And so, it’s interesting to see when they started to upload those new profile pictures, when it became not just a parked social media account, but something that was active and posting content, things like that. And so, I guess, like for us, can’t necessarily say definitively, right, that they were in collections mode or they were just focusing on collecting more intel on the executive they’re targeting. But we can see how they’ll develop these different accounts over time, these different personas, these different campaigns. And so, that’s been pretty eye-opening for us.
Is this benefiting the company or just making my life easier?
30:45.322
[David Spark] Now we’ve talked a lot about AI today. Some of the issues we see are profound challenges for any organization, but others seem like self-inflicted wounds by the lazy. Now case in point, we’re seeing a ton of coverage of law firms using AI to draft filings, and inevitably, the LLM cites a hallucinated case, never existed. And this isn’t just one-person shops using ChatGPT to be more efficient. Reuters recently covered a case in Alabama where the 400-attorney law firm Butler Snow apologized for “a lapse in diligence and judgment in a hallucination-riddled filing.” And it seems big law firms are the leader in using AI. The ABA’s 2024 legal technology survey found that 46% of firms of a hundred or more attorneys regularly use AI in practice. And that’s not something that’s wrong, but this seems like a super simple problem to solve, Andy. You ask ChatGPT to do some searches for you. It finds some citations. Then you go verify them. How difficult is that? Isn’t that the lawyer’s job? But we’re still seeing it happen. Where are we most at risk in cybersecurity for using AI in the sake of efficiency, where we can’t afford to be similarly lazy?
[Andy Ellis] So, much as I hate to defend lawyers, it’s not actually even that simple.
[David Spark] Okay.
[Andy Ellis] This is a hard problem. So, ChatGPT is an LLM, and LLMs do not reason, right? Your mental model of an LLM should not be this is a researcher. It’s a regurgitator. It will give you outputs that lots of people have given outputs. If you ask it a legal reference question for a thing about like one of the amendments in the Bill of Rights, it’ll give you coherent things about that because lots of people have written about that. If you’re asking it to do deep and interesting research, that’s not its specialty. And if you’re asking it to output very specific references, it’s like, oh, you’re asking for legal filing. Legal filings should have references. References look like this, and it generates things that structurally look like a reference, but there’s nothing behind them. Like this is the problem is this is not a reasoning state engine that is creating…
[David Spark] No, but I understand, but still it spits out this stuff. Don’t you as a lawyer, then go, “Okay, let me just make sure this exists”?
[Andy Ellis] No, no. The problem is not that it spit out the stuff at all. It’s that you even asked it to solve this problem. It’s like trying to say which version of OpenSSL should I use for whatever, and it’s going to give you a version string. Or worse, it’ll give you a package that doesn’t exist. LLMs are good at producing output that lots of humans have produced in the past, just structured a little bit differently, as long as that’s the problem you’re trying to solve. Like, you want to generate a marketing website that is full of pablum? Like, an AI is great at that. There’s no thought leadership that it’s going to generate for you because it does not deviate from the mass of humanity. That’s how LLMs work. They generate what the mass of humanity can generate. Lawyers, despite how many of them there are, are not generating things that the mass of humanity have ever generated.
[David Spark] All right. I’m tossing you this, Kevin, and you kind of teased this at the beginning of the show saying that the solution here is frameworks that can sort of self-correct in this manner. How can frameworks help in this situation?
[Kevin Tian] I think it’s around where you use AI, right? And then how do you set up your validation for any output, right? So, for example, granted, I’m not a lawyer, right? But if we’re spitting out all these citations and things like that, like, is there a way that we could automate lookups for each of these citations, right? And making sure that it’s a valid URL, right? And it goes to a case that’s in the right databases, things like that. So, yeah, when we think about cyber, for example, I think a lot of interesting things in terms of how can it go wrong, right? Like, it can go wrong if the LLM just totally misses an alert and says like, “Yeah, this is not that important,” when it really is or flag a whole bunch of noise, right? It’s the same sorts of problems, and I think it comes down to how you make sure that your system and your team is robust and resilient to those sorts of hallucinations.
[David Spark] What would make them a little bit more robust and resilient? I think resilient is what I want to lean in on.
[Kevin Tian] Yeah. So, for example, if there’s a big button that you really want to [Laughter] make sure the AI doesn’t press, right? Or even your humans don’t press, right? What safeguards are you going to put there? And then for reviewing and kind of QA-ing some of the LLM stuff, and I think maybe another day we should have a debate about whether LLMs really reason or not. But let’s say we want to go QA all this sort of stuff, right? Like, how do you put humans in a loop into that process? And I think with coding, right, we’ve seen companies be successful there because they’ve created this UI and this UX where it’s easy for the human to be in the loop whenever these AI coding agents make mistakes, things like that. And that’s really when the adoption took off. So, those are just a couple ideas, right? It’s like safeguards, and then how do you integrate the human in the loop process?
Closing
36:05.332
[David Spark] Well, excellent. That brings us to the very, very end of this show. I want to thank our guest, Kevin Tian, who is the CEO and co-founder over at Doppel. And I want to thank your company, Kevin, for sponsoring not just this episode, but multiple episodes of the CISO Series and also Defense in Depth. Remember, too, our audience – Doppel, defend what’s real, disrupt what’s not. Go to their website, doppel.com to learn more about that. Let me ask you, Kevin, do you have any special offer that you want to make to our audience or are you hiring? What would you like to say about Doppel?
[Kevin Tian] First, in terms of offer, we do offer this to almost anyone, right? If you’re just curious about what your social engineering tech footprint looks like, you’re just curious around, hey, do I have certain executive impersonators, brand impersonators? Do I have active campaigns that are being launched against my employees today? We’re happy to give you a free demonstration and show you that in real time. So, I think that’s something that’s always eye-opening.
[David Spark] By the way, here, let me pause you on that.
[Kevin Tian] Sure.
[David Spark] We have many vendors who offer this and not enough of you listening take advantage of this. I will say, the times that I have taken advantage and seen this, a lot of times people don’t do it because they don’t want to know the ugly truth.
[Kevin Tian] Right.
[David Spark] You want to know the ugly truth. Trust me. Knowing is a lot better than not knowing. Take Kevin up on his offer.
[Kevin Tian] Absolutely. Absolutely. And then hires, we’re hiring a bunch right now. So, go-to-market sales, engineering. We’re also looking for great security experts to join our executive team, right? And help us continue to craft the product the right way and be that go-to-market presence outwards as well. So, looking for both leaders and individual contributors across the board.
[David Spark] Excellent. Well, thank you very much, Kevin. I’m assuming people can reach out to you via LinkedIn, yes?
[Kevin Tian] LinkedIn or Kevin@doppel.com. Always happy to help.
[David Spark] Kevin@doppel.com. Please go. I’m assuming you’ve got a job board too on doppel.com, yes?
[Kevin Tian] Doppel.com/jobs.
[David Spark] Doppel.com/jobs. Go look at that for jobs. Go take him up on his offer to know what’s going on in your environment. And if you actually have any questions about social engineering behaviors, ping Kevin at Kevin@doppel.com. Thank you very much, Andy Ellis. If you haven’t, by the way, picked up his book, 1% Leadership, whatever you’re reading now, it stinks. You should read Andy’s book instead.
[Andy Ellis] I don’t know that I’d say it stinks, but it’s probably not as good as my book.
[David Spark] Compared to that book, it stinks. That’s what I say. And I do want to know per my question at the top of the show, where I asked what are the incredibly inane requests that you’ve gotten that you’re just like, “How? How did this even come about?” I would like to know what those are. Ping us back. We’ll bring it up on a future show. Thank you, everybody. We greatly appreciate your contributions and for listening to the CISO Series Podcast.
[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. We have lots more shows on our website, CISOseries.com. Please join us on Fridays for our live shows – Super Cyber Friday, our virtual meetup, and Cyber Security Headlines Week in Review. This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com. Thank you for listening to the CISO Series Podcast.