Automation is going to save cybersecurity, right? But it can’t do it if all it does is scale our broken processes. In our rush to adopt new efficient technology, why do we insist on dragging old broken workflows into it?
This week’s episode is hosted by me, David Spark, producer of CISO Series and Matt Southworth, CISO, Priceline. Joining us is our sponsored guest, Leslie Nielsen, CISO, Mimecast. This episode was recorded live at Mimecast Elevate25.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, Mimecast
Full Transcript
Intro
0:00.000
[Voiceover] Biggest mistake I ever made in security. Go.
[Leslie Nielsen] Oh man, if I could only think of more than one. But I had a brilliant jerk working for me a few companies back, and the biggest mistake I made was letting him hang around, be toxic to the other people, and then sadly, when I left the company, he became somebody else’s problem and they had to get rid of him.
[Voiceover] It’s time to begin the CISO Series podcast, recorded in front of a live audience in New York City.
[David Spark] Thank you, everybody, and welcome to the CISO Series podcast. My name is David Spark. I am the host of this show. But joining me as a co-host is the man who is sitting to my immediate left, and that is Matt Southworth, who is the CISO over at Priceline.
Let’s hear it for Matt.
[Matt Southworth] Hi, David. Thanks.
[David Spark] All right. That’s Matt’s voice. You’ll hear a lot more of it later. I want to mention our sponsor today. It is Mimecast. Email and collaboration secured by AI block advanced human threats that legacy security misses and reduce human risk across your organization.
We are in a beautiful theater here at Mimecast Elevate, which is a conference that Mimecast is putting on mostly for their customers. And I also want to introduce who our guest is. I’m going to bring this person in early. The person sitting to our far left over here, the CISO over at Mimecast, Leslie Nielsen.
Let’s hear it for Leslie. Say hello so people know the sound of your voice, Leslie.
[Leslie Nielsen] Hello, people. I’m Leslie Nielsen. Glad to be here.
[David Spark] All right. I have an opening question before we jump right into the show, and it is the following. I’m assuming you have seen Sora, this new video creation app.
[Leslie Nielsen] Yes.
[David Spark] Yes. You’ve seen it, Matt?
[Matt Southworth] I have seen it.
[David Spark] You have seen it. Here’s my question. You’re in security. Everyone in this room is in security at some level. I had some friends who were playing with it, showing us all these videos they made, laughing their heads off, and I could only look at it in horror of, “Oh my God, what is this going to bring?”
So my question to you two is, is it conceivable a security professional can actually enjoy the Sora app, or is it impossible because we see what it’s going to do? For those people not aware, Sora is this new AI video generation tool, allows you to create unbelievably realistic videos of yourself doing different things, and other people you know as well.
So what do you think? Is it possible for a security professional to enjoy it, or no, we see too much horror in it?
[Matt Southworth] You’ve got to wake up every morning as an optimist, David.
[David Spark] [Laughs] Okay. Hold it. Do you enjoy it, or can you enjoy it?
[Matt Southworth] I can try to enjoy it. No, it terrifies me. Of course, right?
[David Spark] It’s terrifying. It’s completely terrifying.
[Matt Southworth] Yeah. You’ve got to try to pry out what’s positive about this, right? How do you make an impression on someone?
[David Spark] My friends were laughing, but it’s like a zombie attack. We see it’s coming. We can’t avoid it.
[Leslie Nielsen] You got to enjoy it. And what you got to do is you got to lean into it. We had a great bearded baby talking about application security on an internal video for our Cybersecurity Awareness Month. These things are good. Use it and show people, and then talk about the fear.
Because the reality is deep fakes are there, and they’re only going to get worse.
[David Spark] Or literally better, but…
[Crosstalk 00:03:38]
[Matt Southworth] Good point, good point.
[Leslie Nielsen] Well, eventually you’re going to see yourself and wonder if that’s you. That’s when it’s really bad.
What works? What’s not working?
3:45.256
[David Spark] Quote, “People who automated a bad process ended up with a bad automated process rather than an improvement. Sometimes they ended up in a worse place. The tool faithfully and beautifully executes the underlying brokenness.”
Now that’s Anton Chuvakin from the Google Cloud Security Podcast, encapsulating what many organizations learned the hard way. If you buy that fancy SOAR or SIEM, you’ll automate your corporate dysfunction at machine speed.
Chuvakin argues that, quote, “process is gravity, always pulling technology back into old broken workflows.” So my question, I’m going to start with you, Matt, is there a way to assess whether your processes are ready for a new tool? And we want tools to fix processes, but as Chuvakin argues, they’ll amplify your mistakes.
Does the prospect of AI agents, I’m throwing this also in, offer the possibility to escape this process gravity? What do you think?
[Matt Southworth] I think I’ve seen this a hundred thousand times, right? Where something’s broken and you hope a tool will fix it for you. And that’s on it, right? The processes are an expression of the organization’s priorities. And if it’s a broken process, that’s because it hasn’t been prioritized.
You can throw an LLM at it. It could make it better. It could be like adding an intern to it, but it’s not going to change the organization’s priorities, right? You’re still expressing that the organization isn’t investing here yet.
[Leslie Nielsen] Yeah, I’m totally with you on that, Matt. And the bottom line is you just got to take a step back on processes. What your as is, what your to be. And, you know, the challenge with that is you got to figure out that miracle in the middle.
And candidly, that’s what we’re supposed to be, right?
But people are starting to think AI is going to be that miracle. You can’t fix what’s broken if it’s just so horribly broken that it can’t be automated, outsourced, redone, re-looked at, even evaluated. Look at a maturity model. Look at the best way to do it.
But get it documented. Figure out what it is, and you’re going to find processes that you’re dependent on that you may not even know you have in your company.
[David Spark] Well, is there a situation where you’re not seeing the brokenness, maybe you need a tool to expose it, and then you reel back, “Okay, let’s fix this”? I mean, I’m assuming you’ve all had situations where you did implement something and something literally went awry that may have been simmering underneath.
[Matt Southworth] We never knew that this person was turning this dial twice a day, every day to keep the whole business running. Right?
[David Spark] Yeah. So it was [Inaudible 00:06:26] about clicking that button every few minutes.
[Matt Southworth] Exactly.
[Leslie Nielsen] Yeah. And on the technology side, feeding SIEMs and things like that, you say, “Hey, we need to track this process.” And then you turn it on, and then all of a sudden, you’re giving Splunk way too much money, if I can say Splunk.
[Laughter] All of a sudden, things are going horribly wrong, and you have way too many alerts, etc. But yeah, you can actually just by looking at them and getting through, get some good metrics on just how bad it is, and how much noise it’s generating.
[David Spark] Okay. So an argument sort of not against what Anton’s saying here, which you’re both very much agreeing with, but having that tool to expose it actually helps you see the brokenness too. There’s value to that.
[Matt Southworth] Of course, if nothing else, it’s a dumb person to have a conversation with, which will make you completely explain what you’re trying to accomplish. Right? And, again, if you can teach someone how you do your job, that’s when you truly understand it.
And LLM is the same way, right? If you can get it to repeat back to you what you’re actually trying to do, then you understand the process.
[Leslie Nielsen] Yeah, I couldn’t agree more on that. And the bottom line, people are afraid of AI. And they’re afraid of defining what they do for their job because they think AI is going to replace them. So even getting them to define the process to get it down.
But take a look at it, use whatever tools you can, throw it up against the wall, tabletop it, do all the exercises that you have to be able to just see what’s going on and see if you can break it and see what can be improved.
[David Spark] All right. Quickly, both of you, have you implemented any AI tool, or a tool that’s been AI-enabled, that has improved a process, shown a broken process, just anything that made a process better or exposed it?
[Matt Southworth] Absolutely. Every PR we do, every pull request in our code repo is run by an LLM. And it looks for things like secrets and bad code, and it’s not perfect. But it expands what the engineers can look at and flags something that might have otherwise passed through.
[Leslie Nielsen] Yeah, lots of different things going on. One of the things we’re most proud of is just enriching data with AI from the perspective of being able to… You know, the faster you can respond to an alert, the faster you can get at something bad that’s happening, the better off you are.
And we’ve been using AI to just enrich alerts, IOCs, etc.
Is this really the right strategy?
8:42.044
Quote, “Your roadmap won’t matter if no one’s willing to walk it,” end quote. Now, that’s Will Klusovsky of Appalachia Technologies on the trap that bullheaded tech leaders fall into. Being the quote, “This isn’t that hard. Why don’t you get it?” tech a-hole will result in project stalling.
And I think, Leslie, you were referencing to the brilliant jerk phenomenon earlier, so you’re aware of this.
By the way, for those of you who don’t listen to the show regularly, my co-host, Mike Johnson, is very, very anti-brilliant jerk. It is, for him, the worst possible thing that could be in an environment.
[Leslie Nielsen] Toxic.
[David Spark] Exactly what you had described at the very beginning of the show. I go on here and it says, “Now, people don’t want to work with a leader like that. To avoid the brilliant jerk phenomenon, start with their success, not your solution, ask teams what success looks like for them, and listen to their real problems.”
By the way, other co-host, Andy Ellis, said one of the fastest ways to sort of get little wins, sometimes big wins, is when you come in as a CISO, start asking people, “What is one stupid thing that’s still going on that you would love fixed?” And you’ll be amazed how many simple problems you can solve then.
But let me go on and say, the end goal is to make security as frictionless as possible so the business can do its job. Sounds good, but there needs to be some middle ground. So I’m going to start with you, Leslie. You want feedback. You don’t want to do this guy…
But heck, you need to lead. How do you play the balance?
[Leslie Nielsen] So I lean in with experience. I’m 25-plus years in cybersecurity. Long before we called it cyber, we just called it the people you don’t want to deal with because it implemented your project a lot longer. But I approach people with, it’s a way, not the way.
And I’ve got 25 years’ experience. Everybody in this room has a thousand, right, if you add them together.
So here is a way that I’ve seen it work. Let’s other people talk about it, not necessarily the way. And then just come out with a collaborative approach that’s actually going to solve the problem altogether.
[Matt Southworth] Yeah, I think looking for people who are collaborative, creative, and good communicators is important. And it’s not only knowing what the organization’s successes are, but knowing what individual strengths are. And you might not know your own strengths, all right?
You have to be a little quiet and observe when people come alive and where they actually know they can do something well.
[Leslie Nielsen] Yeah, that’s great. I love it when I come out of a room and people don’t realize I was the most senior person in the room. I love it when other people are driving and doing things and people are like, “Oh, you report to that person over there, right?” It’s like, “Yeah, yeah, I do.” [Laughs]
[David Spark] You know, what’s interesting, you say that because when you stop leading and you let others lead, you realize certain things can come out that don’t happen when you’re leading, right?
[Matt Southworth] Absolutely. I was complaining to my boss just this week that sometimes I feel like my job is going to a meeting and telling people to do their jobs, and was brainstorming how to empower everyone on my team to do that even when I’m not in the room.
[Leslie Nielsen] Yeah, and we’re in a 24-hour world. You have to have your backup. You have to be able to take a little digital dark time, and you have to put trust in people and let them get experience too. We need more good cybersecurity people, and people need to collaborate and share.
And it’s kind of been the, you know, “I could tell you, but I’d have to kill you mentality” within the business for years, but those days are over.
[Matt Southworth] Yeah. Speaking of toxic, right?
[Leslie Nielsen] Yeah.
[David Spark] So I was just in Houston at this conference, HOU.SEC.CON, and two CISOs I spoke to did this exercise. I’m interested to know if you did it. Where they would do the “What if I’m not here?” scenario. “What if I’m not here and this happens?” So I’m not a contact for you to reach out to.
And it was two things. One was trying to see if they can think about what you think, which wasn’t necessarily important, but if they could handle the situation. Have you done these before? Like the, no Matt, no Leslie scenarios.
[Matt Southworth] Yes, we have. So three years ago, maybe when we did a annual tabletop, I was traveling at the time and the team didn’t know this, but I was having internet problems for the first hour of the tabletop. So I let them run it themselves.
And then I joined later to see how it was going, how things were progressing. One thing we learned, there was one person in the company who could declare an incident, and that was me. Oops.
[Leslie Nielsen] Yeah. Live-fire tabletop exercises happen all the time, be it you’re on a plane or you’re out or you’re trying to be digital dark. But the other thing I like to do though, during tabletops is just role reversal. So who’s incident commander, who’s incident manager, and just flip it.
Because it’s really hard for technical people that have come up through the ranks to understand the business side and to see it. And make them role play it, make them feel a little bit of that pain. Get the CEO, the CRO, the CFO, get everybody involved, and make sure that people understand what’s going on.
[David Spark] I want to come back to one last thing and then just wrap this up. It’s about the brilliant jerk. So is there a way, and again, all people are different, but have you ever been successful with a brilliant jerk, or it can never work and you just got to get rid of them?
[Leslie Nielsen] I have, which is partially what made me keep that one I talked about around. But there are just some people that I am not talented enough to bring around, right? I’m not perfect with everybody, and not everybody works great together.
But yeah, there’s good brilliant jerks out there. They just don’t realize it. And a lot of time they just haven’t had good leaders and they feel like they have to just come on so strong, “I’m so right. Everyone should listen to me.” Just take a step back, get them involved, get them collaborative.
80, 90 percent of the time you should be able to bring them out of it.
[Matt Southworth] Yeah. You can do a lot with some feedback given at the appropriate time in the appropriate way, right? But you don’t have to speak first. You are where you are because you’re smart and talented. You don’t have to impress everyone on that every time.
Let the most junior person in the room speak first. Right? These are things we’ve all heard, but just keep repeating it to people who might have a personality that’s not gelling yet.
[Leslie Nielsen] Yeah. And I’m just going to tag onto that. Absolutely what Matt said. And define your acronyms. If you sat in a meeting with finance, you’re going to realize that, “Wow, I don’t know anything about finance, and this is probably what I sound like when I’m talking to other people,” right?
Tell people, “If you leave a room and people don’t understand what you’re saying, you lost that meeting. That’s not a win. That’s a loss.” And that’s a good early indicator of somebody that’s going to be a brilliant jerk.
[Matt Southworth] Also, can you just stick them on the help desk for an afternoon? Really.
[Crosstalk 00:15:15]
[Leslie Nielsen] Oh, good one.
[David Spark] Oh, that is a good one. [Laughs]
[Matt Southworth] I like it.
[Leslie Nielsen] I like it.
[David Spark] And by the way, put a camera on them [Laughter] because that’s going to be entertaining.
Sponsor – Mimecast
15:22.920
[Voiceover] Who’s our sponsor this week?
[David Spark] Today’s episode is brought to you by Mimecast. Let’s hear it for Mimecast. All right. They are trusted by over 42,000 organizations worldwide to secure human risk. Cyber threats are getting smarter every day and threat actors aren’t just targeting your technology.
They’re targeting your most valuable asset: your people.
Now, Mimecast helps you identify and secure risk with a unified, intelligent platform that protects across the spectrum of threats from email and chat to file sharing. Now, with Mimecast, your team becomes your strongest line of defense. Empower employees to make smarter security decisions, stop threats before they spread, and keep your organization resilient.
Do you want to learn more? Well, visit Mimecast.com to discover how Mimecast integrated protection helps you stay one step ahead of cyber threats. Remember, that’s Mimecast.com. And when you go, let them know you learned about them from the CISO Series.
It’s time to play “What’s Worse?”
16:31.187
[David Spark] All right. For those of you who’ve heard the show before, you are familiar with this game. We’ve been playing this game since the beginning of the CISO Series. And here’s how it works. It is two horrible scenarios. They’re bad. Neither one you like.
But it’s a risk management exercise. You have to determine which of the two is worse. I always ask the co-host to go first. So, Matt, you’re going to go first. And here is the scenario.
It comes from Neil Saltman of AHEAD. And here’s the scenarios. What’s worse? Having no data governance policies while your company moves full speed ahead with AI projects, with shadow IT everywhere, and no real visibility into what’s happening? Or relying on AI completely to protect your data with someone who doesn’t know what they’re doing, manning the controls, and the controls have gone rogue, changing at such a rapid pace, you can’t track all the changes?
What is worse, Matt?
[Matt Southworth] Oh, great. Thanks.
[David Spark] This is a tough one.
[Matt Southworth] Well, like Leslie, I’ve been in this game for a minute, and I think my answer will have changed over time. Initially, what’s worse, probably would have thought all the AI processes uncontrolled, whatever. Today, with these gray hairs, not having a data governance program is worse in every scenario.
[David Spark] So just AI going loose and changing, that’s not as worse as no data governance program.
[Matt Southworth] I mean, you could have dumb interns, right? It’s not that different, except they are a little faster. So without your data governance program, you don’t even know whether the rogue AI is doing real damage to the organization or not.
So lack of data governance is worse, final answer.
[David Spark] All right, final answer. I’m going to go to you, Leslie. Do you agree or disagree?
[Leslie Nielsen] I disagree.
[Crosstalk 00:18:34]
[Leslie Nielsen] I’m going to go with B.
[David Spark] B is worse, why?
[Leslie Nielsen] Just because it’s going to cause the proliferation so much faster. You know, not having data governance, you can back into it and start teaching people, etc. But just having rogue AI everywhere, spitting stuff out, not knowing what’s going on, and not even be able to track it back down, that scares me more.
[David Spark] Yeah, that’s pretty bad. All right, I’m going to throw this to the audience. All right.
[Leslie Nielsen] They’re both bad.
[David Spark] They’re both horrible, right. But again, the game’s called “What’s Worse?” So I’m going to throw it to you. So the two scenarios, essentially, the no data governance with the shadow IT going everywhere, or AI is completely running your program, and someone doesn’t know what they’re doing with the manning of controls, and the controls have gone rogue.
So that’s the second scenario. So no data governance, controls going rogue, which one’s worse? By applause, how many people think no data governance is worse? By applause.
[Applause]
[Matt Southworth] Come on.
[David Spark] All right. A few people. By applause, how many people think the AI going rogue is worse?
[Applause]
[Matt Southworth] Oh, I lose.
[David Spark] They went with Leslie on this one.
[Matt Southworth] The crowd has spoken.
[David Spark] They have spoken.
What are these security pros talking about?
19:41.162
[David Spark] All right. This is a fun game.
[Matt Southworth] This is the new game.
[David Spark] Not that new. We’ve played it actually once before. But actually, you know what? It’s new in that we actually have bumper music for it. So that’s what we got for it. So here we go. This is the game. We interviewed a series of cybersecurity professionals, this was actually at RSA, this year on a variety of different topics.
And we’re going to go play their answers. You have to guess what the question was.
So you’re going to hear a series of different answers to a question, but you’re not going to hear the question. You got to figure out. If they don’t figure it out, I will toss to you, the audience, and see if you can figure it out. Okay? We got four rounds of it.
Here is the very first one. Listen. And I can play it a second time if you need.
“You could be 100% secure. People know exactly what they have to do. There is a single way to determine the security posture of an organization.” What do you think the question was?
[Matt Southworth] I think the question was, “What does your CEO believe that’s completely false?”
[David Spark] I’m going to give you that. That is, “What are common misconceptions about cybersecurity?” Very, very good. All right. Way to go on that. Let me give you that.
[Leslie Nielsen] You nailed it.
[David Spark] You get the organ win on that. All right. Here comes the second one. “Multi-factor authentication.” “Timely documentation of everything that they learn and do.” “I wish they would just patch their stuff.” “Not to click everything they see in their email.”
[David Spark] Leslie, you want to try this one?
[Leslie Nielsen] If I could only get my stupid users to do one thing.
[David Spark] That is correct. Very, very good. All right. Good job. All right. Here we go. Another one. “It requires expertise to be a cybersecurity person. It doesn’t. Requires different thought.” “Compliance does not equal more secure.” “Single panes of glass and more single panes of glass covered by single panes of glass.” “Most people do not understand security strategy.” What is the question?
[Matt Southworth] Feels like something that you’d need to educate a talent acquisition or recruitment team on to get the most diverse candidates.
[David Spark] No, that’s not it. Not at all. What do you think, Leslie?
[Leslie Nielsen] I think it’s the misconceptions around security and the things that…
[Crosstalk 00:21:58]
[David Spark] That was the first one. No, no, no. It wasn’t that. All right. I’m going to throw this to the audience. Just literally yell it out if you think you know what it is. Anyone think they know? Nobody? Can I take another shot? You want to take another shot at it?
[Leslie Nielsen] Cybersecurity professional pet peeves.
[David Spark] Exactly correct. Yes. Good job. Excellent. Very, very good. All right. Last one. Let’s play the last one. Good job. Leslie’s gotten two. Matt, you have one. Audience, you have zero. Audience has had zero. Here we go. Last one.
“It’s going to be quicker to do a bunch of things.” “I would say threat detection for information to reference and learn about what is going on and what’s being detected.” “To help augment some of the mindsets of people, and also just basically help streamline all the security practices.” “To be proactive in identifying risk points.” What do you think the question was?
[Matt Southworth] Sounds like what do you want to take away from this conference, or what do you want to do over the coming year?
[David Spark] That is not the correct answer. What do you think?
[Leslie Nielsen] What do you think AI is going to do for you in security?
[Matt Southworth] Nice.
[David Spark] That is correct. All right. Very good, Leslie.
[Leslie Nielsen] But Matt, I like the way you’re thinking.
[David Spark] It was a good answer, but Leslie wins three to one. Sorry, audience. The goose egg. You blew it.
Could this possibly work?
23:24.283
[David Spark] I love this topic. It’s something that’s brought up on the cybersecurity subreddit. We’re a huge fan of the cybersecurity subreddit. So here’s my question that was asked on the cybersecurity subreddit. What overlooked security controls are actually getting the job done?
So that question came up from a sysadmin on the cybersecurity subreddit who wanted to know which are the smaller, less hyped controls that are most effective.
So here are some of the very telling responses. Marking emails from external senders, that topped the list. Outbound firewall rules came up repeatedly. And the simple, but often ignored, practice of deleting old exceptions. So I’ll start with you, Leslie.
What are the, quote, “boring controls” that dramatically improve your security posture? And they don’t necessarily have to be boring, but why are they so effective?
[Leslie Nielsen] So this is non-sexy and it’s going to be controversial, but friction in getting things done, especially user revisioning and access control, because people that can do stuff really, really quickly tend to end up making mistakes and not following process.
[David Spark] I will tell you, there are some friction things I actually enjoy. I’ll give you a perfect example. I have a CRM-type tool that if I try to delete more than one entry, it’ll make me type in two boxes, three, whatever the number is. I have to physically type that whole thing.
That friction I very much enjoy.
[Matt Southworth] Prevents stupid mistakes?
[David Spark] Yes, very much prevents stupid mistakes. That’s a good one. What do you think?
[Matt Southworth] I racked my brain on this, and I am a total pack rat, but if you have an aggressive retention policy, especially for things like email, 30 days, 90 days, your attack surface just shrinks dramatically. If you can turn that on, it’s incredible for helping the organization prevent the loss of data.
[David Spark] Well, that’s a pretty… Does anyone have a 30 or 90-day email policy? I mean, we reference emails years ago.
[Matt Southworth] I’ve seen 90 days.
[David Spark] You’ve seen it? But hold it. How does the business operate like that?
[Matt Southworth] People have to be disciplined about email is not your file archive, right? Put the data somewhere that it’s referenceable. Even if you leave the organization, it also gets you out of these one point of failure mentalities.
[Leslie Nielsen] And it’s not your file transfer protocol. [Laughs]
[David Spark] Yeah. But I couldn’t imagine doing… I don’t know how I could stay in business doing that.
[Matt Southworth] Right.
[David Spark] All right. Any others you got? I mean, I thought these were very good, like the marking emails from external senders, but one thing that I’ve heard is if you use the same color, they start to get glassy eyed. So changing the color is critical.
[Matt Southworth] Just educating users, right? Giving them the tools. Maybe you can buy a password manager for your users. It saves so much pain in the long run.
[David Spark] And this interesting one, the often ignored practice of deleting old exceptions. People throw those in just in a whim and immediately forget about them, don’t they?
[Leslie Nielsen] Oh, absolutely. And your risk exception process, there are so many people that just don’t go back and look at them, right? It’s like, okay, you have this risk, and it’s accepted for 90 days. And then two years later…
[Matt Southworth] You have a firewall rule with a reference to a ticket number from a tracking system that hasn’t existed in 10 years.
[David Spark] And how do you unearth those?
[Matt Southworth] Turn them off and see what breaks. Don’t do it at the end of the year.
[Leslie Nielsen] Amen.
[David Spark] By the way, have you done that practice? The “let’s turn it off and see when the phone rings”?
[Leslie Nielsen] Yes. Yeah. I had a previous company, or a few companies ago, we had a whole bunch of orphan systems from mergers and acquisitions, and we started a system eviction notice program. And we called them centers, SEN program centers, you know, kick them off the network and shut stuff down.
We shut so many things down and nobody complained, I told the team, “You need to push harder. We need to get somebody to complain.” [Laughs]
What about this AI security challenge?
27:27.632
[David Spark] “AI-generated code presents fundamentally different risks than other software,” said Boyd Kane of CubeSpace. “But general beliefs of secure code do not apply to AI-written code, such as software vulnerabilities are caused by mistakes in the code.
You can find bugs by analyzing the code. If you fix a bug, it won’t come back again. It’s like asking the same question twice to ChatGPT and getting two different answers.”
Do you agree? I’ll start with you, Matt, with this assumption that AI-written code or even vibe coding needs a different code analysis treatment? And if so, what should we be doing differently when securing AI-generated code?
[Matt Southworth] I don’t know if the point of AI generation is what differentiates how much you need to review and scrutinize the code. I think what you’re doing with the code, where it’s running, the context, what data it’s touching is much more important.
We’ve been using tab completion and code forever, right? And it’s, again, the dumb intern problem. I’m not sure that the person writing this code is any smarter or less smart than the gen AI tool. Anecdotally, when we look at gen AI-generated code, it’s longer.
That doesn’t mean it’s better, right? And it may not be as efficient.
So there are some known problems with this code, and you might want to tune your reviews, your detections around that. But fundamentally, if it’s critical code that’s touching sensitive data, that’s where you want to invest the review, whether it’s reviewing human code or machine-generated code.
[Leslie Nielsen] Eventually, AI is going to get to the point where it’s saying, “Oh, we don’t even need code. I’ll just write bytecode and stick it right in. Or I’m just going to redesign algorithms or how computers [Inaudible 00:29:12], etc.”
But we’re not there. We’re just generating code. And there are good, secure software development lifecycle practices that need to be followed, and need to be followed rigorously, especially with the proliferation of more code from AI. Do good architecture and design, do threat models, iterate over those threat models and make the architecture and design stronger.
Do static, dynamic, as well as secure content analysis through the process. Do pen testing, red teaming, bug bounty, responsible disclosure. Just do all the things that you’re already doing. Do them better, and you’re going to make it through the AI.
[David Spark] So what you just said may have literally answered my next question, but I’m going to throw this out anyways, is prior to AI-generated code, there was a process to work with developers on being more security conscious, and also thinking about vibe coding, the people who are just literally writing in natural language text to have code spit it out for them and creating the app itself.
Is there a way to have a, “Hey, look, I know the sort of environment of developers is changing, what is constituting a developer is changing. What is the kind of conversation about security you have to have with a vibe coder, someone who literally has no development experience?”
And I’ll just throw this out very quickly, I met a guy in Florida who literally vibe coded his way to an extremely successful app, but he was unbelievably aware, “Oh no, this has got holes in it. I’ve got problems coming up.” He felt like, “My pants are down by my ankles now.” So either one of you jump in.
[Matt Southworth] So I think the problem here is the person generating the code can’t explain what it’s doing. And again this is a problem, whether it’s a human-generated or machine-generated. And I mean, you can start with the five whys. Ask, “Why did you do this?
Why did you make this choice? What do you expect to happen here?”
And then hit them with the QA questions, “What are the failure modes, if instead of what you expect to happen, something unexpected happens, what will occur?” Chances are the vibe coder is not going to know, so you educate them then on guardrails and how to protect [Inaudible 00:31:18]…
[Crosstalk 00:31:19]
[David Spark] By the way, you’re totally talking in a language of developers, I don’t think a vibe coder, any of that crosses their mind at all.
[Matt Southworth] I guess the vibe coders I’m thinking of are technical folks, but not developers. But you’re right, this is a whole new world for people who have never even touched a C++ textbook.
[David Spark] I remember having this conversation about producing a live physical event, and I was talking to somebody, and they started asking me questions, kind of like what you’re doing, and I just… My face just felt like literally none of that crossed my mind, not one thing.
And I have that same feeling that’s happening with people who are coders. There’s this whole slew of questions that just never crosses their mind.
[Leslie Nielsen] Yeah. So I’ll go back to what I was saying in the beginning. It’s architecture and design, threat modeling, etc. When you’re vibe coding, you can’t do any of that because you don’t have a design, you don’t have things to do threat modeling, etc.
You may have to, this is where I would come in, use tools that can reverse engineer that, that can pull it out.
But then it makes it even more critically important to go through static, dynamics, secure content analysis, do all of that, and then pen test the heck out of it. So beat that stuff up and make sure it’s ready for production because that’s the type of stuff that leaks vulnerabilities in zero-days into the network.
[David Spark] Let me throw this out. Because what you just described, why couldn’t that be put into an AI development system, like I say, just vibe code it, spit it out? Okay, you created it. Now, here’s the next stage of testing it [Inaudible 00:32:51].
It could literally, like a wizard, walk you through the process.
[Leslie Nielsen] I would say from a tool perspective, we are getting there. We’re enhancing static and dynamic analysis with that. We’re actually doing proactive pen testing, like pen testing continuously, which could eventually replace dynamic analysis.
It’s getting there, you know, using AI… World War II airplanes were just going crazy, and everybody was like, “How do you shoot down an airplane?” With another airplane, right? Use AI to fight the proliferation of AI.
[Matt Southworth] Yeah. We do have a threat modeling GPT, for example, and that’s helpful if the person reading the results understands what they’re looking at. And that’s what I wanted to ask you, Leslie, is how do you put these controls or processes in place in a way that doesn’t brand you as the team of no, or squashes the enthusiasm and creativity of the people who are vibe coding?
[Leslie Nielsen] Yeah, it’s tough, but you have to follow at least some process, and what you can do is you can turn it around, security awareness training, security champions, like, “Look, you’re ahead of the game, and you’re generating all this great stuff.
Look at these vulnerabilities. Let’s just figure out how we can fix this, and then how you can augment and teach others.” You know, play on…
[Crosstalk 00:34:01]
[Matt Southworth] Yeah, appeal to vanity.
[Leslie Nielsen] Yeah, appeal to vanity. [Laughs]
It’s time for the audience question speed round.
34:04.846
[David Spark] So, in my hand right here, I have index cards, questions from this audience and some other people who I met just earlier this week. And with the little time we got left, I’m going to see how many we can get through. They have not heard these questions, they do not know them at all.
So let’s get some quick answers.
I like this one from a Mimecast employee, Katie Callahan, who asks, “I know you got a lot of fears about AI, but I want to know what’s your absolute number one fear of AI?” Matt?
[Leslie Nielsen] Well, Katie, my number one fear of AI is uncontrolled development and deployment into production, right? And easy to say, but what do I mean? I mean, like any other development process, you need to be strong around world-based access control, secrets management, etc.
And that’s where I worry that those processes and disciplines that we’ve built up over years will fall down.
Social engineering. Just the ability to just strengthen that, you know, from the hacker side, the ability to just really strongly deep fake, better emails, better social engineering, and to get it done and get in. Because, you know, the Scattered Spiders, etc.
of the world, that’s what’s scaring me the most right now.
[Matt Southworth] That’s a very Mimecast answer.
[Matt Southworth] Thank you.
[David Spark] Have either of you fallen for a deep fake video thinking it was real?
[Leslie Nielsen] I have not. I did fall for an Onion article years ago where Microsoft was going to transfer Office onto Linux. [Laughter] [Inaudible 00:35:40].
[David Spark] That is a very nerdy response by the way, Leslie. [Laughter] All right. From Steven Gonzalez of Zencore. I like this one. And just select one. What is one good policy you have either heard about or have implemented yourself around AI governance?
[Matt Southworth] One good policy we’ve implemented around AI governance is awareness. We don’t tell people not to use these tools. We tell them to talk about it. So to make it simple, a Slack channel for everyone using a tool where they talk about what’s going well and what’s not.
[Leslie Nielsen] Just having AI governance. [Laughter] I’m sorry, but no, there are companies out there that don’t.
[Crosstalk 00:36:20]
[David Spark] …turn the on switch…
[Crosstalk 00:36:22]
[Leslie Nielsen] Yeah, exactly. Just get procurement, get legal, get everybody together, talk through it, have just a small committee and talk it through, but bring some of the other people in. But yeah, it’s just having that. There’s so many companies that still aren’t doing that.
[David Spark] All right. This one comes from Masha Sedova at Mimecast. And by the way, we’ve talked about this many times on our show, but I want to hear your thoughts. And I’m going to keep this general because there could be nuances to this, but if an employee keeps failing simulated phishing tests, should they be fired?
[Leslie Nielsen] Okay, I got this one.
[Laughter]
[Crosstalk 00:36:58]
[Leslie Nielsen] I’m sorry, but yes. You have to go through a process, right? Two strikes, three strikes, etc. You have to give them every education opportunity. But if someone is a threat, if someone’s constantly letting robbers into your office, yeah, they have to go.
[Matt Southworth] I think I agree with you, Leslie, but I don’t think this is a decision the security team should make or have to make.
[Leslie Nielsen] Fair enough.
[Matt Southworth] Very much if you’re not able to tell this person’s manager and HR the risks that are presented, if they don’t see it the same way, then there’s a fundamental organizational problem you have to address first.
[Leslie Nielsen] Yeah. So that’s a great point. And that’s acceptable use policy, right? If something’s in the acceptable use policy and somebody is not living up to it, talk to HR, do the right things, report, open the ticket, whatever you need to do.
But you’re absolutely right. Follow the process.
[David Spark] All right. I’m going to throw a wrinkle in this. And this has come up on the show before. So I have a good friend who works in HR for a big company. They had a situation with a mechanic who could not be trained, was just failing phishing tests over and over.
So again, he kept failing, not a knowledge worker, not in front of the computers, not working in finance, not working in accounting, but they did get fired because he kept failing phishing tests. Should that kind of person… Should the rules apply to everyone or vary?
If I’m in accounting and finance, the rules are stricter. If I’m a mechanic, lower. What do you think?
[Leslie Nielsen] Well, re-engineer your business process. Why does a mechanic need email, right? I mean, you know, they’re just…
[David Spark] Well, look, they need to get information from the company, and then payroll information. I mean, just basic community communications.
[Leslie Nielsen] Re-engineer the business process where they have more limited access, where they have a secure portal. Yes, they probably should because they’re potentially bringing harm into the company. It’s tough, but also look at re-engineering business processes.
It’s one of the last defenses.
[David Spark] Matt?
[Matt Southworth] Totally agree. Why do they have email? I understand the nuances there, but then why are the links in their email live? If they’re a mechanic, what device are they looking at these emails on, right? There’s a bunch of places you could put in some simple controls before you bring out the hammer.
[David Spark] All right, from Tom Doughty, who’s the CISO over at Generate:Biomedicines, asks, “What human actions are you comfortable with AI supplanting?”
[Leslie Nielsen] Yeah. I mean, start with a low level. I’ll talk [Inaudible 00:39:22] SOC. We’re enriching, enhancing data. We’re more quickly opening tickets, looking at indicators compromised, pulling all the data in that we need, getting it at our fingertips.
That’s from the security side. On the coding side, I’m fine with AI coding. But again, I’ll go back to what I said earlier. You got to have those processes in place. You got to do the reviews.
[Matt Southworth] I think, yeah, on the security side, what’s the stuff that either we’re not doing today because it’s boring or we don’t have the resources to allocate, start there. Enrichment is huge. Additional eyes on PRs is huge. Great stuff that you can do pretty easily with a couple API calls.
Pet peeve of mine, when I receive a communication from somebody on my team or when I’m looking over a review and it’s obviously been written by ChatGPT and doesn’t have any of that person’s voice in it, oh, I really don’t like that process.
And it’s a fine line to walk because I’ve got people who speak a dozen languages and English is not their first. I want them to clean up what they’re trying to say, but I don’t want it to read like it was written by a robot with an em dash in every sentence.
[Leslie Nielsen] That’s [Inaudible 00:40:24]. And in fact…
[David Spark] Em dash, by the way, that has been a telltale sign of AI writing, the extremely overuse of that.
[Leslie Nielsen] The word “delves” almost always guarantees that an email is generated by ChatGPT. [Laughs]
[David Spark] All right, very last question, and this is very much in the world of Mimecast here. I’ll start with you, Leslie. And you can speak with organizations because I know you’re doing this stuff at Mimecast, but how is your, or you’re seeing other security awareness programs evolving with deepfakes?
And it comes from David Peach, who’s a CISO of Intersection.
[Leslie Nielsen] Evolving with deepfakes, we’re using smaller, quicker, and we’re also nudging and putting stuff out when people do fall for phishing, etc. I don’t know from an evolution on the deepfake perspective. We are including more things in the security awareness, but it’s really shorter, quicker content that’s to the point and delivered at the right time.
[Matt Southworth] Yeah, I think you need to think about who the target audience is, right? I think a lot of the deepfakes we’ve seen are impersonation of an executive, right? So you think about your AP team, you think about your help desk, and you target training to them with the deepfake, with the recording of your CEO saying, “Hey, help me reset my password.” And you target it that way, you make it smaller and more oriented towards their specific role and concerns.
[Leslie Nielsen] Yeah. And David, one last thing. So I’m just going to tell everybody right now, and I send this out in email, cybersecurity awareness [Inaudible 00:41:48], our CEO will never ask you for Amazon gift cards. Never going to happen. [Laughs]
[Matt Southworth] Over WhatsApp.
[Leslie Nielsen] That’s right.
[David Spark] By the way, I want people to send me Amazon gift cards.
[Leslie Nielsen] Yes. [Laughs]
[David Spark] I would love that. I would really, really appreciate that.
Closing
42:05.647
[David Spark] Well, that brings us to the very end of the CISO Series podcast. Let’s hear it for my guests. Let’s hear it for them. It’s Leslie Nielsen, who’s the CISO of Mimecast, and also Matt Southworth, who is the CISO over at Priceline. Thank you both for coming.
This was a lot of fun. I want to thank also Mimecast and Mimecast Elevate for bringing us out to New York to do this live show.
Remember, go to Mimecast.com for essentially all your human risk factor needs, if you will. Any last words? Oh, question I always like to ask. Are you hiring over there at Priceline?
[Matt Southworth] We are hiring at Booking.com in Amsterdam, beautiful city. Go check it out.
[David Spark] I’ve been out there. Yes. And are you hiring at Mimecast?
[Leslie Nielsen] I got some recs open. They’re on LinkedIn, and I’ll make sure they get pushed.
[David Spark] Awesome. Well, thank you again, everybody. Thank you for coming out to see us live. I say it all the time, but we really, really appreciate your contributions and for listening to the CISO Series podcast.
[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. We have lots more shows on our website, CISOseries.com. Please join us on Fridays for our live shows, Super Cyber Friday, our Virtual Meetup and Cybersecurity Headlines Week in Review.
This show thrives on your input. Go to the participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com.
Thank you for listening to the CISO Series podcast.