Starting from square one on risk management isn’t easy. Most GRC tools assume a degree of process maturity that many organizations don’t have. So how do you get the ball rolling to allow you to use those tools down the line?

This week’s episode is hosted by me, David Spark, producer of CISO Series and Matthew Southworth, CSO, Priceline. Joining them is our sponsored guest, Saket Modi, CEO, Safe Security. This episode was recorded live at FAIRCON25 in NYC.

Got feedback? Join the conversation on LinkedIn.

Huge thanks to our sponsor, Safe Security

Full Transcript

Intro

0:00.000

[Voiceover] Biggest mistake I ever made in security. Go.

[Saket Modi] Thinking security is a technical problem and not a business problem.

[Voiceover] It’s time to begin the CISO Series Podcast, recorded in front of a live audience in New York City.

[David Spark] Welcome to the CISO Series Podcast. My name is David Spark. I am the host and producer of the CISO Series Podcast. Sitting to my immediate left is the guest co-host for today’s episode. He’s actually done this show as a guest co-host before.

Please, warm round of applause for Matt Southworth, CSO of Priceline. Thank you. All right, Matt, just quickly, we are at the FAIRCON25 Conference. You come to many of these conferences. Here’s my quick question for you. When you come to an event like this, you talk to your colleagues.

Everyone talks about the conversations being the most important part of the show. My question to you is, what are the questions you ask your colleagues when you come to an event like this?

[Matt Southworth] Few things. So everyone likes to talk about the new vendors, but I like to talk about what are you not doing? What vendor, what service, what process are you dropping? I think that’s interesting. And sometimes people aren’t even aware of what they’re dropping.

It’s always good to find out what everybody else’s board is asking about. Somebody mentioned quantum crypto to us and that’s a project for next year. And what are the open source tools your engineers love to play with?

[David Spark] That’s very good. All right. Let me also introduce our sponsored guest for today’s episode. Very thrilled they’re on board. Many of the reasons why many of you are here today, to my far left, it is a CEO of Safe Security, Saket Modi. Let’s hear it for Saket.

[Saket Modi] Thank you, David.

It’s time to measure the risk

1:55.273

[David Spark] “What’s actually driving risk in your environment right now?” This is Lisa Begando of Health Catalyst. She argues that too many security teams never answer that very critical question. Instead, they’re consumed with busy work, filling out third-party questionnaires and chasing compliance scores.

She proposes a popular trend, modernizing your GRC system to automatically pull real time control data from your environment and feed it into a risk engine that quantifies how control failures impact risk as they happen. Now, if you throw a rock, you’ll hit any number of AI -powered tools promising continuous visibility into your control health.

But if AI gives us a genuinely better understanding of our security posture, how should that change your legacy GRC program? I’m throwing to you, Matt, first. What’s becoming antiquated? This references what you just said at the beginning of show. What becomes antiquated that we should simply stop doing of our traditional GRC?

[Matt Southworth] I feel like this is somewhere that I’ve always struggled is what we do here, measure risk, and how do we talk about that? What I think we can stop doing is questionnaires, everything that our vendors and that our partners are looking for from us because the truth is those are going to be filled out by an LLM, whether we tell our vendors that or not.

For that to work and for us to stop doing the busy work, we need to make sure that we are doing a better job documenting incidents, documenting controls and just keeping a corpus, it doesn’t have to be well-organized, of information about our program so that we can still answers and new information.

[David Spark] All right, Saket, same question to you. I want to know just high-level, what should you stop doing with your GRC program?

[Saket Modi] I think the first piece is GRC has three letters, governance, risk and compliance. I think let’s start focusing in all three, if not equally, at least giving them that degree of priority to understand that there is a risk, not just compliance.

So to what Matt was mentioning very rightly, that when you do questionnaire and when you’re very, very compliance-led, to say, “Hey, is this something compliant? Is it checking the box?” that is not really risk management. How do you actually get to the bottom of real data, real telemetry?

And there’s enough sources today where you can pool telemetry from and then compile that together, not just to see how compliant you are, not just seeing what is the control maturity in your environment, but that compliance and controls can lead to what are your business risks and going ahead and tracking that.

So as we say, start with the why. The “why” has to be we have to do risk burn down, not just achieve compliance. So that is what we have to stop doing, being more very compliance focused. We have to go and change the conversation to be very risk focused.

[David Spark] Have you had to do that shift? Because we’ve heard this line many times before, nobody throws stones at me, compliance does not equal security. But at the same time, most businesses know that if we don’t do this, we’re going to be automatically fined.

I mean, can you make that shift or is it just… I mean, it sounds like it’s just like it’s a cost of doing business.

[Matt Southworth] It is, but it’s also a little bit of an opportunity sometimes. If you’ve got a compliance program, you’ve got strategic goals. You can connect them however loosely and drive where you actually want the program to go with a little veneer of compliance on top of it.

[Saket Modi] Yep, and I think compliance is a subset. Nobody’s saying don’t do compliance, but think of it like this. 99.9 % of financial services companies, which have been hacked in the last 10 years, have been compliant to PCI, to the popular stuff.

They still get hacked. What does that show? It means that you need something beyond compliance. So you’re not saying don’t do compliance, but that’s the floor, not the ceiling of how you manage risk.

Do you trust this LLM?

6:02.150

[David Spark] “Who is responsible when an autonomous agent makes a mistake?” I love this. Ritu Jyoti argued in a piece on CIO.com that because AI agents lack legal personhood, responsibility must fall on the humans who deploy them. So we need clear ownership, revocable credentials, and audit trails for AI.

Now, we don’t always blame humans for making mistakes, we expect it, but agentic AI will make mistakes, too, probably at scale. So should we need clear ownership, revocable credentials and audit trails for AI. Now, we don’t always blame humans for making mistakes.

We expect, but agentic AI will make mistakes too, probably at scale. So should we treat AI errors the same way we treat human errors or fundamentally different? What does accountability even look like when an autonomous agent screws up? So I’m going to start with you, Saket.

The blame game could be endless here. We know this. And more practically, how are you building governance today that can actually trace responsibility when things inevitably go wrong?

[Saket Modi] I think the way you look at that is, firstly, David, this depends on how you define a mistake. There are different degrees of mistakes.

[David Spark] Yeah, but more of the fear is the speed that these mistakes could start happening.

[Saket Modi] That’s right. And that can be a difference between somebody very smart in your team doing things very quickly versus somebody who takes two weeks to do the same thing. What I’m trying to get towards is if you thinking of Gen AI agents as autonomous human beings and treat them like identities, I know this is like a Pandora’s box, but the reality is that, given the capabilities of these AI agents that we’re talking about, they have to be treated in a similar way, if not exactly the same way, of how we look at human identity.

And the moment you look at that, you’re talking about the guardrails, the identity and access controls, you’re talking about the privileges that they have access from a data access perspective, what can they see, what can they act upon. So again, the first principles remain the same because it’s human intelligence or artificial intelligence.

Intelligence is still intelligence and that needs to be guarded with the same degree of governance. It’ll be slightly different, but not entirely different, in my view.

[David Spark] We got applause on that. Someone really appreciates that.

[Saket Modi] I’m glad somebody liked my answer.

[David Spark] All right, hold it. Before I toss to you, Matt, one of the things that we have heard often is maybe treat these AI agents as a brand new employee. You can’t assume a new employee is going to know all your systems and not only that, you’ve got to create guardrails around them.

You can’t have them running wild in your environment. AI has a little bit of a semblance of that of treat them like a new employee, but a new employee that could be crazy efficient or cause a lot of damage really quickly.

[Matt Southworth] Yeah, I think a new employee and intern’s a good perspective. Things we’ve learned in working with these is you can’t use an agent where you can never accept a wrong result. They are going to make mistakes or if that’s assigning too much intelligence to them, they’re going to give you unexpected outcome.

The other thing we’ve learned the hard way is bugs don’t stay fixed. We think we resolve something, we updated the prompt, it’s repeating the same mistake. So just like a new employee or a dumb intern, sometimes they get set in their ways and you need a human to review and take action upon what the results look to be.

I love your point around the identity or the non-human identity of these agents. It’s a very important place for control. But that comes after the human thinks about the process and thinks about the business goal they’re trying to solve.

[David Spark] Let me just skip to a big concern a lot of people have. What is your number one? Just please, isolate it to one thing. What is your number one concern around AI?

[Saket Modi] The known, unknown ways that AI can act.

[David Spark] Just the known, unknown? I would think the unknown, unknowns.

[Saket Modi] So the reason I say know, unknowns is because I’m saying the known, unknown is very large. So again, that’s a subjective debate, but the reason I say that, if you go back to AlphaGo, that’s the movie very popularly made where for the first time, an AI system defeated the world’s greatest Go player.

What was scary and the most scary part of that was something called Move 37, which was the fourth game, where AI makes a move and that move, not only by his opponent, which was a human being, the world champion in Go, but pretty much everybody who was giving the commentary said, “This is the most stupid move somebody can ever make.

Why would somebody do that?” And everybody thought that AI has lost. What ended up in that game, AI won. So what scares me is not that, oh, there’ll be new ransomware identities or there’ll be new phishing attacks. That’s the known side of the problem.

The unknown ways by which AI can reason and almost say this is the way to infiltrate systems, organizations, governments, identities, that, in my view.

[David Spark] That’s a really good point because one of the jokes that I used to always make about the RSA conference, I used to call it the scare the crap out of me festival because every year I’d start talking to people and I’d be one of those like, “What?

They can do that now? No, really?” And so what you just described is it’s that at a pace we’ve never seen before.

[Saket Modi] Totally.

Sponsor – Safe Security

11:40.709

[Voiceover] Who’s our sponsor this week?

[David Spark] Our sponsor this week, it’s Safe Security, Saket’s company. They are the autonomous cyber risk management company. Now, today’s security leaders face more risk, more vendors, and more expectations and have outpaced the tools we use to manage it.

Security leaders are buried in dashboards, heat maps, and disconnected data across business units and vendors. The board wants financial clarity. The business needs speed. Security teams need to focus on the risks that truly matter. That’s where Safe comes in.

Safe is the category leader in cyber risk quantification and the first platform to unify and automate cyber risk management across your enterprise, third parties and AI. Built on the FAIR framework, Safe uses a fleet of specialized AI agents to continuously assess exposure, quantify risk in dollars and automate everything from onboarding to board-ready reporting.

Now, if you’re ready to scale with less manual efforts, stronger board and business alignment and real-time visibility, you’ve got to go visit their site. It’s safe.security for the future of cyber risk.

It’s time to play “What’s Worse?”

13:04.883

[David Spark] All right. For those of you who are fans of the CISO Series and have heard the show before, this is a game we’ve been playing since we first started the show many, many years ago, in fact, well over seven years ago. All right. So the way it works is we have two scenarios.

They’re sent in by listeners. This comes from Eric Bloch of Illumio. And they’re two crappy scenarios. You won’t like either one of them, but it is a risk management exercise. You will not be able to pull out any quantification here to do it. You’re going to have to do it on the fly, all right?

So let’s figure this all out. You’re going to answer first, though, Matt, and you’re going to agree or disagree, Saket. Here we go. What’s worse? And by the way, audience, we’re going to want your feedback too on this as well, so come up with your answer as well.

I’ll ask for applause on that onto what you think is what’s worse. And remember, it’s what’s worse, not the one you prefer. Scenario number one, you pay a million dollars a year for a custom LLM co-pilot for your SOC analysts, but it is a black box and no one trusts it.

So they still escalate 90% of the alerts. So it’s kind of useless. You’re just flushing a million bucks down the drain. Or you don’t invest in AI at all and onboarding new analysts takes nine months and 100K in training and turnover costs. Which one is worse?

Matt.

[Matt Southworth] Both terrible, right?

[David Spark] They’re both awful. Yeah, you have to tell me which one’s worse.

[Matt Southworth] So which one’s worse is the second option, no AI, just onboarding humans because at least with a broken million-dollar budget, I can do something with that. I think there’s an opportunity there to drop the existing vendor.

[David Spark] It’s going to probably burn out your staff because you’re still escalating 90% of the alerts.

[Matt Southworth] That’s going to happen in either scenario, so at least I’ve got a million dollars to play with here.

[David Spark] Well, you may not have that many alerts. And in fact actually in the other scenario because you don’t have AI escalating anything.

[Matt Southworth] Sure. Sure. So at least if I’m hiring analysts that are willingly walking into an environment with zero AI tooling, I’m probably hiring the wrong people.

[David Spark] All right. I’ll throw this one to you, Saket. Which scenario is worse here? You pay a million bucks a year for custom LLM, black box, no one trusts it, escalate 90% of the alerts or you don’t invest in AI at all and onboarding new analysts takes nine months to 100K a year.

And by the way, the nine months of onboarding, that means they’re essentially ineffectual for most of the year.

[Saket Modi] Yeah, well, the moment you said you don’t invest in AI, my natural option was number of what Matt selected. So I agree with what Matt said, but for a different reason. The reason is very simple that it’s like any product in the world. Do you know this, when iPhone was launched, it did not have cut, copy paste?

[David Spark] Really?

[Saket Modi] Yeah. The first two years, iPhone did not. You could not cut, copy, paste on your iPhone. And look, today, what is the iPhone? The same way, every single product, even if today it has 90% false positives, it is bound to be better. How much better, it depends on the execution, but I’m very confident if it’s 90 today, and if it’s any company worth their salt, it’ll be 90 to 85 to 80, et cetera, et cetera.

So the accuracy, even if it’s a black box, will keep increasing and that’s a way better place to be than investing in humans, which again can be turnover and they churn.

[David Spark] So that is a really good point. So even though the first scenario stinks, it’s awful, at least you’re starting in the right direction, where the second scenario, you’re not even beginning.

[Saket Modi] Totally.

[David Spark] What do you think of that, Matt?

[Matt Southworth] I think he’s a very bright guy. I agree with him agreeing with me.

[David Spark] All right. By applause, how many people agree with them that the second scenario is far, far worse to just invest in the people and not do AI at all, by applause? I most agree with you on that. By applause, how many people think the first scenario where you’re spending a million a year and the AI is just causing more havoc by giving you a ton of alerts for you to go chase?

By applause, how many people think that is worse? No one does. So everyone agrees with… oh, no, wait, one person, one person. The outlier. We have to have an outlier. Thank you so much. I appreciate it.

What are these security pros talking about?

17:30.092

[David Spark] All right. Okay, so we have this new game for you and you’re going to like this. We have interviewed a lot of people recently at the HOU.SEC.CON. conference and the RSA conference and we asked them a series of questions. And what we have here is three to four, maybe even five people, answering the same exact question.

I’m only playing the answers. You have to figure out what the question was, okay? All right, here you go. Audience, if they can’t get it, I’m going to toss to you to see if you can figure it out.

[Voiceover] Zero incidents.

[Voiceover] Protecting your data everywhere it goes.

[Voiceover] Not having to deal with the product anymore.

[Voiceover] Being able to help security teams show that what they’re doing has impacts. 4

[David Spark] All right, either you can guess what the question was they were all being asked. Those were all the answers. Any idea?

[Saket Modi] Spoke about the ROI of security controls. I’m biased. I think it’s CRQ, cyber risk quantification.

[Matt Southworth] I think you’re asking people to come up with the slogan for their new company.

[David Spark] No, definitely not that. Let me play one more time.

[Voiceover] Zero incidents.

[Voiceover] Protecting your data everywhere it goes.

[Voiceover] Not having to deal with a product anymore.

[Voiceover] Being able to help security teams show that what they’re doing has impacts.

[David Spark] I will admit, this is the toughest of the four questions. We’re starting out hard. Any idea what do you think this is? Okay, I throw this the audience. Just literally yell it out what was a question that these were the answers? Anybody?

[Voiceover] What keeps you up at night?

[David Spark] No, no, not what keeps you up at night.

[Voiceover] What do you wish you can have in cyber?

[David Spark] What do you wish you can have in cyber? That is close.

[Voiceover] The promise of AI.

[David Spark] No, not the promise of AI. I’ll give one more and that I’ll tell you what it is.

[Voiceover] Board reporting.

[David Spark] Board reporting? The question was what is the cybersecurity equivalent of a home run? All right, I’m going to give you another one. That was the toughest, I’ll be honest.

[Voiceover] Those users that may fall into the same trap again and again.

[Voiceover] People asking for local admin access.

[Voiceover] Pushing from the vendors to get deals done when it’s not important to the customer.

[Voiceover] People skip over learning networking and go straight to buying certs.

[David Spark] Any idea what that question was?

[Matt Southworth] Sounds like what are the biggest mistakes that your coworkers make?

[David Spark] That is extremely close, but no. Saket, you want to try on this one? Remember that he was close, but no.

[Saket Modi] No, I think, yeah, what’s been the biggest learnings in your career of cybersecurity professional.

[David Spark] Not exactly. Here, I’ll play it one more time for you.

[Voiceover] Those users that may fall into the same trap again and again.

[Voiceover] People asking for local admin access.

[Voiceover] Pushing from the vendors to get deals done when it’s not important to the customer.

[Voiceover] People skip over learning networking and go straight to buying certs.

[Matt Southworth] Biggest time wasters.

[David Spark] Want to try again?

[Saket Modi] No.

[David Spark] Anyone want to guess on this one what was the question?

[Voiceover] Pet peeves.

[David Spark] Pet peeves, yes, correct. It is pet peeves. All right, we got two more. Let’s try this.

[Voiceover] I turn off Wi-Fi and Bluetooth on my device. Sometimes I turn my device off.

[Voiceover] Leave my devices at home is the first thing. And don’t go online.

[Voiceover] Other than hiding my badge so that vendors don’t scan, stay conscious from an electronic perspective.

[Voiceover] Hold on to your stuff and put your phone in flight mode.

[David Spark] What do you think that is, Saket? What’s the question?

[Saket Modi] Yeah. Well, how do you stay protected while you are with a cell phone?

[David Spark] Close, not really.

[Matt Southworth] Sounds like what’s your conference operational security?

[David Spark] Yes, that is 100% correct. Good job. All right, very close. Very good.

[Matt Southworth] I do none of those things, by the way.

[David Spark] Last one, last one. This one, I think you’ll get.

[Voiceover] Shirts, apparel in general, knickknacks, a backpack.

[Voiceover] Whiskey cubes.

[Voiceover] A good multi-plug, especially one that goes into an AC plug with all the different multi-plugs.

[Voiceover] The cell phone holder, of all things.

[David Spark] All right, what was the question?

[Matt Southworth] What’s the swag you leave behind in your hotel room when you’re going home?

[David Spark] Actually, no, no, no, that’s not the answer. You’re close, but you have the opposite answer. Saket, take it away.

[Saket Modi] What’s the swag that you get at an RSA conference shop floor?

[David Spark] Well, it’s essentially what swag do you still have? What swag do you still have? So it was the opposite of what you were saying.

What’s the starting point for a CISO?

22:12.630

[David Spark] All right. “Likelihood is 50/50—either it happens or it doesn’t. Impact is super critical—my asset is the most important thing on God’s green earth. Response is accept, since I still don’t give a damn.” That’s how one frustrated security manager on the cybersecurity subreddit described their staff’s approach to risk management.

They’re starting from near square one, no risk culture in place, stakeholders making wild guesses on risk. The advice they got ranged from, “build a simple impact/likelihood table in Excel,” to, “you need process before tools,” to, “just use historical incident data.” I’ll start with you, Matt.

When you’re starting a risk management program from scratch or inheriting what amounts to security theater, what’s your first move? Do you start with the spreadsheet in workshops, invest in a tool to force structure or focus entirely on building risk culture before touching any frameworks?

What do you think?

[Matt Southworth] I doubt I have the latitude to build a risk culture over five years before showing any results, but definitely process over tools. And I don’t hate the idea of workshops and brainstorming, finding the people who understand the business and grilling them on what’s important.

Everyone thinks their world is best place on God’s green earth, but narrow it down and then talk about it from a business perspective, not even a risk perspective. Focus on what they already understand.

[David Spark] I like it. All right. This is right up your alley, Saket. Where do you start? You want to do risk quantification management, but, I mean, so many places to start. Where do you begin?

[Saket Modi] I’m going to add to what Matt said because I totally agree with him. And I’m going to add one more variable. It’s actually people, process and then technology. You start with people because if you don’t have…

[David Spark] So building the culture first?

[Saket Modi] You don’t build the culture? I actually feel that there are companies which are very risk driven and then there are companies who doesn’t care about the risk at all. You can’t go as a CSO or a CI or a risk manager and change the mindset of the leaders of the company.

So it’s just the reality. That’s where you start with people. And if there is appetite to understand that there needs to be risk that needs to be managed, you start with processes, basic ones and then you go ahead and invest into tools in technology.

Technology is not the answer.

[David Spark] No, and as we know, like we talked about that, if you try to implement technology, you fix a problem, all you’re doing is going to take whatever problems you have and make them go a lot faster, correct? That’s assuming you don’t have the process in place, like you pointed out.

[Saket Modi] I agree.

[Matt Southworth] Saket, how long do you think you should ask for to show some results?

[Saket Modi] So a couple of points. Even before you show the results, the way you calculate the results is important. So you want to start with that to say, look, this is the first phase where even before I show or I’m burning down the risk, the way I calculate the measurement of risk is important.

And that is the reason why we see that we come in very early stages on the maturity cycle, where a company wants to start their journey in understanding and making a risk based decisions and getting the instrumentation right. And once that is in place, again, it depends on the speed of the company.

When we are working with technology companies which are software developers, they move very fast in burning down the risk when they see there’s something critical. On the other side, there are companies which are very, very comfortable saying, “Hey, we’ve never got hacked.

We don’t think we’ll get hacked.” And they’re okay with that. So as I said, it’s the mindset of the people where that is really where rubber meets the road. It’s not the technology. It’s not the process.

[David Spark] So question about process. What are some kind of key processes you need to have in place before a tool enters the picture?

[Saket Modi] I think the big thing is, again, process is a very heavy and a loaded word.

[David Spark] Yes.

[Saket Modi] I think something like what they say about culture, the culture of the company is the behavior of the founder and the CEO and that really gets reflected across the company on what is considered as normal. The same thing is applicable for risk.

The way the CEO and founder looks at risk and how do they manage risk, there’s some people who will just wing it and say, “Hey, we’re good and we’ll do that.” No matter what process you have in place, it doesn’t work. On the other side, there’ll be CEOs, I know, for example, there was a big town hall meeting with Bank of America.

And one of their staff told their CEO that because of this crazy multi-factor authentication that we have, we’re actually losing business. And this was on a town hall. And Brian, their CEO, actually said, “Guys, I want to make this on record. I don’t give two cents about the business.

Even if we lose business, it cannot come at the cost of cybersecurity.” When the CEO of your company makes that statement in the town hall…

[David Spark] That’s a big, big statements.

[Saket Modi] These were the exact words and this was actually published on a few threads, so that’s why I read it and I found it out. But the reason I’m saying this to you is that that is the moment you know that that is a company ready to be managed by risk and no doubts, Bank of America is the gold standard in a lot of ways.

[David Spark] And I’m sure you’ve had situations where a lot of sort of a sales process of this is just education of the platform.

[Saket Modi] Not of the platform, of the risk that they’re actually sitting on because a lot of times, yeah, because people don’t care about platforms or tools. What they care about a lot of times is that, hey, I didn’t even know if something goes wrong, it can cost me $50 million.

The moment they understand that, they don’t know that they are worse.

[David Spark] Right, because it does look like a black box.

[Saket Modi] It looks like a black box and like, “Hey, it’s not going to happen to me,” until it happens.

[David Spark] Well, until it happens. And it’s one of those… yeah, it’s because it comes out of nowhere.

[Saket Modi] Totally.

[Matt Southworth] So before process or tools, we have people. How do you identify within the organization, the people who substantially own the risk, whether it’s in their JD or not?

[Saket Modi] The owners of the risk are the owners of businesses. So if you’ve got five businesses, the CEO or the business unit head is the one who owns the risk. This is a general misconception. The GRC team, the CSO does not own the risk. The job of the CSO is like the job of the CFO.

The CFO doesn’t own the business of each business unit. The job of the CFO is to show the mirror to each business of how well/not well are they doing. That’s exactly the job of the CSO to say, “Hey, this is the risk you’re sitting on. It’s fine if you want to be okay with that, but this is how you can burn it down in the most effective way possible.” And that’s the core job of the CSO.

So to your answer, Matt, who owns the risk? It’s always the business owner.

[Matt Southworth] And those people is who you need to invest your time with.

[Saket Modi] Yeah.

What would you advise?

29:08.372

[David Spark] “We spend millions on EDR and firewalls, but our real weak point is a 10-minute phone call to a tier one agent.” A debate on the cybersecurity subreddit had some arguing for eliminating human touchpoints entirely, for example, for self-service password reset with MFA or in-person resets only.

Others pointed out that help desk agents are underpaid, outsourced, graded on ticket velocity and incentivized to be helpful above all else. And some pointed to a middle ground, callback verification with managers and identify assurance tools. One commentator tried to find a silver lining saying, “If most user requests are automated and go through some nasty ServiceNow UX, then most requests are protected against social engineering.” So where’s the line?

And I’ll start with you, Matt. When does removing human judgment from the help desk actually improve security and when does it just create a worse user experience that people will find a route around?

[Matt Southworth] I think most people love self-service and would rather click a couple buttons and get their problem solved and not pick up a phone, at least as a basement dwelling indoor cat, that’s me. 10-minutes before we started, my boss called me to ask me about a fishing email.

You’re the CFO. You don’t need to worry about this, but some people need that human conversation because they’re going to have follow-up questions. So answer your question, as much as we can give guardrails to a bot to do things automatically, that reduces the attack surface, as long as you’ve tested and you know its constraints.

Humans are always going to want to escalate, ask a question, have a follow-up. And I think the solution there is have those frontline help desk folks feel like they’re part of the security team. Have them a little bit empowered, but heavily drilled and tabletop about what could go wrong, send them the worst -case scenarios and let them express human judgment.

[David Spark] That’s actually a good point. We talk about tabletop, but tabletop for the tier one as well.

[Matt Southworth] Oh, absolutely. Yeah, that’s probably more important than me. They need to see it all the time. And it’s good cross-team relationship building. If the head of security engineering sits with the help desk and walks in through scenarios, that’s great for both sides.

[David Spark] Saket.

[Saket Modi] Yeah, the challenge there, Matt, let me challenge you there, is that when you talk about a few hundred people, that’s still okay, but when you’re talking about a help desk for an insurance company, which has probably 5,000 people and doing that for every single person in a scalable way, generally you only do it with videos and these video awareness, but that clearly doesn’t work.

So that can be a challenge.

[Matt Southworth] Absolutely. If it’s an organization at scale, if it’s a customer service org where you can only give one answer then yeah, you’re stuck, but what happens when the CFO calls that customer service org? You’ve got to have some sort of executive care escalation for them.

[David Spark] But can’t you do some kind of period phishing test for your service? Yeah?

[Saket Modi] I actually think period phishing tests are actually now more expensive than just giving a simple AI bot because if you think about it, again, replacing the agent with a AI bot, which a lot of companies are trying to do will still take time.

If it would have been solved, there would be no contact center agents today itself. So it’ll still take time, but recording a call which is already happening, looking at what is being told and actually flagging that, “Oh my God, this looks like a scammer.

Oh, my God, this looks like somebody’s trying to fool us,” and giving a big red sign on the screen and that’s easy. That’s not very difficult. I think those are the measures. Again, AI to the rescue. I actually think that would be the thing which will change that industry and that problem.

[David Spark] Is this line clear or is it very blurry, I’m assuming blurry, where you sort of cross the line of making things too difficult for people or making it too permissible where tier one agents can fall victim? What do you think? Do you know where it is?

[Matt Southworth] No, I don’t know where it is, but I’d actually step back and look at the incentives on both sides. The people who are calling out, do they have incentive to gain the system? Are they trying to get something from nothing? Are they trying to get some free product or something?

Then you’re going to tighten things down. On the other side, how are you incentivizing the agents? Are they compt because of the number of tickets they close? Are they compt because the person calling got the outcome they wanted?

It’s time for the audience question speed round

33:49.564

[David Spark] All right, we are coming to the end of the show here, but before we close up, I have questions from you, the audience, right here on these index cards. And with what little time we have left, I’m going to get through as many questions here as possible, so quick answers to all of these.

You have not seen these at all. They don’t know what these are, so they’re going to be surprises for both of them. All right, either one of you jump in here. When you see data or a dashboard… this, by the way, comes from Greg Wham of Trend Micro. When you see data or a dashboard, what questions do you ask of the data to see if it is actionable, meaning should I act on this?

[Saket Modi] Yeah, a simple thing, I like the data behind the data. So you’re asking a simple question of so what and who cares is a very, very simple litmus test to say, is this even useful?

[Matt Southworth] Yeah, I think this morning I asked, am I going to do anything different with this data? And if the answer is no, I’m going to stop looking at it.

[David Spark] All right, good quick answers. All right. From Henry Stanley of Fabrik, who asks, what incentives can we provide to get suppliers to share their security and assurance data? Is there something we can do to make them go, “Oh, yes, I want to give you this data?”

[Matt Southworth] So we tell a story about how we don’t compete on security, how we’re all in this together and then we hug at the end of the day and we buy them drinks at Black Hat, but we don’t share our data. Exactly. So what’s the incentive? That’s a question.

[Saket Modi] There are only two dimensions, time or money. You give back time by saying, “Hey, I’ll make it easy. Just drag and drop this and you’ve got it. No more wasting time on that.” Or you give discounts or you say, “Hey, I’ll pay you more.”

[David Spark] So it’s money or time?

[Saket Modi] Yep.

[David Spark] That’s a good way to putting it. All right, I like that. All right, from Paul Love of Delta Airlines asks, I like this, for both of you. I’ll start with you, Matt. Give me just one storytelling technique you found successful when you’re talking to the board.

[Matt Southworth] I fall back on using eye statements. I try not to talk about anyone but me and what did I do, what did I feel in this situation.

[Saket Modi] That’s a simple one. Use Safe.

[David Spark] That is good. I like that. All right. From Ayoub Fandi, with the GRC Engineer Podcast, how do you build a product when everyone thinks there’s a snowflake? This is definitely for you, Saket.

[Saket Modi] Well the fun part of the company snowflake, that’s the reason why you’ve got a hundred billion dollar company. And I think, again, thanks to the flexibility of the world of agentic AI, you actually can because the old way of doing things when every company is a snowflake is you needed 200 clicks.

And that’s why the user experience in one of your question it says shitty user experience of almost every B2B product. That is changing thanks to AI. So I actually think it’ll be much easier in the world of AI to build a great user experience while having something very personalized.

[David Spark] So the answer for all the snowflakes, more AI.

[Saket Modi] Yes.

[David Spark] All right. In fact, I’m going to give you a follow-up question, Saket, from your colleague over at Safe Security, Megan Maneval. She wants to know what was your “aha” moment to start Safe. Did you have an “aha” moment?

[Saket Modi] Well, I was in college and I didn’t know if I would be employed, so I started this in my third year. It’s been a good ride.

[David Spark] Okay. So it was more of a fear. It’s like, “Is anyone going to hire me?”

[Saket Modi] Well, I never applied anywhere, to be honest, but then I was like, “Yeah, I guess people do something in their third year, so why not do a startup?” And that’s what happened.

[David Spark] All right. This is last question from Jennifer Chui of the Alliant Credit Union for both of you. Start with you, Matt. What are the pitfalls of a CRQ program, cyber risk quantification program? What are the pitfalls?

[Matt Southworth] I think it’s just garbage in, garbage out. It’s where is your data coming from? And the biggest problem I’ve seen is with complete list of data. You have some data. It’s accurate, but does it tell the whole story?

[Saket Modi] Well, Matt, I shared three learnings of the problems of CRQ programs. You took the exact center one and the right one. I literally had on this screen garbage in garbage out, which is one of the biggest problems, so I totally agree with him.

I think the big problem in my view, a bigger problem in my view, because you’re talking about when the program is being implemented, but I’ll tell you why programs are not even starting. That’s because the biggest problem of CRQ is the name itself.

If you ask anybody, “Do you need quantification?” “No, I have dashboards. I have numbers.” No, but the moment you understand the value that that delivers, do you want to know if something got hacked, are you vulnerable to that, how exposed are you to that, something crowd strike detected, how does that change my risk posture, how do you compare with your peers, how did your risk burn down with your budget, these are value.

Everybody wants to know that. So CRQ itself, if you just say CRQ, it’s not useful, is what most people think, especially the smart people because they don’t want to double click there.

[David Spark] So the pitfall is missing the value?

[Saket Modi] Is the name itself.

[David Spark] Oh, so you’ve got to rebrand it. It’s your fault now, Saket.

[Saket Modi] Decision intelligence. CRQ was branded before I was born, probably. So that’s the problem.

[David Spark] So you’re not going to take the blame here?

[Saket Modi] We go to decision intelligence. We like that better.

Closing

39:07.390

[David Spark] All right. Well, that brings us to the very end of the show. I want to thank our guest here, Saket Modi, CEO of Safe Security. Let’s hear it for Saket. Also, Matt Southworth, who’s the CSO of Priceline. Let’s hear it for Matt. And huge thanks to our audience, thanks to SAFE, thanks to the FAIR Institute for bringing us here.

And also remember, Safe Security, autonomous cyber risk management reinvented with agentic AI. We are thrilled that Safe sponsored us for this. Remember, go to their website, safe.security. As always to our audience, those here and those listening to us right now, we greatly appreciate your contributions and for listening to the CISO Series Podcast.

[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. We have lots more shows on our website, CISOseries.com. Please join us on Fridays for our live shows, Super Cyber Friday, our virtual meetup, and Cybersecurity Headlines Week in Review.

This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at david@CISOseries.com.

Thank you for listening to the CISO Series Podcast.