At some point, all the hype around AI has made it hard to identify meaningful innovation. In a space where everyone can’t stop talking about how they are integrating AI, how do we find what’s worth our attention?
This week’s episode is hosted by me, David Spark, producer of CISO Series and Jerich Beason, CISO, WM. Joining us on stage is Jack Leidecker, CISO, Gong. This episode was recorded live at HOU SEC CON 2025.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, Vorlon Security
Full Transcript
Intro
0:00.000
[Voiceover] Best advice I ever got in security. Go!
[Jack Leidecker] The best advice to me initially started as the worst. So, I had a previous boss who wanted us to fix our phishing issues, and we’re going to fire everyone after they get phished three times. I had to say, “Actually, we would end up firing you.” But more importantly, it helped us reevaluate what we wanted to do for phishing, which is changing behavior, and if someone thinks they’re going to get fired, they’re not going to tell you, they’re not going to report.
[Voiceover] You’re listening to CISO Series Podcast, recorded in front of a live audience in Houston.
[Applause]
[David Spark] Welcome to the CISO Series Podcast. My name is David Spark, I’m the producer of the CISO Series. And to my immediate left is my guest who was here last year when we did the show. It’s Jerich Beason, the CISO for WM. Let’s hear it for Jerich!
[Applause]
[David Spark] Say hello to the audience, Jerich.
[Jerich Beason] Hello, audience.
[David Spark] That’s what my other cohost, Steve Zalewski, says it just like that. In fact, people have come up to him and said, “Hello, Steve.” Just like that he introduces. By the way, we are recording live at HOU.SEC.CON, which by the way, I pronounced incorrectly, I did like HOW.SEC.CON.
or something stupid like that. I got definitely admonished for that, but we’re at HOU.SEC.CON. Let’s hear it from the HOU.SEC audience!
[Applause]
[David Spark] And we are available at CISOseries.com, where all our wonderful programming is. Our sponsor for today’s episode, the reason we are here in Houston, Vorlon Security, enterprise SaaS security that’s light years beyond the legacy SSPM tools, more about that later in the show.
Let’s hear it for Vorlon for making us come here to Houston!
[Applause]
[David Spark] All right. Before we begin, one of the staples of the CISO Series is a game we play called “What’s Worse?” which we will play today, and as I understand, you play a version of it at your office. Some of your staff members are here, Jerich.
How do you play the game differently?
[Jerich Beason] All right. So, we call it “Pick Your Poison” because, you know, copyright, and what we do is we have a scenario, different people on our team provide that scenario, and the team is divided on if they go with option A or option B, but their goal is to pick the option they think that I would want to take in front of the board.
That is the mindset that they have, and it helps them learn who I am, how I am, and a lot of decisions are made now without me having to be in the room because we play that game.
[David Spark] That is awesome. The fact that they can essentially think like you and… But now, but hold it. How do you then, if they’re all thinking like you, how do you get that very diversity of sort of understanding of cybersecurity incidents?
[Jerich Beason] So, one, usually the majority is wrong.
[David Spark] Okay. [Laughter]
[Jerich Beason] And for me, that’s actually best because it’s a learning opportunity, and at the end of the day, incidents are a shared responsibility. So, I’m going to still find out about everything, but when we have to make risk-based decisions, they can say, “Well, what would Jerich do?” It’s another version of WWJD.
[David Spark] I like it. All right, let’s bring on our guest. He’s to our left. You heard him at the very beginning of the show. He’s a CISO for Gong. Let’s hear it for Jack Leidecker.
[Applause]
[David Spark] Say hello to the audience, Jack.
[Jack Leidecker] Hello.
[David Spark] That works for me.
What’s the motivation to do this?
3:24.139
[David Spark] Open source devs of the world unite? “It is long past time that maintainers stopped letting them (being the organizations) take advantage of their (the maintainers’) good nature.” Now that’s Justin Warren of Pivot9, arguing that organizations have developed a parasocial relationship with open source software.
It’s seen as a naturally recurring resource rather than human labor requiring support. His solution? Listen to this. Maintainers should go on strike, stop rushing to fix things for those freeloaders, and patch security flaws at a leisurely pace, unless someone’s paying.
All right, Jerich, do you agree organizations are taking advantage of open source’s goodwill, and if so, can this continue, because it has been, or should something be done? I mean, Warren’s suggestion is pretty dramatic.
[Jerich Beason] Yeah, I would agree that most IT shops with some type of development function are definitely heavily dependent upon open source, and it is a disproportionate level of consumption versus contribution. That being said, I would look at that parasocial concept a little bit differently.
It’s not completely one-sided. Large organizations are contributing code, testing, in some cases, even funding, but by and large, the majority of organizations are more consumption than contribution. This whole concept of striking, I think that could be catastrophic.
[David Spark] Yeah, it sounds pretty rough.
[Jerich Beason] Log4j, XZ Utils, maybe a year ago, showed us that a small set of maintainers can have impact over millions of systems. And what we learned when we thought MITRE was going to get defunded is that companies are willing to pay if they’re going to lose their CVE database.
I think that’s the blueprint that we follow. I would agree, ultimately, that the status quo is not working, but the solution should fix the ecosystem, not weaponize it.
[David Spark] That is a good closing line there. All right, Jack, I throw it to you. What’s your feeling on this?
[Jack Leidecker] Yeah, no, I mean, I think it’s a little messy, right? Because on one hand, people want open source so they can have transparency, see what they can do. It is hard when you’re not being paid for it, but I also think a lot of the volunteers want to be able to be part of a project, but I think there does need to be a bit more of a balance, right?
How do people contribute to it? Are companies being able to sponsor it? I think if you move straight into commercial, it kind of defeats the whole core of what open source was, right?
[David Spark] Right. I mean…
[Crosstalk 00:05:46]
[Jack Leidecker] Which is, we want to share, and we want to do it.
[David Spark] You’re a commercial software, it’s something different.
[Jack Leidecker] Exactly. And even a lot of the people that are part of the project really don’t want it to go commercial, right? Because they got frustrated with vendors not being able to actually update it correctly, being able to change it. So, it’s being able to find what that balance is, but I agree, like striking isn’t the right answer.
[David Spark] No.
[Jack Leidecker] But I do think whether it’s fundraising or pushing a bit more or saying, “Hey, if we can’t get this,” like you said, with MITRE, you can get something without having to go full commercial, which to me kind of defeats the purpose of open source too.
[David Spark] All right. So, what is the happy balance here? And my feeling is a lot of times people do this for the recognition and the love of it. So, is there a better or a different recognition we can offer? What do you think, Jerich?
[Jerich Beason] I think the Wikipedia model might work, right? Like we consume Wikipedia, and we know that when we go there, they say, “Hey, if you want to help us stay on, you can pay,” right? I think people even having an opportunity and a path to contributing would be helpful, but if you create a more commercial model, SLAs come along, structure comes along and expectations come along, and now we’ve lost the spirit of what we’re trying to accomplish.
[David Spark] Have either of you hired someone who supported open-source software?
[Jerich Beason] If you have a developer, you have someone that supports open-source software.
[David Spark] Okay. That answered that question. All right.
[Jack Leidecker] And I would say I’ve actually had some that were very involved in some projects, and it was actually interesting because they supported some Security Onion work and other things in the past. So, there’s a benefit from that perspective.
It helped them get experience, but it is a bit of a different model, right? As Jared mentioned, I think if you go commercial, SLAs is going to kill it.
Is AI going to help us, or hurt us?
7:15.410
[David Spark] “Almost two-thirds of IT leaders agree that generative AI challenges the geopolitical status quo, allowing smaller nations and non-state actors to emerge as near peer cyber threats.” I’m quoting you, Jerich, as you were referencing the 2025 Armist Cyberwarfare Report.
With AI in the mix, threat actors no longer need deep expertise or vast resources, just intent and access. Now, something we saw initially with Ransomware as a Service, some regional conflicts that wouldn’t normally be on many companies’ radars now have global implications.
So, I’m going to start with you, Jerich, on this, since you kind of brought it to our attention. Does geopolitical intelligence need to be part of more company’s risk assessments? Because some small companies, it doesn’t even hit their radar at all.
And if so, how do you decide what geopolitical events fall into your risk profile, and how are you even tracking this once you define it?
[Jerich Beason] Man, I wrote that?
[David Spark] You didn’t write all of that. You wrote the beginning quote that I quoted you on.
[Jerich Beason] Yeah, so historically, we would look at a small set of nation-states that were well-funded, and that was the adversary we had to focus on. But now it’s not about capability. It’s about desire because with a hundred dollars, a couple thousand dollars, we’ve now lowered the bar for how you can actually enter into some type of cyber attack.
And so, now we don’t just have a few countries. We have the entire world that could potentially be targeting us. And AI is making that so much easier – as a service, as you pointed out – is making it so much easier. So, I look at it a few different ways.
There are about three questions you need to ask in your risk assessment process. One, what are the services and critical operations we have and what are the supply chain dependencies? The second one is are there any specific nation-states that are targeting our industry in particular?
And then lastly, are there any business endeavors, M&A activity, sponsorships, you name it, that could be viewed as opposing to one side of a conflict? And those questions will help you determine likelihood and impact.
And we can no longer look at cyber risk in a silo. It has to be viewed collectively and holistically. And at the end of the day, it’s not should we include this information. It’s how do we operationalize it as fast as we can because an attack that’s happening on one side of the world can end up at your doorstep if you’re considered to be on the opposite side of a conflict.
[David Spark] All right. I throw this to you, Jack.
[Jack Leidecker] So, I mean, I think we’ve been seeing this change for a while anyhow, right? I mean, as cybersecurity and especially the other side has been commercialized, right? I can go buy passwords, I can move things quickly. AI accelerated some of this.
I think it is important to understand what’s my risk, what’s going on. I think the aspect of, hey, we may not be a target because of something is something you really want to reevaluate. Because a lot of times, we’ve seen some of the smaller groups just try and make a name for themselves, right?
They may not even be going after what you expect. They want to create an impact. If you’re vulnerable, they’re able to attack you a lot faster. So, I think here is just more you need to be aware of what your risk is, how can you respond to it, and really, regardless what industry you’re in, someone’s going to be interested in being able to take you down.
You need to be able to understand what that is and how you might be able to mitigate it.
[David Spark] So, the thing is, just going to the last things you were saying there, is that I’m a small company with a small security team. I can’t be dealing with everything. Is it enough to just build up my security program and not be paying attention to the threats?
Because I don’t know if I have even the bandwidth to even think that.
[Jack Leidecker] It’s hard when you’re in some of the smaller companies, right? I think there, this is where you see lots of partners and other ones to try and help with it. But honestly, if you’re not aware of what’s going on, especially from a threat intelligence perspective and everything else, you’re going to miss what some of your vulnerabilities are.
[David Spark] What would be a bare minimum? Like we talk about this, quoting Wendy Nather, she talks about the security poverty line. What would be the equivalent of the security poverty line for threat intelligence?
[Jack Leidecker] I mean, there’s a lot of open source that’s out there, right? And I would say most of your SIEM vendors, even your EDR, have things that you can add in. It’s not super fancy. You may not be able to get, hey, I’m scanning dark web just for me, but at least being aware of what’s going on out there, right?
Because especially if you see big campaigns, they usually start off smaller. So, if there’s a big ransomware campaign that’s targeting retail, I’m a small retail business, I need to understand if the attacks that they’re using are something that I’m more vulnerable to or not.
[David Spark] All right. Same question for you. What do you think that that sort of minimum line of security awareness/threat intelligence is?
[Jerich Beason] Well, you kind of alluded to the small and medium business. Most of them don’t have large security teams because they’ve outsourced it. So, instead of you doing that work, you should make sure your outsourcer is doing that work. Threat intel is something that is not really understood, but at the end of the day, it’s all about understanding your adversary and what they’re after.
You want to know if your data, your systems would be a target, and you want to know what tactics that the bad actors are using so that you can protect yourself. And if you see a geopolitical conflict in place and X, Y, and Z group is involved, you know that they’re after financial data.
So, let’s look at our financial data. You know that they use calling to your help desk. Let’s make sure our help desk is ready. Whatever it may be, understanding your adversary is step one in understanding where your attack is going to come from.
Sponsor – Vorlon
12:37.749
[David Spark] Before we go on any further, let me tell you a little bit about Vorlon, our phenomenal sponsor. They sponsored us here last year at HOU.SEC.CON. All right. Big update for security teams thinking beyond basic SaaS security posture management.
Vorlon just redefined SaaS security with a platform that brings context and control to your entire SaaS ecosystem. Here’s what’s new. Vorlon’s patent pending data matrix technology gives you near real-time visibility into every connection, secret and sensitive data flow across your SaaS and connected apps and services.
Now you can actually see what’s talking to what, spot risky data sharing, and catch abnormal activity as it happens. Now, worried about dormant OAuth tokens, over permissioned service accounts or shadow integration sneaking in? Vorlon alerts you and helps you respond in minutes.
Revoke risky secrets, investigate suspicious events, and even automate remediation with your SIEM, SOAR, or ITSM.
Now let’s talk compliance. Whether you’re wrangling PCI, HIPAA, SOC 2, or just prepping for the next audit, Vorlon gives you audit-ready reports and evidence at your fingertips. No more screenshot marathons. Now, the best part? Vorlon’s agentless proxy free setup means you’re up and running in under an hour without disruption for instant value.
And of course, you get world-class support trusted by Fortune 500 companies and high-growth teams alike. You can see, detect, and secure your SaaS and AI ecosystem. You can get started. You got to go to the website, though. Vorlon.io, and when you go there, let them know you heard about them from the CISO Series.
It’s time to play “What’s Worse?”
14:33.815
[David Spark] All right, we are going to play again. You’ve played this game many times, Jerich, and you play it with your team many times.
[Jerich Beason] I have.
[David Spark] All right. So, you are familiar with this game, Jack?
[Jack Leidecker] A little bit.
[David Spark] All right. You get two crappy scenarios. One of them you have to deem is worse – so you don’t choose the one that’s better, the one that’s worse – and explain why. And you can disagree or agree on this, but here we go. Comes from Erik Bloch of Illumio and here are the two scenarios.
Scenario number one, you spend a million dollars a year on an LLM-powered detection and response tool that reduces your analyst workload and their burnout, but there’s no measurable reduction in risk or incident volume. So, the value you get out of it is your team is not getting burnt out or overloaded.
Okay? That’s the value, but that’s what you’re spending a million bucks on. Or you spend no money on an LLM-powered tools, relying on human analysts who are constantly overwhelmed, and they turn over every 9 to 12 months. Which one is worse?
[Jerich Beason] Oh, man, that sucks because LLM powered is a problem in itself. It’s not agent based. [Laughter] Well, I’m always going to err on the side of caring for the human, and so…
[David Spark] You’ll drop the million dollars a year to keep them alive and healthy.
[Jerich Beason] There might be some AI SOC vendors in here and none of them are million dollars.
[Laughter]
[Jerich Beason] So, the fact that an LLM-based one is a million dollars is a problem. So, I’m going to eat the fact that I’m not making a good financial decision for my company, but the humans that I care for are cared for, and so I’m going to go with that option.
[David Spark] All right. So, the business is going to be furious with you.
[Jerich Beason] They’re not going to know because I’m the security guy and…
[Crosstalk 00:16:28]
[David Spark] Well, supposing you’ve got a million dollars to spend on this. We’ll see.
[Jerich Beason] Yeah.
[Jack Leidecker] It depends on what your budget is, right? If I can hide that, then that makes it easier, but turning over people every nine months, that sucks.
[Crosstalk 00:16:40]
[Jack Leidecker] Your SOC is ineffective at that juncture.
[David Spark] But you’re probably not… Well, I don’t know. Will that equal a million dollars, 9 to 12 months?
[Jack Leidecker] Oh, easily. Yeah.
[David Spark] Yeah.
[Jack Leidecker] You’re turning over that many analysts.
[David Spark] Yeah. You’ll be running through that pretty fast.
[Jerich Beason] That’s seven analysts fully loaded, maybe.
[David Spark] No, but you’re still spending the same amount of money, but I’m talking about the cost of the turnover each time. Are you spending a million dollars on the cost of the turnover.
[Jack Leidecker] Yeah, but it takes a while for someone to become effective, right? So, if I’m turning those people over every nine months, you’re also not going to get people that are actually evolving. And like my…
[Crosstalk 00:17:10]
[David Spark] Oh, wait. Are you agreeing with Jerich on this?
[Jack Leidecker] I would be, sadly.
[David Spark] Yeah.
[Audience member] [Laughter]
[David Spark] All right.
[Jack Leidecker] I really don’t want to, but I think I’m going to…
[Crosstalk 00:17:18]
[David Spark] So, you think you’d be burning a million dollars in effectiveness and turnover if you chose the other option?
[Jack Leidecker] Yeah. Well, not even just effectiveness and turnover. Risk, right?
[David Spark] Okay.
[Jack Leidecker] Because you’re not going to be able to detect what’s going on if your people are turning over that quickly. They’re not going to be good.
[David Spark] No, but the thing is that in both cases, your security is, like, the same. Your effectiveness is the same. One case, you’re spending a million dollars to keep people not burnt out. The other case is you’re not spending the money and they’re burning out every 9 to 12 months.
[Jerich Beason] So, they say one bad apple spoils a bunch. If my SOC is unhappy, I guarantee you that’s permeating to other parts of my team. So, this is a bigger impact than just the SOC.
[David Spark] Okay, so you see this trickling over. All right. Let’s go to the audience now. All right. So, the two scenarios are you spend a million bucks, and you keep your SOC from burning out, or you don’t spend the money, and they burn out in 9 to 12 months.
In both cases, your quality of security and incident response is the same. So, again, you want to applaud for the worst scenario, the one that you think is worse. So, the first one is the one that they both went with. By applause, how many people think that’s the worst scenario?
[Silence]
[David Spark] The people raising hands, they can’t hear you. Hold it. Wait a second. I think the whole audience is going to go against you on this one.
[Jerich Beason] I think they agree with us. They misunderstood.
[David Spark] Yeah. Okay. Well, no. You think they agree. Again, the worst scenario, let me see. You think the worst scenario… Oh, no, I’m sorry. I take that back.
[Jack Leidecker] Yeah, you got it backwards.
[David Spark] I screwed it up.
[Jack Leidecker] That’s why no one clapped.
[David Spark] That’s why no one applauded. Nobody agrees with you. One person raised their hand. Okay, you guys are following. I’m not saying the right thing. The second one they said is the worst scenario, turn over 9 to 12 months. By applause, how many think that’s the worst?
[Applause]
[David Spark] All right. And let me go to the first scenario. How many people by applause? Not raising your hand because I can’t record raising your hand on this microphone. By applause, did anyone think that spending the million dollars to get essentially no results is worse?
[Clap]
[Jack Leidecker] Way to be brave.
[David Spark] Way to go, Rich. Way to go.
[Jack Leidecker] We got one.
[David Spark] We got one person with a sarcastic clap, I appreciate it.
It’s time to play a brand new game!
19:19.138
[David Spark] All right. You guys are all familiar with The Family Feud, right? You know how that game goes.
[Jerich Beason] I am. I yell at my TV all the time.
[David Spark] All right. It’s a fun game. So, we kind of did this with our audience as well. We put up a survey on our site. We have a Participate page, we’re going to put up another one. Hopefully soon, we’ll have some more questions up. And we ask five questions, and we’ll see how many of these we can get through here.
We ask five questions of our audience. So, I’m going to ask you these questions and I want to see if you can get the most popular answer. Shout it out if you know it. And if you can’t get it, we’ll go to the audience and then we’ll also see if the audience can get some of the other answers as well.
All right. The first question is – remember, and we got 70 responses here – name something you should never share online.
[Jack Leidecker] Your password.
[David Spark] Password. That is number one response, 27 responses on that. All right. Very good for you, Jack.
[Applause]
[David Spark] Jack gets a point for that. All right, let’s see. Can you get any of the other responses? Passwords being number one.
[Jerich Beason] Passport.
[David Spark] Passport? I will put that under personal identifiable information, that’s number three with 11 responses. We have a number two…
[Crosstalk 00:20:38]
[Jack Leidecker] So, would Social probably would fall under that, too, right?
[David Spark] What’d you say?
[Jack Leidecker] Social Security.
[David Spark] Social Security, number two, all right. You got first and second, all right.
[Applause]
[David Spark] All right. We have three more popular responses. You think you can get the last three, either one of you?
[Jerich Beason] Your vulnerabilities.
[David Spark] Vulnerability? No. Individual. Personal things. No.
[Jack Leidecker] User ID.
[David Spark] That would fall under PII.
[Jack Leidecker] Your address.
[David Spark] Contact or location information. That was number five. We have four and six, what do you think?
[Jerich Beason] Kids’ names.
[David Spark] Kids’ names? No. Well, that would fall under contact and location information. If you get this one wrong, I’m going to the audience and see if we can get the last two.
[Jack Leidecker] Your date of birth.
[David Spark] No, no, no. All right. Two more. By the way, there’s three responses for the content. We have two more, number four and six. And just shout it out, what do you think?
[Audience member] Credit card.
[David Spark] A credit card. I will say financial information, yes. And then the very last one.
[Audience member] Health information.
[Audience member] Any health.
[David Spark] Health? That would be under PII. No. Anything else? Oh, this one’s a good one. No one’s getting it.
[Audience member] Company credit card?
[David Spark] No. Nudes! Don’t share nudes.
[Laughter]
[David Spark] All right.
[Jack Leidecker] I don’t know that I would have guessed that one.
[David Spark] All right. Good job. Good job. Good job, everybody. Nobody got the…
[Laughter]
[David Spark] All right. Here we go. Question number two. Name something in cybersecurity that gets harder the longer you wait. Jump in when you know it.
[Jerich Beason] Asset management.
[David Spark] Asset management. Oh, it’s, ooh, number six. Asset inventory, number six.
[Jack Leidecker] Patch management?
[David Spark] Patching updates. Number one with 11! All right. Two for two here. Excellent.
[Applause]
[Jack Leidecker] Good job. All right. Keep going, I got a total of eight here. What do you think?
[Jerich Beason] Identity and access management.
[David Spark] Identity response or… Well, hold it. Identity? No.
[Laughter]
[David Spark] By the way, patching and updates and vulnerability management is the same thing. I have that as one.
[Jack Leidecker] Yeah, patching and vulnerability would be the same.
[David Spark] But that’s the same thing. That’s the same thing. All right.
[Jack Leidecker] Secret management? Although that could be pass…
[Crosstalk 00:22:35]
[Jerich Beason] Respond to incidents.
[David Spark] Incident response and breach management. Yes, that’s number two.
[Jack Leidecker] Phishing.
[David Spark] Phishing. No, that’s not on our list at all. I’m going to give you one, but he’s striked out.
[Jerich Beason] Regulatory compliance.
[David Spark] Compliance and regulations. Yes, it’s number three.
[Applause]
[David Spark] All right. Again, name something in cybersecurity that gets harder the longer you wait, and I have three more. I have number four, five, and seven here.
[Jerich Beason] Hiring a CISO.
[Laughter]
[David Spark] That’s career and professional development. I’ll give that to you. That’s number five.
[Laughter]
[Jack Leidecker] That’s a stretch.
[David Spark] Number four is a little vague. It says fixing the organization. All right. You know what? Then number seven, I’m going to the audience on this one. This is a tough one, but it is something that definitely I’m going to stress, the longer you wait, the more painful it becomes.
What is it?
[Audience member] Training.
[David Spark] No.
[Audience member] [Inaudible 00:23:30].
[David Spark] No.
[Crosstalk 00:23:33]
[David Spark] No. I’m going to give you a big hint. You work for a very big company like maybe your company or a healthcare company or an energy company.
[Audience member] [Inaudible 00:23:45]
[David Spark] What did you say?
[Crosstalk 00:23:48]
[David Spark] No, not vendor. Hold it, what’d you say over here?
[Audience member] Shadow IT.
[David Spark] No, not shadow IT.
[Audience member] [Inaudible 00:23:53].
[David Spark] No. Legacy tools, legacy tools. All right. Let’s keep going on this.
[Jerich Beason] The audience agrees.
[David Spark] All right.
[Jack Leidecker] [Laughter] Boo.
[David Spark] Here we go. Question three, we’ll just do it. Here we go. We’re going to finish on number three here. This is a good one. Name a cyber security mistake everyone makes but will not admit to. This one’s easy. You should be able to get this one.
[Jack Leidecker] Reusing a password.
[David Spark] Yes. Number one. Good job.
[Applause]
[Organ music]
[David Spark] Good job. All right. Can you get any of that? We’ve got a total of seven here. Thirty-three responses, by the way, on that one.
[Jerich Beason] Using unapproved software?
[David Spark] No, that is actually not on our list.
[Infrasound]
[Jerich Beason] Oh, now we bring those out? All right.
[Jack Leidecker] Bypassing your security controls.
[David Spark] Oh, no, not really. It’s not on our list here. I was trying to see if we could fall into for that.
[Jack Leidecker] Clicking on a phishing email.
[David Spark] Click a phishing email. Yes.
[Background voices] Sir, yes, sir!
[David Spark] Good job, but that’s the second one. Seven responses to that. Oh, you know what? Hold it. Risky behaviors was number three, and that’s using public Wi-Fi, accepting cookies on websites, spam calls. It could be a lot of different things.
All right. So, we get another one with that. Okay, so I have number four through seven. What do you think?
[Jerich Beason] I gave you my good stuff, man.
[David Spark] Gave you good stuff. Let’s go to the audience.
[Jack Leidecker] Downloading… Oh, okay.
[David Spark] What do you got?
[Jack Leidecker] I was going to put a downloading pirated software using Tor.
[David Spark] No. Okay, we got four more here. Let’s see if the audience can get them. Shout them out if you know them.
[Audience member] [Inaudible 00:25:29]
[David Spark] Sharing credentials. Sharing access issues, yes. Number six, four responses for that.
[Audience member] Using [Inaudible 00:25:36].
[David Spark] Using their computer what?
[Audience member] Leaving their computer…
[David Spark] Leaving their computer open. That would be risky behavior. That would fall under that category. Would someone yell out here?
[Audience member] Not always following [Inaudible 00:25:47].
[David Spark] Not following change… Well, neglecting updates and patching. Yeah, that would fall under that. And then there’s one good one actually only came in seventh that I thought, but that would also fall under security assumptions and overlooking basics.
There’s one more that’s a good one. Anyone going want to guess it? Default settings. You all do it, you know it.
How is AI going to solve this problem?
26:11.854
[David Spark] All right. What specific pain does your AI solve that simpler methods cannot? That’s the number one question. Caleb Sima of WhiteRabbit says you should ask any vendor patching AI security solutions. If you ask, be ready for a dearth of answers.
Other AI hype red flags include vendors claiming 98 to 100 percent detection rates, promising to solve most security problems with a single platform, or being evasive about implementing timelines and concrete metrics. It’s good to have healthy skepticism of a new buzzy AI tool, but the reality is we’re all under pressure to embrace AI solutions at the same time we’re securing them.
So, I’m going to start with you, Jack. How do you separate a legitimate AI innovation? And by the way, this is a good tip as we look at the vendor hall here. Legitimate AI innovation for marketing hype when you’re evaluating vendors.
[Jack Leidecker] Yeah, so this one’s kind of near and dear to me, especially since I’m also at an AI company in this perspective, but at the same time, I think because there’s so much hype, there’s two things that I usually focus on. One is, can you quantify impact?
Because a lot of times we’ll throw fancy things, but if I can’t quantify it, it’s meaningless. And then also, what do they actually even mean by AI? Is it just a wrapper? Are they actually using their own models? Do they fine tune it? Really understand what are they doing that’s different, because if all they’re doing is taking your data, throwing it in a giant LLM, and spitting it back out, you can do that yourself, probably a lot cheaper and more effective and there’s just so much stuff that you can’t quantify and there’s too much hype right now with this.
[David Spark] All right. How do you verify? And by the way, you must deal with this all the time because everyone’s got AI in and everyone’s making claims.
[Jerich Beason] Yeah. And I’m a consumer as well. My team is in the room. They know I kind of set out a path with AI heavily involved. When I ask vendors, I ask a few different questions. First, I ask, what is it that your tool is going to bring me that hiring a few people won’t?
Number one. The second one is I ask them around what makes their models fail. I find that vendors with the best failure analysis actually have the best products. And then I also look to see if they ask me a question. If they don’t ask me about data governance or data quality and the AI tool is based off of my data, then I realize they’re just giving me a hyper-powered automation tool and not true AI because AI is based off of clean data.
[David Spark] I love that, I love that they should ask you a question. I love the fail one. Because by the way, this is something that we ask on our show Security You Should Know, “Tell me what your product does and what it does not do.” But the fail question is really a good one specifically around AI.
What are some of the answers you’ve heard on that?
[Jerich Beason] Most vendors don’t have that answer because the salespeople haven’t been given that level of understanding. Most of them understand the overall objective, but behind the scenes, they usually say, “Well, I have to get my CTO on the call,” or something along those lines.
[David Spark] Have you heard any answers to that question?
[Jack Leidecker] Yeah, no, I mean, I think there’s some good answers on it like, “This is what we do. We have an AI governance team. We’ve evaluated.” Are they 42001? But it did bring up kind of a different thought of another thing to ask, too, that kind of separates – what is their data team inside?
Do they have data scientists? Are they building their own stuff? Because I’ve had some vendors where they actually didn’t have any data scientists or anyone really that understood AI on their technical team, which I’m like, “How are you selling me a solution you don’t even understand?”
[David Spark] Chris Hoff had a post about this on LinkedIn, and he was arguing you should ask them some really basic questions about understanding AI. If they don’t have basic understandings of AI, it’s like run away. Like you don’t want to be dealing with something like that.
[Jerich Beason] That’s a great point. Most of the vendors are not building their own models. They’re heavily dependent upon one of the three or four popular ones. So, I also like to ask them, “How has your product evolved as this model evolved?” Because you’re basically at you’re depending on those models to improve for your product to improve.
So, if you can’t show me the difference between those two, then you’re probably not doing an AI as well.
[Jack Leidecker] And I think even with that, you can push like is it fine-tuned or not? How is my data being utilized? But like to that point, you don’t even need to go down that aspect because if you’re going like, “Hey, what’s your governance? Do you even have data scientists?” If the answer’s no, you’re kind of already done.
It comes down to the basics.
30:24.501
[David Spark] All right. “The hardest part of vulnerability management isn’t discovery. It’s everything after the scan.” And that’s Rinki Sethi, who’s the CISO at Upwind Security, cutting right to the heart of a problem that’s been plaguing security teams for decades.
We still struggle with the basics. Yaron Levi, CISO over at Dolby, he would say the fundamentals because if they were basic, we would have figured them out. Security teams are held back not by a lack of intent or tooling, but by fragmented asset inventories, lack of business context, and mountains of unprioritized CVEs.
For Sethi, security teams need real-time asset intelligence, AI-driven contextualization, and empowerment to remediate continuously without slowing down the business. AI tools are promising real-time asset intelligence and contextualization. Isn’t that what we all want?
I’m going to ask you, Jack, shouldn’t we be solving this problem already? I mean, this, I hear this again and again. What is stopping us from figuring out the basics, the fundamentals, whatever the heck you want to call it?
[Jack Leidecker] I think sometimes we’re looking at the problem wrong, quite frankly.
[David Spark] Mm-hmm.
[Jack Leidecker] Because security isn’t the one that typically owns it, unless you’re in an org where maybe you’re blending your IT and development with your security team, the problem is giving us better data around CVEs and other things I don’t think is a problem.
It’s if we’re going to deploy a package, we’re deploying software, do we have a way of how that’s going to be maintained? If you don’t have that in place, you’re going to be chasing contextualization and everything all day long, but the biggest delay isn’t the fact of updating the package.
It’s really easy to update a package. It’s the testing and the business that needs to maintain that and make sure it still operates because I’ve had some where it’s like, hey, we just updated all our containers and then it doesn’t work. So, it doesn’t help anyone, right?
So, I think the bigger issue is fundamentally we want to skip the hard, basic things, which is what is our asset management lifecycle? What’s our patch management? And if we want to bring in a container, a package, etc., who is going to be responsible for owning and maintaining that?
Because it’s real hard to do that after a couple years, and if you add tech debt on top of it, that’s where everyone’s having the pain. It’s not validating that it’s an issue. It’s who’s going to be responsible for testing and actually getting that functional.
[David Spark] All right. So, the problem is it’s just making sure the thing you bought actually works and doing the thing it’s supposed to be doing.
[Jack Leidecker] No, no, I would actually… It’s who is going to maintain it.
[David Spark] Well, yeah, but that’s part of the job of doing that.
[Jack Leidecker] It’s part of it, but like people skip that, right? Like, hey, Finance wants this new tool. Who’s going to be maintaining it, right? Who’s updating it when it needs to get updated? Do they know how to test it? Do we need to do things?
Who’s managing the vulnerabilities from those vendors to make sure like, hey, I need to update my packages.
[David Spark] Also, by the way, I’ll tell you something I’ve heard from CISOs before is they only now look at platform plays because of this fear of if I bring on another tool, I need to train somebody on this other tool, but if I get on a platform, well, hopefully it’s going to work alongside everything else I have, hopefully.
All right. I throw this to you, Jerich. Why is the basics, the fundamentals still a giant pain in the butt? Like, we’ve been doing this for years. This comes up all the time.
[Jerich Beason] Yeah, I agree with Jack. I have a little bit of a soapbox, so I apologize if I take a little longer here.
[Laughter]
[Jerich Beason] Security doesn’t patch. Like most of the time, security doesn’t patch. DevSecOps, something like that. But I’ve consulted for 50 companies in the Fortune 500 and multiple three-letter agencies. IT does the patching and they’re at the behest of the business that says, yes, the system can go down.
Yes, it’s okay if the system goes slower for a little while because you didn’t test the patch. Or yes, it’s okay that we’re going to have three weeks of exposure while we test this patch before we move it to production. All security does is provide context.
Security provides asset intelligence. Security provides threat intelligence, and so this is not a security problem. This is a business problem. And so, when you ask me what’s stopping us, what’s stopping us is that we still look at security. CISOs are always raked over the coals when there’s a breach, and they say they had X, Y, and Z vulnerability.
The CISO never can patch that vulnerability.
And so, I look at it like a doctor. If a doctor diagnoses you and says, “You have this issue, here’s this medication, work out, eat healthy, do all these things,” if the person decides not to do those things and they get sick or God forbid they die, do you blame the doctor?
No. But in security, you blame the CISO every single time when people don’t follow the prescription that was handed to them. It’s a business problem.
[David Spark] As a CISO, do you feel this pressure?
[Jerich Beason] Could you hear the passion in my voice, sir?
[David Spark] Yes, I did.
[Laughter]
[Jerich Beason] Absolutely I feel this pressure.
[David Spark] What’s your advice to other CISOs to alleviate this then?
[Jerich Beason] Well, number one, communicate to the leaders and the SLT and the board and so forth, “Hey, we’re going to have a bad day one day, no matter what. Just expect it.” My goal is to help you understand where that bad day can occur, and my vulnerability reporting, my timelines for how quickly we address those vulnerabilities, those are all indications as to how well we’re actually performing.
And I always say “we” because it is not me. We always say security’s a team sport. This is one of the areas where they say, “No, it’s a sport of one when it comes to vulnerability management.”
[David Spark] That’s a lot of weight on your shoulders.
[Jerich Beason] But it’s really a team sport.
[David Spark] All right.
[Jerich Beason] Hundred percent.
[David Spark] Were you cringing or did you feel some of Jerich’s pain as he was on his soapbox there, Jack?
[Jack Leidecker] Yeah, no. I mean, that’s even why I mentioned before. I don’t think contextualization is our issue, right? It’s an alignment issue. It’s a business priority issue. Do we agree that we’re going to be maintaining stuff and what’s going to be our schedule?
Because otherwise, to Jerich’s point, where the doctors that someone’s still smoking and eating junk food every day and okay, they gained weight. Surprise, surprise. Or worse, you have a heart attack, right? Like we need to be able to get into the point of how we’re aligning with the business, and it needs to be from the top executive down.
We agree this is a priority because it’s really easy because it’s like, “Oh, that’s your metric, not mine, in some cases,” which is part of the issue, right? Do you hold the business accountable if that doesn’t happen? Or are you being held accountable for something you don’t control?
Because that dynamic is where you can tell that you have a big problem. You need to make sure that they’re being held accountable for the right metrics, otherwise you’re going to fail.
It’s time for the audience question speed round.
36:33.773
[David Spark] All right. We have a good amount of time to get through a bunch of these questions. These are questions I got from last night. We had a fun meetup of fans of the CISO Series. Now, this comes from Aaron Hipple, and I just want you to just answer these as quick as you can.
And I’ve run into people who are looking for jobs here in cybersecurity, and Aaron is looking himself, so positive. And when I say hacking, I’m talking about like sort of working around the system, not literally breaking in.
[Jack Leidecker] Hacking should not always be a negative term.
[David Spark] No, it’s a positive. We’re using it in a very positive way. So, either one of you jump in first on this. What are the best examples of hacking the hiring process that you have seen that you were impressed by?
[Jack Leidecker] I think if you understand what someone’s looking for, and you can show quantifiably how you can do that. Or even presenting at a conference, I’ve seen people actually get things from that, contributing to open source we talked about.
I think there’s lots of different ways to be able to do that. And then also a lot of it’s just networking, getting to know people, right? Like that’s going to help you at least. It may not get you the job, but at least it’ll get you through the front door in a lot of cases.
[David Spark] What about you? What’s the best example you’ve seen?
[Jerich Beason] This hasn’t happened to me, but I had a CISO tell me a story where he was hiring a threat intel analyst, and the analyst showed up with all this OSINT on the person interviewing him, said, “Here are all the things that are out there in the wild.
Here are all the things that we could have done that I have seen. And by the way, this used to be your password. I can do that for the company. Hire me.”
[David Spark] Yes, that’s good. And the OSINT is out there. It’s pretty visible. All right. That’s a good one. All right. This comes from Bill Brenner of CyberRisk Alliance. By the way, one of the things we were talking about, it’s like that CISOs and security professionals need a space where they can just complain about issues, maybe the pressure the business is giving them, like you’ve done right here on this show.
But the problem is there’s some issues that you would like to speak about anonymously, like you don’t want your name attached to. So, what do you think people would talk about if there was an anonymous space – Reddit can sometimes be that – to talk about cyber and cyber stress.
What do you think that we’re not talking about, that people would talk about more if they spoke about it anonymously. What do you think?
[Jerich Beason] Excuse my French, but [Beep] in the industry. We can’t always talk about them, but if you knew that you could share who was and who is, I think that that would be something that we’d share a little bit more frequently.
[David Spark] Okay.
[Jack Leidecker] I think also some of the internal roadblocks you get sometimes, right? On a closed forum, maybe you feel open on it, but honestly, I think if it’s more anonymous, people might even talk more about, “Hey, I tried to do this and I got shut down,” or “We wanted to patch, and we were just told no.” And then something bad goes happen, it’s like, oh, I’m not necessarily surprised, sadly.
[David Spark] Would you see benefit from either reading or participating in an anonymous cybersecurity forum? And again, Reddit could be that. What do you think?
[Jack Leidecker] I think the problem we have sometimes is when it gets anonymous, it’s hard to make sure that they’re adding value, and it doesn’t just become trolling.
[David Spark] Good point. What do you think, Jerich?
[Jerich Beason] I mean, if you just want to vent, sure, but if you want to have a dialogue, then the anonymous aspect of it provides a disservice at that point.
[David Spark] Okay, I like this one from Josh Dray over at San Jacinto College says, “For your own security environment, how are you making space for AI innovation?” And can you make space? Do you do that?
[Jerich Beason] Yeah, I mean, I was just, I challenged my team, I mean, they’re in the room, right, to find ways to do what you’re doing more efficiently, harnessing and leveraging AI.
[David Spark] Mm-hmm.
[Jerich Beason] If you succeed, great. And if you fail, nothing has changed.
[David Spark] Nothing’s.
[Jack Leidecker] I think you just need to be conscious about it, right? What are you hoping to accomplish? What are you doing? Do you have a way to kind of measure yourselves with it? And also automation is key here, right? Some of the AI really helps extend some of the automation, even Soar back in the day that you had.
So, being able to actually see what you’re doing and replacing manual work.
[Jerich Beason] Yeah. Set the goal and objective. Don’t say, “Go use AI.” You got to set an objective, whatever that objective may be. If it’s be more efficient, if it’s move at scale, if it’s move faster, whatever it is, set the objective and then see what happens.
[David Spark] All right. This, try not to take this as too global a question. We’re going to get through two more really quickly. What are ways you’re managing the integrity of AI data? Because this is a big issue. What do you look for? Like what’s a first step when you’re dealing with that?
[Jack Leidecker] So, I think for that, do you even understand where your data is, and do you have a governance model? Right? I would say for us, when we went through that, being able to understand how many models we have in-house, what’s fine-tuned, what do we use that’s outside?
What’s open source? Where do we have AI models inside of our vendors? Like just getting that inventory to me is step one. And then too, defining what controls you want on it, right? Hey, if this is something we’re doing, fine tuning ourselves. Do we do bias testing?
What guardrails do we implement? So, like having a strategy on that to me is key.
[David Spark] All right. Last question. And comes from Brian Zabeti of Pliancy. What’s your best negotiating technique with a stakeholder? You got to get them to do something, how do you get them? What’s your best technique?
[Jerich Beason] My go-to are always analogies and metaphors. Hopefully, one that puts up a mirror and helps them understand the decision that they’re making. Usually when people understand a risk, they’re willing to take an action, but no one’s willing to open up their pay book for something that they don’t truly understand.
So, it’s my job to help them understand it, and then from there, the negotiation’s a lot easier.
[Jack Leidecker] Yeah. So, mine is pretty simple with it, especially since we’re a B2B company. It’s impact on customers with it. Hey, if we don’t do this, this is the impact, or we’re not going to be able to get this business. Relating it back to sales to me has always been the easiest way to make it a bit easier to get them on board.
Closing
42:16.379
[David Spark] Well, that brings us to the very end of our episode. Let’s hear it for our guests today.
[Applause]
[David Spark] Jack Leidecker, who’s the CISO of Gong. Also, Jerich Beason, who’s the CISO over at WM. And our audience, and for HOU.SEC.CON, let’s hear it for yourselves.
[Applause]
[David Spark] And lastly, Vorlon Security, enterprise SaaS Security that’s light years beyond legacy SSPM tools. Remember, go to Vorlon.io, let them know you heard about them from the CISO Series. My very last question for you, gentlemen, are you hiring?
[Jerich Beason] Yes.
[David Spark] Are you hiring, Jack?
[Jack Leidecker] Yes, I am. You can always look at the job board.
[David Spark] They have job boards, and I asked them earlier, you can contact them. We will have their LinkedIn profiles linked on the podcast episode for this very show. Thank you very much. Thank you to HOU.SEC.CON for making this possible. And thank you, everybody.
We greatly appreciate it.
[Applause]
[David Spark] Thank you for listening and contributing to the CISO Series.
[Applause]
[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. We have lots more shows on our website, CISOseries.com. Please join us on Fridays for our live shows – Super Cyber Friday, our virtual meetup, and Cyber Security Headlines Week in Review.
This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com.
Thank you for listening to the CISO Series Podcast.