Recent research shows that CISOs feel their jobs are harder than ever with higher levels of stress. This comes from unrelenting and escalating threats to organizations as well as higher levels of regulatory scrutiny. Yet the same research shows CISO job satisfaction increasing. How do we make sense of this contradiction?
This week’s episode is hosted by me, David Spark (@dspark), producer of CISO Series and Andy Ellis (@csoandy), operating partner, YL Ventures. Joining us is our guest, Shawn Bowen, svp and CISO, World Kinect Corporation.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, Silk Security
Full Transcript
​​​​​​[Shawn Bowen] I love that we are never done, but it’s also something I hate about it.
[David Spark] Welcome to the CISO Series podcast. My name is David Spark, producer of said CISO Series, and joining me for this very episode is my cohost, one of your favorites, whether you like it or not, Andy Ellis. He’s the operating partner over at YL Ventures. Say hello to the audience, Andy.
[Andy Ellis] Hello to the audience, Andy.
[David Spark] There you go. That’s it. That is one of the most classic old jokes. Like “walk this way.”
[Andy Ellis] …and sometimes, that’s just all I’ve got going.
[David Spark] There you go. We’re available at CISOseries.com, and our sponsor for today’s episode, a new sponsor with the CISO Series, is Silk. Revolutionize your risk resolution process with Silk—we’ll be explaining exactly how they’re doing that later in the show.
So Andy, one of the side things I’ve done, which is technically not part of the CISO Series but is in the world of security, is starting the San Diego Cyber Group with Rick McElroy. It’s a meetup group.
[Andy Ellis] Mm hmm.
[David Spark] Are you involved in any meetup groups in the Boston area yourself, Andy?
[Andy Ellis] Bostonians don’t really do meetups. I mean, they do exist, but there’s not nearly as many of them. We spend like four months out of the year where you don’t leave your house, and certainly not to go drive on roads.
[David Spark] The great thing about San Diego is you can leave your house at any time, except when it’s raining.
[Andy Ellis] You don’t have weather; you have climate.
[David Spark] Yes, it’s kind of nice. As I like to brag, I never turn on my heat or air conditioning. Ever. It’s kind of nice.
Anyways, it’s the San Diego Cyber Group. For those of you listeners who live in the San Diego area, just go to Meetup.com and search “San Diego Cyber Group.”
It has been a pleasure, a thrill. We just started it a little over half a year ago, and it’s been great. We’ve got about 60 people showing up to our meetups now. Sadly, we keep having to move locations because of issues with the breweries we’ve been using.
You would think they’d be tripping over themselves to have a group of people show up once a month and give them a lot of money.
But for some reason, they can’t figure out how to manage that issue. And then we have to keep moving to the next one. We’re hoping we find the right location soon.
But doesn’t a promise of continuous money seem to be an attractive factor for a business? Don’t you think?
[Andy Ellis] You have to recognize that sometimes the costs outweigh the income you bring in. We’ve talked about firing clients before, right? Sometimes you just have customers that you’re like, “Yeah, I actually don’t want your business because it’s not worth it.”
[David Spark] Hold it. Now you’re assuming that I’m difficult to work with, Andy, with that insinuation.
[Andy Ellis] Oh, there is no assumption required.
[David Spark] I am not difficult to work with. In fact, that’s quite the opposite. I try to make it as easy as possible, but it’s always been one issue after another, and that’s just my complaint about it. But we have a new location now that we’re super happy with. And in fact, when this episode drops, the next day, we have our event, which is on the 27th.
We do it on Wednesdays.
So if you’re hearing this tomorrow, we’re having an event. Please join us.
[Andy Ellis] Excellent.
[David Spark] All right. Let’s get our guest on, who is, I think, our most frequent guest across all our programs. He’s been on all our shows, except I think Capture the CISO. I believe he hasn’t been on that. No, you have been on that.
[Shawn Bowen] I was on the very first recording.
[David Spark] Ah, then I take that back. He’s been on every show multiple times, and he’s back here again. Thrilled to have—
[Andy Ellis] And finally, he has graduated to get to be on a show that I’m hosting.
[David Spark] You consider that a graduation, not a demotion?
[Andy Ellis] We test run with everybody else before he gets to come hang out with everybody’s favorite cohost.
[David Spark] He gets to level up, Andy.
[Shawn Bowen] It’s the boss level.
[David Spark] He’s the CISO for the World Connect Corporation, Shawn Bowen. Shawn, thank you so much for joining us.
[Shawn Bowen] Thank you for having me again. As always, I enjoy it, and I will be in San Diego in September. Hopefully, it lines up with the meetup.
The Great CISO Challenge
4:12.692
[David Spark] Today’s topic is about the changing regulatory landscape. We talk a lot about uncertain times ahead for CISOs. That might lead you to assume that there is increasing job dissatisfaction in the role. But that doesn’t seem to be the case, with a recent survey from ESG and ISSA showing CISOs outpace other cybersecurity roles in average job satisfaction.
However, Jon Oltsik at CSO Online noticed that those same figures show that roughly a third of CISOs think the job is much harder than two years ago, well ahead of the overall industry, with six in 10 saying they feel stressed out most of the time.
So how do we make sense of this, Andy? CISOs seem to feel more stressed, more often, in a job that they think is harder, yet they remain satisfied. What’s the correlation here?
[Andy Ellis] So this is one that I’m always very cautious about—things that look like longitudinal surveys but might or might not be. Satisfaction is up, but at the same time, people are saying it’s harder than it used to be. This is a very important distinction. This isn’t a case where we interviewed the same people two years ago and then again today and asked how hard it is.
We interviewed people today and asked them if they think the job has gotten harder than it was.
So maybe we’re not actually talking to the same CISOs. I know a lot of CISOs in the last two years who have permanently retired from being a CISO.
[David Spark] By the way, we’re speaking to one right now.
[Andy Ellis] Yeah, I retired three years ago from being a CISO. So, I could tell a story—I don’t think it’s the true story—that there are more people now who are CISOs who were not CISOs two years ago, and they think the job is harder, but they’re way more satisfied because they’re getting a better paycheck. That’s potentially one argument.
[David Spark] Wait, are you saying they interview people who are no longer CISOs?
[Andy Ellis] Not necessarily people who are no longer CISOs, but imagine someone took this survey who became a CISO in the last two years for the first time. They got a huge pay raise, so they’re very satisfied, but at the same time, they think, “This job is much harder than my predecessor had it. My predecessor had an easy job.
I don’t know what they were talking about.”
[David Spark] So they don’t assume that the respondents were CISOs two years ago?
[Andy Ellis] Not from what I was able to skim looking at it. So always be careful about looking at surveys like this and drawing too many conclusions. Is the job more stressful? Absolutely. Speaking as someone who’s on the outside looking in, I think CISOs today have it harder than I had it four years ago.
[David Spark] I think just the complexity makes it so much harder. It just keeps getting more complex. Don’t you think?
[Andy Ellis] I actually think it’s getting simpler.
[David Spark] Oh, Shawn, you call him out on this. Shawn, who is currently a CISO and just admitted at the beginning of the show that he loves and hates the fact that the job is never done, is it getting easier, harder, or simpler as Andy believes it to be? And he’s not in the mix of it all, so he really doesn’t have much to say here, does he?
[Andy Ellis] I can’t wait to see all the comments that I’m going to get after this drops, by the way.
[Shawn Bowen] Yes, okay, so I’ll add fuel to the fire. In some aspects, I’ll pick on Andy since we have a long relationship and we came from similar backgrounds in the military. I also think there’s a divide that’s not recognized between CISO affiliations. There are the “CISOs of Silicon Valley,” like the CISOs of IT companies and SaaS providers.
They have a very different look on the CISO role and the landscape than the non-IT company CISOs like me.
I work for a fuel management, logistics, energy company—a very different organization than those types of companies.
So, in some cases, I think it’s easier in the sense that I came from the government. What I missed about the government was I had rules that I could lean on. When I needed to get something done, I could always say, “Well, the White House said, Congress said, or the Pentagon said,” and that trumped whatever my supervisor didn’t want to do.
When I came to the corporate world, I learned I have to be really good as a salesman to convince people that security is important because they didn’t necessarily believe it was as important as it is.
And so, with some of the regulation changes, it’s becoming easier, but it’s also significantly harder in the sense that a lot of the corporate CISOs in non-IT companies have a gigantic security debt to clean up.
There was a conversation recently with some CISOs talking about a very specific product space, saying that it was just a waste of money. “Why are CISOs buying it?” And I said, “Yeah, that’s nice to work in a company that’s all modern, started in the cloud, and is currently in the cloud, compared to a company that is cleaning up 20-year-old technology on the regular.” And that’s not uncommon.
Thankfully, we’ve gotten rid of a lot of that, but when I first got here, that was one of the big problems that we had.
And I’m probably the anomaly by cleaning that all up. So yeah, it’s definitely an uphill battle, but I think that the recognition and the structure that’s coming is good, and it’s something I’ve been advocating for.
I would love to have a GAAP for security, some sort of standardized protocols that we’re all measuring the same way, whatever it might be. That doesn’t mean that we have to get the same success, just like in GAAP, it doesn’t say that everyone has to have the same profit margin.
It just says you have to measure profit the same way. We need to get to some common measuring standard. I think that will also help us tell the story in a consistent way.
Why are CISOs leaving the profession?
9:53.795
[David Spark] We’re going to continue talking about stress. We’ve talked before about how common stress and burnout are in cybersecurity. A recent post on the cybersecurity subreddit blew up, talking about the factors that lead to burnout, ranging from issues like disconnect with IT and sales, failures in change management, and just feeling alone in their security mission.
The post author said they were lucky enough to save up for a long sabbatical to rethink their career.
It raises the question: What are the factors that should cause you to either rethink your job or look to shift your career? Now, both of you are still in cybersecurity. Have you questioned your career at multiple times? Be honest here. I’ll start with you, Shawn.
[Shawn Bowen] 100%. I think one of the things—and my boss gave me this feedback—he says that it’s very clear that I have a passion and I love what I do. And that makes it easier for me to do what I want to do because it’s very stressful, and I have to deal with defeat on a regular basis and find comfort in that.
But because of my passion and love for this space, I’m able to push through. So there are definitely times where I’ve gone into the job and said, “I see the end result so clearly. What am I doing wrong that I can’t get the rest of the people to see that clarity?” And so that, that’s been the focus for me, that challenge.
I think when I started in my career, I loved being in the server room. I was that guy that swore I’m never going to leave the server room. I don’t want to be anywhere else. And it was because I loved solving the problem. I loved solving the technical piece. And now I’m realizing that the problem of people is much more fun for me.
And so trying to convince people to see what I see is how I look at my daily life, and I don’t always succeed. So I come back and try something new. And that’s, that for me has made me enjoy the job, showing up despite some of my frustrations.
But I think you have to have your external factors, right? I have my family, I skydive—that’s something people know about me quite well. So every once in a while, I just need to check out and go do something fun and then come back with a new attitude towards it.
So it’s there, but I think we had a lot of people in security join for the money. Frankly, I think there were a lot of people that were in the cert boot camps, getting every cert they could, chasing that dollar figure. And now they’re realizing that they’re expected to actually perform, and they suck at it.
[David Spark] Well, that’s one reason you get paid well—because of the stress and the knowledge you need to have.
[Shawn Bowen] But they suck at critical thinking. They suck at basic things like, they might pass the test of connecting A to B, but they can’t interpolate or extrapolate when there’s a problem. And that’s the critical thinking skill that we want.
And then there’s also this added factor that you can no longer live in the basement and be successful. You have to be a human interface for your space. And that’s not something that everyone is comfortable with.
And I think there’s also the fact that we teach people security, and we don’t teach people risk. So you work your way all the way up to director, focusing on security, security, security. And then all of a sudden, you have to interface with the business and you have to balance risk. And we don’t teach our career path to think about risk; it’s all about “secure at a hundred percent or else you failed.” And we have to teach how to balance risk a lot sooner in our career.
[David Spark] All right, Andy, I want to know—tell me one moment that you questioned, “Should I still be a CISO?” Now, you are no longer one, so you did finally make that decision.
[Andy Ellis] Yeah. So for me, really easy. I’ve questioned it a lot. And I’ve been where the person who wrote this Reddit thing was. And I will tell you, go read the Reddit post if you’re listening right now. And I think we put it in the links here.
If you agree with this person, like you can empathize with them, but if you sympathize and you’re like, “Oh, they’re so right,” then you need to question whether you’re in the right career field. Because I will tell you this very brutally hard truth: If you’re in the cybersecurity career field, you are one of the least important people in your company.
[David Spark] That does not feel good.
[Andy Ellis] No, no. Like you are like the sidekick to the sidekick. Now look, we’ve, we’ve all seen places where like the sidekick to the sidekick fails and the hero loses. But we’re like Jarvis in the Iron Man suit. Like we help Iron Man get stuff done, but we don’t do anything ourselves. And what happens when Jarvis gets a lot of power?
He becomes Ultron and tries to destroy the world.
And that’s kind of the security problem, which is we try to make things perfect. And the way it would be perfect is if you didn’t have a business to protect. And that’s the actual logical outcome of perfect security that Shawn was just talking about.
So it’s not just that we have to understand risk. We have to understand the business. So your mission, your vision for yourself as a security professional, should be something like “we enable the business to make better risk choices.” Like we’re going to help them by making things that they don’t need to pay attention to easy, and the things they need to pay attention to, we want to give them clear choices about the risks that they’re taking.
And if you don’t think that’s your job, maybe you should go find a different job.
[Shawn Bowen] I do want to say we’re not trying to be discouraging. There are a lot of great, talented people in this space, and I am impressed on a regular basis with some of the thought processes and the execution that people are doing. But this particular post and the stress that people are identifying with, that is a part of the job, and it’s not often talked about.
And so we’re not saying everyone should quit and get out.
It’s just saying, have an understanding of reality—there’s more than just getting to play with computers.
Sponsor – Silk Security
15:46.172
[David Spark] Before I go on any further, I do want to tell you about our absolutely awesome sponsor, Silk. Silk, they’re awesome. Let me tell you, are you feeling anxious about risks lurking in alert backlogs? Are you frustrated with remediation requests stuck in the twilight zone?
Instead of toiling over spreadsheets and watching alert backlogs build up, Silk helps security teams regain control over their risk backlog and posture. Designed with security practitioners in mind, Silk consolidates and contextualizes all of your vulnerabilities, misconfigurations, and app sec findings across your IT footprint and assets.
So you can quickly focus on the most urgent risks to your environment and business.
Silk then automates assigning the right remediation recommendation to the right owner through their “right to do” workflow tool. Using AI and environmental analysis, Silk’s predictive ownership assignment eliminates the time suck of back-and-forth emailing to identify asset and remediation owners.
With Silk’s centralized reporting and ticketing integrations, there’s no more logging into multiple systems to create tickets for asset owners and track down remediation or exception requests. Security teams get visibility in one place across all remediation tasks by team, by remediation category, and can easily communicate with stakeholders through a single dashboard to provide guidance or keep track of exception requests.
Security teams get to both resolve more critical cyber risks in a fraction of the time with Silk and shift organizational dynamics through a comprehensive remediation life cycle for better accountability and transparency. It’s like the whole workflow, and Silk can help you with it.
So you can learn more if you go to their website, it’s silk.security. That’s it. Silk.security. Go check it out.
It’s time to play “What’s Worse?”
17:45.002
[David Spark] All right, Andy. I’m just going to say this comes from Olivia Phillips of Amtrak, who’s given us stuff before. And I will tell you, both of these situations are catastrophic. All right.
[Andy Ellis] Okay. I might be on Amtrak next week, or I guess last week after this drops. So we’ll see. I hope it’s nothing catastrophic there.
[David Spark] Let’s hope not. All right, here we go. Scenario number one: your organization falls victim to a ransomware attack where malicious actors encrypt your critical business systems data and are demanding payment in exchange for the decryption keys. Sounds like a classic ransomware attack. Here are your specific situations:
[Andy Ellis] How much are they asking?
[David Spark] I’m going to get to that. There is no network traffic to sift through due to budget cuts. You do not have the 50 million the attackers are asking for, and you do not have cyber insurance. All right. So your basic, really bad situation.
[Andy Ellis] Yep. I love this scenario. I’m happy with this one. Let’s see the next one.
[David Spark] All right. Now, scenario number two: the organization becomes a target of a sophisticated and persistent cyber espionage campaign conducted by nation-state actors or advanced cybercriminal groups. This APT attack is aimed at stealing sensitive information, gathering intelligence, and disrupting operations.
Pretty much got the big three right there.
The outage has caused an extended period of disruption of operations, causing significant challenges for detection, attribution, and mitigation. The servers are brought down, and the last backup was three years ago. Now, obviously, somebody wasn’t doing their job. And all employee PII information was deleted.
So these both seem awful. Andy, which one’s worse?
[Andy Ellis] So they’re, they’re awful, and they’re differently awful. I’m going to say the second one is marginally worse because I look at this and I’m saying I’ve got a 50 million budget based on this first one, right? What do I do with 50 million? I’m going to throw out all of the bad IT systems that just got compromised, shift to being a SaaS-based corporation overnight, give everybody brand new systems.
Be like, “Yep, here we go.” Because we can’t, we don’t trust that we’re going to get our data back, and we don’t have 50 million to throw around.
[David Spark] So you essentially, you’re just rebooting the company.
[Andy Ellis] Yeah, because honestly, if you’re getting nailed by ransomware that can take out all of your backend systems, your backend systems need to go away anyway. So do you want to pay the ransomware and then not be able to afford to upgrade? Just do the upgrade, figure out how you’re going to recover from paper-based stuff.
Cause I’m sure that Shawn’s still shipping paper-based invoices around somewhere.
It’s gotta be somebody in your company doing it.
Shawn’s like, “No, no, not me.”
[Shawn Bowen] We’ve modernized with OCR.
[Andy Ellis] But I think, I think the second one is actually worse than the first one because they are not going to go away. No matter what you do, as you try to clean up.
[David Spark] All right. Shawn, what do you think? Which one’s worse?
[Shawn Bowen] All right. Here’s, this is exactly what you want, David. You want, you want me to disagree with Andy, and I’m going—
[David Spark] I’m, I’m all for just, you don’t have to, but I am all for—
[Andy Ellis] He loves it when that happens.
[Shawn Bowen] I’m prepared to do it just for the sake of the show, but also because I actually believe it. The second one is theoretically easier for me because of our friends at the Bureau. As soon as you bring in espionage, I’m calling my buddies in the government and having them take over this whole thing.
And I have free labor to help me with a lot of this work. And if necessary, they’ll bring in the National Guard. They’ve done all of this in the past with other situations. So depending on the criticality of your business, obviously, and if it’s coming from Amtrak, I’m going to assume that for the transportation space, this would definitely receive that level of support.
And so for me, the second one, I have a little bit more help. I would definitely get help from the Bureau with ransomware, and so I would reach out to them in the same situation for that. And we’ll do our best through that space there. But with espionage, that is their bread and butter, and I would look forward to the help that they would jump in on.
[David Spark] So Andy has been laughing his butt off through this, your whole answer here. And by the way, this is not assuming that it’s happening to Amtrak. It just happened to be the submitter who
[Andy Ellis] This, that’s who submitted it. Let’s not.
[Shawn Bowen] My company, my company is also a critical infrastructure enough that I can make that phone call. So
[Andy Ellis] Just, just to point out that I think the sixth ever person convicted for economic espionage was targeting me and us. And I will tell you that was a four-year investigation in which we got almost zero support because the U.S. attorney’s office was more interested in building a case in their spare time than in helping us do anything.
[Shawn Bowen] Andy, you’ve been retired. You need to go away.
[Andy Ellis] If you’re going to rely on the FBI to show up and save your bacon. Ooh,
[Shawn Bowen] I will advocate for them strongly. And, and I know David’s had some of them on their show recently and, and they have definitely made a change for the better in the last couple of years since your retirement. So I asked you to come back and join us in the CISO force and experience some of that, because there’s definitely some things that they still got to work through.
I mean, we know this from the government life that there’s still some, some evidence acquisitions and case, et cetera. But. Their cyber task force has done a tremendous job in, in getting their hands dirty with the partners in the private sector. And so I don’t disagree four or five years ago, I would have given them the same grade that you just gave them.
But I think in the last year or so, director Ray and his staff all the way down through the cybersecurity side has done a tremendous job, Brian Borden and David, and so there’s a couple of folks out there that are doing a phenomenal job there,
[Andy Ellis] I’m glad to hear that then, but I hope nobody actually needs to call them.
[Shawn Bowen] But you should, if you work for a private company, you should know your local FBI contact.
What’s your security advice?
23:31.746
[David Spark] Given that so many commercial apps are built on open source code that can be easily tainted, security leaders have become obsessed with securing the software supply chain. Now everyone realizes it’s challenging and multiple vendors have popped up to offer solutions. Clint Gibler and Francis Odum of TL;DR Sec recently analyzed the vendor landscape.
Finding vendors focus on three distinct areas in this space of trying to secure the software and supply chain. So source code, build and deployment. They found quote, the greatest value lies in vendors who can seamlessly bridge the gap between security and engineering teams. Given that there is a great need and there are solutions.
How should CISOs evaluate where a software supply chain vendor, those offering solutions to secure the actual supply chain, makes the most sense in their organization? What should you be looking at to analyze?
[Andy Ellis] So I think that I would take the original premise and say, you’re not trying to necessarily protect against malicious code injection all the way upstream in a software in the open source world. And in fact, if you’re a vendor in this space and that’s your argument is you should buy me so that you can detect it, you’re doing it wrong.
You should just be doing that pro bono of like, we’re going to monitor the top 20, top 50 open source projects and protect them. And then here’s what we do downstream. So when you think about the software supply chain, I actually really don’t like that phrase anymore. I actually think that’s the application ecosystem.
I think what we have done as an industry, as we’ve said, protecting applications is a problem that we’re fragmented. Oh, you have a WAF for the operational environment and you do your source code security and you do secrets detection, you do all of these different things. And what we actually need to say is the software supply chain is your application ecosystem from the first piece of code that gets written until it’s out in production.
And so what I want to see is how are you protecting everything in an integrated fashion, not just point solutions that do one thing here, one thing there, but like if you’re doing secrets detection early in code, how are you helping me do some form of tokenization or key management later on? Or if you’re going to do prioritization of defects, how are you also doing application security testing?
How are you also doing your integration with my WAF? So I think we’re going to make this shift. I think Gartner has recently talked about the new ASPM application security posture management as being something like this that says, treat your application ecosystem, which is the software supply chain that does include ingest, but stop freaking out about ingest.
Think about the whole ecosystem. In your environment and how are you protecting it holistically?
[David Spark] Shawn, do you agree with this philosophy of we should be shifting our thinking to the application ecosystem rather than the software supply chain?
[Shawn Bowen] Yeah, I think not just application, but a lot of the other in general, the way we’ve approached all of our securities, we looked at it as layers in the cake and we need to look at the slice of cake that’s go into a particular individual. So that application from top to bottom needs to get its own view because that’s how their development cycle works.
I think the other piece and one of the things that I think is very pointing on yours. It says the greatest value lies in vendors who can seamlessly bridge the gap between security engineering teams. I think one of the failures is on the security teams. When we meet with the vendors, we meet with them by ourselves, and if you’re not bringing your engineering team or your development team or whoever it might be into those meetings to be part of those sessions, you’re starting off behind the curve already.
And so I still, I’ve still this from one of our phenomenal architects, Mike Villis.
He says he steals it from Atlassian, but I only ever hear him say it is get in the path of the developer. And that’s what we try to do. When we look at whatever tooling we have is we try to make sure that the tooling isn’t a security pet that we get to keep and have fun with, but it’s a tool that we both get to use.
We get it for our values, but they can leverage it at the time that they need it during their development, doing their engineering, not showing up a week after they’re done producing or publishing their product.
And we go, here’s your report. You failed. We want to get that at the time of development or time of creation.
And so bringing them in and finding their value stream and seeing how the products can, be part of their build cycle, their CICD process, whatever process you happen to use in your company, you want them to be satisfied with how you’re implementing the tool against them.
And so it’s a partnership that we need to look at internal to the company.
But I do agree that what we need to be from the product space, we need to be looking at things more holistically and not layers, because for me, we have dozens of team squads that are focused on various different products. We have several dozen products that we produce, and if I’m using one tool across it, I look at the single number, but that doesn’t help anyone.
I need to be able to divvy that up and give that to individuals.
And then I go to the next product and I do the same thing. And so being able to, in an ASPM space, look at the vertical slice helps me tremendously.
Surprising research just in!
28:58.418
[David Spark] Since LLMs went mainstream, we’ve been hearing about how they could be used by threat actors. Now, we’re starting to see what that could mean in reality. Last month, Microsoft and OpenAI released a new report about threat actors using LLMs to sharpen their cyberattacks.
Hackers Bait provided research looking at the effectiveness of LLMs hacking sandbox websites. They saw a sea change in LLM effectiveness against a sandbox set of known vulnerabilities, with GPT-4 able to succeed at exploiting 73% of the time after five tries, compared to just 6.7% success by GPT-3.5 and virtually no success with other models.
So Hackers Bait suggested sites use a WAF, patch regularly, and use secure coding practices. All seem logical here, nothing new. But what changes when it’s so much cheaper to attack and the volume of these attacks will go up? I’ve addressed this multiple times, but I want your take on this. It just seems like it’s not different, it’s just more and more effective.
[Andy Ellis] So take this one with a grain of salt because if you keep reading in the study, when they took this GPT-4 and said, “Okay, now that you’ve worked against the sandbox, you’re 73.3% effective. Let’s go hit 50 unmaintained websites,” it found one cross-site scripting attack.
So this is GPT-4 can now basically pass a course in exploiting vulnerabilities, like on a training set. Here’s an academic environment to show you how to do the basics. It gets a C grade against what a human could do. That’s still better than that complete F grade GPT-3.5 was getting, but this is still not at the level you want to have it at, not for replacing anything in your ecosystem for doing your pen testing.
If you went and grabbed GPT-4 and said, “Oh, I don’t need to have a human analyst.
Go look at my website,” you’re completely missing the point here, which is this is good for finding the bare bones minimum. You throw a GPT-4 at it yourself, and it finds 80 things, then you’re in a real world of hurt because it’s not good enough to find anything that a human can look for.
[David Spark] Shawn, I throw this to you.
[Shawn Bowen] It’s not surprising to me. Look at the history of mankind, right? We invent fire, and then we burn people with it. We invent spears, we throw them at other people. We invent guns, we use them for bad reasons. Like everything that’s built for good has been used for bad. Go figure that bad people would find a way to exploit LLMs or any of the AI technology to do bad things.
So not surprising. Yes, I would call into question some of the testing as being a little bit clickbaity, but that’s not to say that AI is not going to be in play. I mean, the things that it’s doing in the medical fields and investing and everything that we’re doing in the space, it’s very clearly just needs the right trainers.
And in this case, the wrong trainers, right?
Uh, and so it’s definitely going to be a problem.
[Andy Ellis] Yeah. I’m not actually convinced that the LLMs are going to be the source of the real problem in website analysis. I don’t think the LLM technology is the right way for AI to push down this path. There’s other paths that then LLMs can manipulate and use, but this, this is more around different sorts of NLP and different sorts of—
[David Spark] But the LLM is just speaking to the speed and the effectiveness and the quantity, I think is really what the issue is.
[Shawn Bowen] Well, so I would first, I mean, obviously I think this is a nuance, but I think it’s going to be SLMs that are going to be the winner in this, the small language models, because you want to get it hyper-tailored. The fact that it’s able to go through generative decision-making, it’s as long as we teach that generative decision tree to be malicious, which I don’t think is going to be very difficult.
I think, yeah, I think it’s going to become a very powerful tool for folks to come at you.
But your ultimate question is, “What changes when the cheaper time?” I mean, the same thing that’s always changed, right? Twenty years ago, denial of service required significant coordination by threat actors. Today, it’s $5 on the dark web, and you get thousands of bots.
So this is the ebb and flow. This is the spar that we are constantly in. Back to my original statement, right? What do I love and hate? We’re never done. As soon as something good comes out, we have to immediately start thinking about how someone’s going to use it for bad. And that’s what I love about it, is everything I learned five years ago is useful today, but not really useful at the same time.
So we’re going to, we’re going to go through the same iteration that we’ve always done, but I’m, as a practitioner in cybersecurity, I’m excited for this fun. Like this is like we’re upping our game, and I hope that we on the blue team side can up our game equally.
[Andy Ellis] Right. I want to see this built into—going to the previous question about the application security ecosystem, the supply chain—like this should be part of your ASPM that’s going to go do things. It should be AI-enabled to say, “Hey, I’m going to do the application security testing, and I’m not just going to like run a script.
I’m also going to run a language model.”
[Shawn Bowen] Andy, now I’m going to get about 12 salespeople pitching me some ASPM thing.
[David Spark] All right.
[Andy Ellis] Call Shawn Bowen. He has budget for this.
[David Spark] Andy, always making our guests’ lives more difficult. Thank you, Andy. That brings us to the very end of the show. We greatly appreciate having you on again, Shawn Bowen, who is the CISO over at World Connect Corporation. I’ll let you have the very last word, but first, let me mention our spectacular sponsor, and that’s Silk.
Remember, go to their website, silk.security, to revolutionize your risk resolution process. We’re talking about the whole process. Deal with everything because finding things is great, understanding what you need to do is great, but actually solving them is ultimately what you want to do. Check them out at Silk.
Andy, as always, we greatly appreciate you being on the show and your great wisdom. I know if those of you who have not picked up his book, 1% Leadership, which by the way, if you ever have a Zoom call with him, it’s always persistently in his background, so you can’t avoid it. And also, just so you know, if you thought my read of the sponsor was excellent, you can credit Andy because entirely he was trying to distract me while I was doing that.
[Andy Ellis] Is it with hand puppets?
[David Spark] Yes. Mr. Shawn Bowen, who by the way, has hired people because of his connection through the CISO Series, are you in any way still hiring at this moment?
[Shawn Bowen] We actually are hiring right now, and we’re going through some org design decisions first, but uh, we are hiring.
[David Spark] All right. Well, he does like listeners to the CISO Series. So please contact Mr. Bowen. There’ll be a link to his LinkedIn profile on this very episode. Any other last words, Shawn?
[Shawn Bowen] I do want to say, as we talked about earlier, this space is stressful, but it is a ton of fun, and you meet a lot of good people. I do not think that there is an industry that has more collaboration than cybersecurity because generally, the rest of the business is competitive to others, so they don’t want to share their secrets.
We have a tight bond with our CISO peers, and obviously, the security professionals within our teams love to bond with others. And so there is a strong community in the cybersecurity space, and I encourage you to reach out and be part of that. And it will, it will help with some of that stress and help you learn and attack your job in a different fashion.
And so please reach out to security folks and join if you’re in San Diego.
[David Spark] Yes. Come tomorrow.
[Shawn Bowen] And then it’s not just about drinking beers, but it’s about meeting people that are going through the same stress that you are.
[Andy Ellis] And drinking beers with them.
[David Spark] It’s also about drinking kombucha because where we are, they’re going to be selling kombucha as well. But if you can’t come tomorrow, we’re trying to aim for the last Wednesday of every month.
I’m sorry. Are both of you doing a talk at RSA? Yes. I know you are—
[Andy Ellis] I am.
[Shawn Bowen] I talk a lot in general, but not on stage.
[David Spark] Not on stage.
[Andy Ellis] Yeah, no. So, so I’m pitching, I want to, I want to push for my RSA talk because I’m doing a talk on “You Can’t Measure Risk.”
[David Spark] It is an anti-FAIR presentation.
[Andy Ellis] I’m anti-FAIR. So if you’re like, “Oh my God, FAIR is the greatest thing ever,” come and feel free to heckle me. Tell me how I’m wrong. But if you’re very interested in understanding how to communicate about risk, you should come find me Wednesday afternoon at 2:00 in the South Center.
[David Spark] Go see him out. Thank you, everybody. We greatly appreciate your contributions and for listening to the CISO Series podcast