We see third-party breaches in the news all the time. Odds are, most of those companies produced clean audit reports and filled in questionaires. If all of that didn’t actually reduce risk, why are we still being consumed with this compliance theater busy work?

This week’s episode is hosted by me, David Spark, producer of CISO Series and Andy Ellis, principal of Duha. Joining us is Vikas Mahajan, vp and CISO, American Red Cross.

Got feedback? Join the conversation on LinkedIn.

Huge thanks to our sponsor, Adaptive Security

Full Transcript

Intro

0:00.000

[Voiceover] Best advice for a CISO, go!

[Vikas Mahajan] My best advice for a CISO is make friends, not enemies. You must learn to get along with your peers. They are critical for you to be able to get the security things you want done. And it’s also important for you to make friends of your key executives.

They are the ones who manage risk and they’re the ones who need to understand cyber risk.

[Voiceover] It’s time to begin the CISO Series Podcast.

[David Spark] Welcome to the CISO Series Podcast. My name is David Spark, I’m the producer of the CISO Series, and joining me as my co-host, it’s one of your favorites. It’s Andy Ellis, principal over at Duha. Andy, say hello to the audience.

[Andy Ellis] Shubh dopeher, ya aap duniya main kahin hain, iske adhar par, shubh rapat, shubh sandhya, ya shubh ratri [Phonetic 00:00:56].

[David Spark] What language was that?

[Andy Ellis] That was Hindi.

[David Spark] Well, you see how little Hindi I know, which is zero.

[Laughter]

[David Spark] For those of you just tuning in every now and then, Andy likes to say a welcome to wherever you are listening to us in whatever time zone you’re in, but the problem is only certain people can understand it. Although you could take that, throw it into AI and say, “Translate that for me in English,” and you could find out.

[Andy Ellis] You could.

[David Spark] Or I could just tell you what he said. It’s not that exciting.

[Andy Ellis] It’s very exciting.

[David Spark] We’re available over at CISOseries.com. If you haven’t checked out our other programming, we have four other shows, why not go check them out? Lots of great shows that deliver wonderful things for your cybersecurity knowledge and entertainment.

Our sponsor for today’s episode is Adaptive Security – security awareness employees love. More about that a little bit later in the show. But first, Andy, I want to talk to you about something, and I’m sure you’ve had this happen either with some organizations that you’re joined, schools that your kids are with, that you see privacy or security violations that make you go, “Ooh,” and you realize they don’t have the talent in-house to deal with that.

And so, I saw yet another one, one where essentially birth dates of all participants were published very publicly to the day and year. And by the way, the information was in custom field number two, which [Laughter] could have easily been removed.

[Andy Ellis] They didn’t even name the field.

[David Spark] Yeah.

[Andy Ellis] Come on. At least name the field “birth date.”

[David Spark] No, custom field number two. I just sent to say, “Hey, you have information about our age. You don’t need to publish our birth dates. Could you please remove this?”

[Andy Ellis] So, is it published just within to the whole community or is it public?

[David Spark] Oh, it’s easy to find publicly, yeah. I mean, it was to the community, but it’s pretty easy to find publicly, yeah.

[Andy Ellis] Yeah.

[David Spark] I just said, “Can you remove this please?” And I think this is just a case of they didn’t do it maliciously. It’s like, “Well, we just did it. We didn’t know what was going on.”

[Andy Ellis] Well, I think there’s two different issues then playing here, right? So, one is like, did they have other data in this document that was also privacy?

[David Spark] Yes, I think what they just said is publish everything. Most of it is perfectly fine to publish, but it was one of the things like, “Oh, these two fields, maybe we shouldn’t have published these,” kind of a thing.

[Andy Ellis] Yeah, within a community, you have this challenge, which is you kind of want to publish everybody’s birthdays to the community so that people know and like can wish each other happy birthday.

[David Spark] But also you don’t need to put the year down either.

[Andy Ellis] Well, the years are kind of obvious if it’s school kids.

[David Spark] Well, school kids, no, but I’m talking, oh, this was all ages, kids to adults, the whole nine yards.

[Andy Ellis] Yeah, I mean, so I try to try to figure out like what could they use this for positively? And then is the problem that they made it public or is the problem that they’d let the community see it?

[David Spark] It’s public is the issue.

[Andy Ellis] And often we combine those two and say, “Oh, you shouldn’t have published it at all.” When it’s, “No, you shouldn’t have published this publicly.” Even just the members of the community, you shouldn’t be publishing their names.

[David Spark] Well, I have another issue with another organization and they’re at the point of ghosting me, which I’m like… [Laughter]

[Andy Ellis] Yeah.

[David Spark] I’m just like, oh God. But their offices are a few blocks from my house, and I think I just have to knock on the door is what I need to do. [Laughter]

[Andy Ellis] That would be funny. You should bring a camera.

[David Spark] Oh, [Laughter] I’m not going to do that. Exposing them will make a problem for me. So, I don’t. I just want to fix the problem [Laughter] is what I want to do.

[Andy Ellis] There you go.

[David Spark] All right, let’s get to our discussion at hand. And actually, a new guest we’ve yet to have on the show, thrilled we’re getting him on. It is the VP and CISO over at the American Red Cross, Vikas Mahajan. Vikas, thank you so much for joining us.

[Vikas Mahajan] David, Andy, it’s an honor to be here. Thank you. Namaste, by the way, and a pleasure to be here.

Are we creating more problems?

4:46.502

[David Spark] “Being busy is the new stupid. No one cares how many hours you logged or how many documents you churned out. They care about outcomes.” That’s CISO Tradecraft host Ross Young’s take on why third-party risk management has become compliance theater.

Most breach companies could have produced clean SOC 2 reports and completed consensus assessment initiative questionnaires, or is it pronounced cake? C-A-I-Q?

[Andy Ellis] C-A-I-Q, but yeah, I think you can go with cake.

[David Spark] Cake. Which means the entire enterprise was box checking, not risk reduction. He advocated for outcome-driven vendor contracts with actual incentives. Pay vendors a base rate plus monthly bonuses if they meet your patching SLAs. It turns vulnerability management into a market-tested system where vendors are motivated to actually fix issues fast instead of just filling out questionnaires.

It’s an interesting theory that I like that Ross proposed. Andy, could this work or is this dead in the water? What do you think?

[Andy Ellis] [Laughter] Absolutely not.

[David Spark] Okay.

[Andy Ellis] And I don’t usually say that. I love Ross.

[David Spark] Ross is great.

[Andy Ellis] Shout out to Ross.

[David Spark] By the way, I like him poking the bear here on this one. He has correctly put his finger on a problem, which has been since day one. Like TPRM is this, are you big enough to fill out my questionnaire test? And that’s all that it does. And like I actually have had this before.

I had a team that worked under the CIO that did vendor assessments, the team was complaining that like half of their people were now doing assessments and like reviewing all of these, and I said, “Okay, so you’ve been doing this like for…” It’s one and a half people, it’s a three-person team.

So, “One and a half of you for the last two years have been doing the vendor assessments. How are we safer because you’ve done them?” And they started to say, “Well, we verified their sphere.” I’m like, “No, no, no. Have we ever not signed a contract because of your [Laughter] findings?

Have we ever gotten a vendor to change what they do?” And the answer was no. And I’m like, “So why are we doing this?” Like they fill out the questionnaire, that apparently was good enough. So, just stop. And then nobody’s going to just be like, “Oh, if you fill out a questionnaire, we don’t even read it,” but that’s kind of the state of the art.

And so, I think Ross is onto something, which is we have a problem. Now, why did I just reject Ross’s suggestion right out of the gate?

[David Spark] Why?

[Andy Ellis] Which is having been a vendor before, I can tell you that this would be laughable. If somebody came to me and said, “Well, look, you’re proposing a 100K a year deal. We’re going to instead pay you 80K, and if you can meet our compliance SLAs and demonstrate them, then we’ll give you the extra 20K.

I’d be like, “Okay, by the way, I’ll take the 20K, but that’s now on top of a 125K deal.” Because it’s going to cost me money to interact with you and your vendor procurement team.

[David Spark] No, but he argues, but the thing is it still takes you time to fill out these darn questionnaires. So, like, let’s not do that, let’s do this.

[Andy Ellis] Oh, do you know how cheap it is to fill out the questionnaires now?

[David Spark] Well, now with the AI tools, yes.

[Andy Ellis] Well, even before we had the AI tools, like, yeah, I had one person at Akamai, I think we ended up with like one and a half, who just did questionnaire filling out as a side piece of their job and then manage customer audits. Because we didn’t know what all the answers were.

We had seen all these. If you sent us, oh, this is the one derived from the SIG, we could just like copy and paste a column of answers. And now there’s startups that do this for you.

[David Spark] Yeah.

[Andy Ellis] It’s like you go, I’m not going to name anybody since we’re careful not to name vendors who aren’t currently sponsors, but there are startups that will, like, you fill out one questionnaire, hand it to them and they’ll fill out the rest of your questionnaires for you.

So, this scales just fine from a vendor perspective. Prove to you that I met whatever crazy SLA you have that you don’t define right in my environment – that’s expensive and will probably take me more time than any of the services we’re providing to you.

So, this proposal doesn’t reduce what you pay me, it’s going to increase what you pay me. That’s why it’s dead in the water.

[David Spark] All right, Vikas, what’s your thoughts? Do you agree with Andy? Or could this or some version of it work, what Ross proposes? Again, this is financial incentives to meet SLAs.

[Vikas Mahajan] Yeah, I agree with Andy in that Ross has an interesting idea. Yeah, it’s a problem. It’s a challenge. The simple fact is we do not manage our third-party vendors’ environments. We will never know the truth of what’s in there. You can certainly try to enforce something, like a reverse SLA is basically what we’re proposing, right?

We’re saying, “Oh, you have to meet our requirements and pay us back, or we’ll pay you more the other way if you meet these requirements.” But again, you have to define some criteria. No one patches at 100% all the time, every day of the week, it’s impossible.

So, then where do you set the bar, right? And then we have legacy systems. You have all kinds of things in an environment. Which systems are you looking at? I think that would be difficult.

My personal opinion is the better approach these days, the business cares about not only reducing risk, but they care about operations. If that is a critical vendor and they’re a key supplier who’s producing something for you, I care more about them being able to get back online and producing whatever it is they’re producing than I do about anything else if I’m the businessperson.

So, I would rather focus on resiliency. I would rather focus the questions I have with them on what are you doing to survive from an attack or survive from an outage of some kind of disruption? What have you done in your infrastructure there? I think the other option also that we should really be considering is inviting these critical suppliers into our tabletop exercises and saying, “Okay, if you want to do business with us, you need to work with us through this exercise so that we have a plan together of how we can survive a cyber incident of some kind.” And I think that’s going to be more productive in the end.

No one’s environment is perfect. It’s impossible, right? We all have to be prepared for the worst case.

Didn’t we solve this already?

10:41.621

[David Spark] “Cybersecurity didn’t get better. We just got better at pretending.” That’s Joshua Copeland of Crescendo’s unpopular opinion about where security stands today. Despite all our new tools, every breach still starts with the same five failures we documented in the 1990s.

Missing patches, credential reuse, misconfigurations, flat networks, phishing. Security teams don’t usually battle state-based actors every day. They’re too busy not doing their homework. Copeland says we built an industry around looking secure instead of being secure.

Now, are we losing our focus on security? I’m going to ask you the cost on this. Are the many market forces such as analysts, research reports, VCs, and media distracting us from doing what we say all the time on this show, which is focus on the fundamentals?

What do you think?

[Vikas Mahajan] Well, I can’t argue that focusing on the fundamentals is absolutely the number one thing you can do in any organization. Blocking and tackling, as we say, right? In a football analogy. You’ve got to be able to patch. You’ve got to be able to identify your vulnerabilities.

You’ve got to be able to keep up with those things. But that doesn’t necessarily mean everything else we’re doing is just theater either. I sort of look at the TSA, right? You walk through the lines. We all hate it. We all go through it, and you’re feeling like it’s all theater.

But there’s things going on behind the scenes that none of us see that are also out there protecting us, looking at things. If you go to Israel, you’ll know they do full background checks on people. There’s a lot more in-depth scanning.

And it’s the same thing with our tools. A lot of the security these days is things that our users don’t see. We’re looking at behavior analytics. We’re looking at the way they’re logging in, where they’re logging in from, what they’re doing, and we’re trying to get some sense of whether that person is who they claim to be, and we’re looking at a lot more data.

Is that making things better? I think overall it is. We’re also doing a better job educating our users about this and making it more of a priority, but there will always be the human element, the human mistake, the human error. And we have to have tools that can help us detect and respond and recover from those very quickly.

But yes, absolutely, you have to do the basics. You have to do them well. But I don’t think what we’re doing is weak.

[David Spark] But we get this complaint all the time. All these sort of entities I mentioned – vendors, research reports, VCs, media – are they distracting us from doing everything you said we should be doing?

[Vikas Mahajan] Good question. There’s definitely a lot of noise out there and you’ll get the same information over and over again. And I think the problem is there isn’t always context. Is something critical, is something out there that you have to worry about?

It’s applying that context to your environment that’s important. Log4j, great example. Huge news for people who have no idea what in the world that thing is and suddenly, everybody’s worried about it. But do I need to worry about it? Do I have healthcare manufacturing equipment for a blood environment that’s using something at Log4j?

How likely is it that a hacker can get into that equipment because it’s on an isolated network? Do I need to drop everything and prioritize that? And then how do I explain that to my executives or my leaders who may be asking, “I heard about this thing, what does it mean to me?” Overall, I’m glad there’s more talk about cybersecurity.

I’m glad there’s more out there that it’s on people’s minds, but we as CISOs, as cybersecurity professionals, have to be able to translate that and explain what that means in our organizations so that the priorities are in the right places.

[David Spark] All right, Andy, I’m throwing this to you.

[Andy Ellis] I’m going to say, and I love Joshua’s when he does these unpopular opinions, but I think he’s just flat out wrong here.

[David Spark] Okay.

[Andy Ellis] I just want to start with that. Security has gotten way better since 1999. If I go back to where I was in 1999, which was doing security for the US Air Force.

[David Spark] But also, it’s far more complex than it was back in 1999.

[Andy Ellis] Right. First of all, the IT environments are amazingly complex, but just to be clear, like we had – in fact, I’ve got one of the boxes right behind me – the Network Systems Group Border Guard 2000 was our line rate packet filter. And literally we were looking for plain text strings because nothing was encrypted that were like, “Oh, people are sending admin commands to SMTP servers.” Like, that’s the level of vulnerability of the things we were dealing with.

This is not like basic, like we wouldn’t even call this basic patches anymore. The amount of just insecure by default and by you’re stuck there software was through the roof. In fact, to connect into this, our security system, we had to use Telnet.

[David Spark] That’s pretty old.

[Andy Ellis] We didn’t have SSH. We’re in the clear logging in across international boundaries because we’re deploying these in theater. So, we had to run an IP tunnel, it was IP type 40, encrypted tunnel to then Telnet across like doing functional Jerry encapsulation – very flaky – just to get to our devices.

Like this is the sort of stuff. In 1999, yes. At a high level. Oh, credential reuse was, yes, it was still a problem.

[David Spark] But everything is more today, like across the board. More attacks, more technology, more complexity.

[Andy Ellis] Let’s talk about basic patch management and I’m going to pull up an iPhone. You can’t see this if you’re just listening.

[David Spark] People know what an iPhone looks like, Andy.

[Andy Ellis] You either have one or one of your family members does. I don’t know how many of you remember when the first iPhone came out what it took to implement patches. You wanted to update your iPhone. You had to install special software on your computer.

You had to download the pad, the update. You had to then plug your phone into the computer using a cable that only worked with that phone and like make the update happen. Now over there updates. You never interact with them. Like your phone just stays secure.

And that’s the default. Your web browser like updates every time you reboot it, and if you don’t reboot it often enough, it says, “Hey, [Laughter] restart the web browser so that I can take my updates, you idiot.”

[David Spark] So, this is an interesting take. In all aspects of our world, we hear people complaining, “Oh, it was much better back in the day,” kind of a thing.

[Andy Ellis] It was not.

[David Spark] And people say, “But the reality is no, it’s better today because you don’t remember.” Also, our memories are short. We’re just worried about right now. We didn’t remember how much stunk back then.

[Andy Ellis] Right. Thirty years ago, there was almost no software, and it was all awful. We were basically trying to secure environments that had no security consideration anywhere in the design environment, and now we’re 30 years later, and yes, there are still problems at the edges, but the core of our environments are functionally secure, and we have built a trillion-dollar economy.

I’m just pulling that number out of thin air. So, please don’t yell at me for making that one up.

[David Spark] [Laughter]

[Andy Ellis] This huge economy on top of the internet and computing, and it’s mostly fine.

[Vikas Mahajan] And all I have to say is, Andy, my EV gets its updates from Tesla automatically. My car, right? I’m driving on this thing. What in the world? How different a world it is today.

[Andy Ellis] Yep.

Sponsor – Adaptive Security

17:45.112

[David Spark] Hey, you remember at the top of the show, I told you that today’s episode is sponsored by Adaptive Security? And here’s something else you might not know. They are the first cybersecurity company backed by OpenAI. Now here’s something you do know.

All CISOs and security professionals know that the playbook has changed. I mean, attackers don’t need malware to get in anymore. They can use AI to impersonate your CFO on a call, clone a vendor’s voice, or send a perfectly written email that looks like it came from inside your organization.

The new attack surface is actually trust. Adaptive is built for that reality. Their platform runs realistic, deepfake, and social engineering simulation so your team experiences these attacks the way they happen in the wild and learns how to spot them and respond fast.

And with Adaptive’s AI Content Creator, security teams can take a breaking threat, a policy update, or a compliance requirement, and instantly turn it into an interactive multilingual training without designers and without delays. So, time-sensitive stuff – pretty darn impressive.

Adaptive is actually trusted by Fortune 500s and backed by Andreessen Horowitz and OpenAI. You can learn more if you go to their website. It’s adaptivesecurity.com.

It’s time to play “What’s Worse?”

19:07.586

[David Spark] Vikas, are you familiar with this game? Two bad scenarios, you have to pick one.

[Vikas Mahajan] Okay.

[Andy Ellis] Okay, just to be very clear, they’re both going to be very awful scenarios.

[David Spark] You’re not going to like either one.

[Andy Ellis] We judge them as presented. We don’t get to create arbitrary things.

[David Spark] You take them as they are.

[Andy Ellis] Take them as is, and I’ll have to go first so you can decide whether you agree with me or not. So, I take the hits of going first.

[David Spark] All right, this is kind of a long one, but I’m going to kind of read through and abbreviate some stuff. This comes from Steve Wingate of CyberGuard Advisors, and he’s actually dealing with a variation of this right now.

[Andy Ellis] Okay.

[David Spark] Okay? Okay. Scenario number one, and I’ll give you kind of the headline for it, and then like the issues that are going under it. So, scenario number one is you have no enterprise-wide AI strategy. Your governance relies solely on an acceptable use policy description.

That is it. Okay?

[Andy Ellis] [Laughter] Okay. So, this is what everybody is doing right now.

[David Spark] Pretty much, [Laughter] yeah. Well, they’re probably not paying attention to the acceptable use policy, too.

[Andy Ellis] No, I think that’s the point. I mean, I think governed by a clause in the acceptable use policy is his polite way of saying…

[David Spark] Whatever.

[Andy Ellis] The lawyers are pointing and saying, “We have a policy, but nobody pays attention to it.”

[David Spark] All right. So, employees are permitted to use AI tools based on general AUP, acceptable use policy, outlines behavioral exceptions, provides no architecture, no control mechanisms, or organizational alignment. All the usage is organic. So, what you got, you got the shadow AI issues, you know, untracked use of every tool.

[Andy Ellis] Okay, are we still in scenario one or is this a new scenario?

[David Spark] Yeah, so we’re still in scenario number one. Inconsistent risk posture, no framework to classify, operational chaos. You’ve got compliance exposure and the leadership doesn’t have a clue what’s going on. Okay, so in summary, it’s governance by policy alone.

[Andy Ellis] So, I’m trying to figure out, do you actually have a policy, or it sounds like you don’t really.

[David Spark] You do have a policy. You totally have a policy, and people are told to follow it. So, the policy is sound, everything else is not.

[Andy Ellis] Oh, okay. So, you have a very sound policy that is well-written and observed only in the breach.

[Vikas Mahajan] Got it.

[David Spark] Hold it. Maybe, maybe not. We don’t know. We don’t know.

[Andy Ellis] No, but I think that’s going to be key when I’m comparing it to the next one, and so that’s why I want to land it before we get to the next one.

[David Spark] Okay, so here’s the second one. There is no enterprise-wide AI strategy and there’s no AUP, no acceptable use policy, but here’s the good news. Technical controls are implemented for a single AI provider, that being Claude, okay?

[Andy Ellis] Mm-hmm.

[David Spark] So, the organization implements access to monitoring controls for Claude. So, SSO, DLP integration, prompt logging, but no enterprise strategy defines broader AI objectives, integration standards, or vendor neutrality, but there’s no adherence or knowledge of an AUP, so employees could be entering anything they want into Claude, no perceived limits.

All right, which one is worse?

[Andy Ellis] So, this one’s fascinating because I think a lot’s going to come down to the definition of that first one. So, my experience of people who point at AUPs and say like, “Oh, our policy is there,” is somebody wrote a policy and nobody actually pays attention to it.

[David Spark] People know it exists.

[Andy Ellis] People know it exists because there’s a checkbox in the security awareness program that says it exists.

[David Spark] Yeah, that’s the unknown. [Laughter]

[Andy Ellis] Yeah. So, I’m going to posit that if I’m evaluating this scenario, I have to assume the AUP is toothless and people don’t actually follow it, it’s not very helpful to them. And so, while we feel good because we CYA’d around a policy, scenario number one is complete Wild West.

Like sure, we wrote stuff down, but there’s no governance, that’s posited. We don’t know what people are doing. So, scenario number two is we don’t know what people were doing, but at least we have securely integrated with Claude. So, given that I think that’s a reasonable way to frame these two, I will [Laughter] take the first one as worse because I have nothing to build on other than a policy nobody reads.

[David Spark] But do you have, and I don’t know the answer to this at all, but do you have better legal protections in scenario A than you do scenario B?

[Andy Ellis] I don’t care about the legal protections on this one.

[David Spark] No?

[Andy Ellis] In fact, I would actually argue that “We wrote a policy and people followed it,” is not actually as legally protecting as people think it is because all you have to do is show. Like this was one of the things, an argument I once had with our general counsel.

We’d just written one of the policies that was like, “You can’t use your cell phone while driving,” and we had an incident, and the general counsel got called onto the incident. He said, “Well, I’m in my car. So, yeah, it’ll take me a minute to respond so I’m going to keep you on mute.” And I said, “You have to get off this call because we have a policy that says you can’t be on the call.” He says, “That’s okay.” And I’m like, “Oh, this is fascinating.”

[David Spark] [Laughter]

[Andy Ellis] Like exactly how effective is that policy at CYA? Because it’s written from a liability perspective that if somebody gets in a crash while they’re on the call, the company wants to say, “Well, they violated the policy.” But if the person who signed the policy violates it, he didn’t mean anybody to follow it.

I don’t think you get real liability protection from that. So, I would much rather be in scenario two where we securely integrated the one enterprise grade AI that we’re using.

[David Spark] All right. Vikas, throwing this to you, do you agree or disagree with Andy?

[Vikas Mahajan] From a risk perspective, a cyber perspective, Andy’s 100% right. You have a controllable environment. You have a place where you have control, right?

[David Spark] No, hold it. Again, control, I’m putting this in quotes that you see, but it’s like now in the first scenario, they are told to do the right thing. Second scenario, they’re like, “Eh, do whatever the heck you want.” [Laughter]

[Vikas Mahajan] Yes, the reality is that’s one fire you have to fight if there’s an issue, right?

[David Spark] Yeah.

[Vikas Mahajan] One place to go, you know where it all is.

[Andy Ellis] Yeah,

[Vikas Mahajan] So, that makes it much matchable.

[Andy Ellis] Yeah, and Claude gives you controls at least that you can go look at of what people are doing. Even if you haven’t turned on the enforcement, you at least have the accessibility to a lot of data.

[Vikas Mahajan] Yeah, yeah.

[David Spark] You have a clue what’s going on, and you could shut things down as you see them.

[Andy Ellis] Yep.

[Vikas Mahajan] Now, back to the first scenario, while I don’t like it as much, I think that’s the reality of where most people are, right?

[Andy Ellis] Right.

[Vikas Mahajan] And I think working within that is not unreasonable either, but clearly, we’re putting a lot of trust into our users and our folks doing the right thing. And that’s a matter of trust in your business model.

[Andy Ellis] This is one of the things, I see this failure actually a lot in security teams, and I’ve had to beat it out of a lot of people who worked for me. The last thing you should do is write a policy, and too many security practitioners think it’s the first thing you should do.

It’s actually the last. The first thing that you do is you go and you secure a system and you figure out how to operate it and you provide operating guidelines. You say, “Here’s how to do it right.” I’m not writing a policy. I’m saying, “Here’s what you should do.

Here’s our key principles.” Then you can write the security practices that say, “Here is actually what we do to secure this.” Then you can write guidelines, and so you basically work your way back to a policy. And so, when you finally write a policy, it’s already being followed and the policy is actually weaker than the guidelines and the practices that you have implemented.

So, people look at the policy and they’re like, “Oh, I’m doing better than that, I’m fine.” Too many security practitioners write the policies. What I want you to do, even though you’re not even on step one, the policy is perfection.

[David Spark] But writing the policy I understand because it’s an exercise to see where we’re all at. You know what I mean?

[Andy Ellis] No, it’s not.

[David Spark] It’s not?

[Andy Ellis] When most people write policies, it’s an exercise in writing the dogma of where we believe we ought to be, and it’s written from this very academic perspective of this is what perfection would look like, ivory tower. And when a practitioner looks at it, they’re like, “Well, I don’t follow that and nobody else in the company does so it’s clearly not the policy.” It’s basically like walking around and telling somebody to get off the playground and everybody else is on the playground because they look at you and say, “You can’t kick me off the playground because you haven’t ever kicked anybody off the playground.” So, they stop listening to you.

[David Spark] By the way, you know what this reminds me of? Is my wife and I used to work together. We no longer work together and luckily, we’re still married. So, there you go.

[Laughter]

[David Spark] I read so many articles about spouses working together and almost every single one said, “Well, don’t bring your work home, don’t bring it.” That is impossible. It is not reality. Yes, that is a sky-high goal, vision. That’s BS. [Laughter] You can’t avoid it.

[Andy Ellis] Well, 90% of the people who write about spouses working together are you both work for the same company, which is very different than you two are the company.

[David Spark] Yeah, yeah. Well, yeah. Well, that was the case at the time, we were at the company.

[Andy Ellis] Yeah.

[David Spark] Vikas, I cut you off. What were you about to say?

[Vikas Mahajan] Oh, I was just going to say that we’re doing, Andy, kind of what you were saying, which was show the users how to use the tools responsibly.

[Andy Ellis] Right.

[Vikas Mahajan] When ChatGPT first came out and then the Microsoft version of it, we actually went through and showed our users, “Here’s how you can responsibly use this type of technology, here’s how it can be of assistance, and here’s not what to do with it.

Do not put this type of data, do not do things.”

[Andy Ellis] Right.

[Vikas Mahajan] And it was, I think, more effective than just me going out and blocking everything and saying nobody can use anything. Then users go and find workarounds. It’s so easy, right, to do that. So, I think that approach is much better.

What’s the starting point for a CISO?

28:38.874

[David Spark] Where do you start when you’re building a SOC from scratch? Now that came up on the cybersecurity subreddit from a new CISO who never worked in a SOC but needs to launch one quickly. The top advice? “Don’t start with, ‘Which SIEM should I buy?’ Start with what are my crown jewels, what can I actually see today, and what can we realistically respond to.” But the overwhelming consensus was actually don’t build one at all.

Outsource it. The math is simple – 24/7 coverage means 1.5 to 2 million dollars in salary alone, plus tools, plus at least five to six people minimum. Commenters said you need a company of 5,000-plus staff to justify it financially. An MSSP or MDR gets you better detection coverage across multiple customers than you’ll build internally.

Some pointed out that an outsourced SOC treats you just like “another client” and then deliver boilerplate service. So, I’ll start with you, Vikas. So, when does building a SOC from scratch actually make some sense versus leaning on an MSSP?

[Vikas Mahajan] I think that’s an interesting question. And yes, it does vary on maybe the type of business you’re in. Maybe if you’re in the federal government contracting space and you need to run and watch over these things yourselves, maybe it’s better to run it in-house.

I don’t really know. My personal opinion is I agree with the latter half, which is in this day and age, it’s better to procure the service. Those folks are trained in how to run and manage and monitor and alert on these things. They deal with it all the time.

And I think I’m going to get a higher quality of service based on that experience and expertise they’ve already built. They already built all the rules for the SIEM. They’ve already built all the rules for everything you need to know. Why not take advantage of all those years of experience and expertise in your environment?

[David Spark] I mean, people are complaining about boilerplate, but I don’t know how boilerplate it really is. I mean, if they can get things up and running quickly for you and maintain it for you, that’s better than you struggling along to get it going, I would assume.

[Vikas Mahajan] I agree. But the best of both worlds is sort of what we’ve tried to implement, which is we manage our tools and we have access to them. I don’t outsource the SIEM logs to someone else. I want to have those so that our teams can do investigations as well.

So, we found vendors and providers who work in that model. It’s our data, it’s our environment, but you have access to it and you’re monitoring the systems. That for me is sort of the happy medium.

[David Spark] All right. I throw this to you, Andy. When does it make sense? You worked with a company that was quite large, so you probably had your own SOC.

[Andy Ellis] We had multiple SOCs.

[David Spark] Yeah, but also it was critical to also to your business model, too, for that matter.

[Andy Ellis] Yes.

[David Spark] But you also work with startups, which I’m assuming probably don’t have their own SOC.

[Andy Ellis] Yeah. So, a lot varies here. You have to ask, what am I actually trying to accomplish? And then you can figure out how you want to deliver it. But my core answer is if your question is, “I want to build a SOC,” I say, “Who is your tier four engineer?” And most people like look at you and they’re like, “What’s tier four?” right?

Tier one is the alert fires. They run very simple triage response playbooks. Tier two is you got to tweak the response playbook, maybe do escalation. Tier three is you get something unusual that you can brainstorm. Tier four is you get something nobody in your company’s ever seen before and you have to go deal with it.

We often call them threat hunters now. Like that’s your first hire is that person. And then you figure out who is doing all the work that escalates to this person because they’re either the person who’s managing your MSSP, not from a contract perspective, but from a day to day.

Like, MSSP has issues. This is who they’re going to talk to, and you have to have that person on your staff. The places where I’ve seen MSSPs go bad is somebody just hires the MSSP, outsources everything to them, and there’s nobody in the company who knows what the MSSP is doing.

So, yeah, you get treated like just another customer. They don’t actually care about what you’re doing because you’ve demonstrated that you don’t care about what you’re doing.

[Vikas Mahajan] Andy, you said it well. You’ve got to manage the vendors, that’s so fundamental, and I think something we as CISOs sometimes forget, or cybersecurity professionals. You can’t just hire someone and outsource it. You do have to manage the relationship.

And that’s the key. If you want good service, you have to put in the time and effort to manage the relationship with the vendor, have those quarterly updates, meet with them regularly, have them take you out to lunch, talk to you about their program, their new features, their new capabilities.

If you’re showing interest in them, they’re knowing that you’re watching their services and they want to deliver. So, definitely you have to have that relationship.

[Andy Ellis] And then the second thing that I would suggest is, do you already have an IT help desk? Because if you do, odds are your SOC should be tightly tied into your help desk services as well. You already have an operations team. You should not be trying to stand up one completely in parallel.

You should be standing up more capabilities inside that team. And maybe they’re going to manage your MSSP for you. Now you’ve got a spare help desk. Or maybe you can just take your security alerts, and if you’ve written good playbooks, you can hand them to your help desk.

Could this possibly work?

33:56.862

[David Spark] “Shift left has become a cringe-inducing or eye-rolling phrase in cybersecurity,” writes Derek Fisher of Temple University. It’s simply just saying, “Be proactive with your security program.” Fisher proposes lightweight threat modeling at the user story level, using Adam Shostak’s Four Questions.

What are we building? What can go wrong? What are we going to do about it? Did we do a good job? Simply a conversation on these topics before you dovetail into requirements code or penetration test. Fisher admits he’d prefer a more thorough process, but when speed is the business priority, this might be the best option.

Does this gut check conversation water down security to make it palatable to developers? Or could this actually help us with the mystical shift left, Andy?

[Andy Ellis] So, this is sort of a weird one because like shift left gets a bad rap because it like means a lot of different things besides do security architecture reviews early.

[David Spark] It was also overused for a period of time, too.

[Andy Ellis] Oh no, it’s still overused, just to be clear.

[David Spark] By the way, I just want to say everything overused in cyber gets slammed, period. [Laughter]

[Andy Ellis] Right, but that said, let’s just take the nuts and bolts here, which is when you’re the security person who’s first engaging with a product, what’s a great way to start the conversation? This is a really fine icebreaker because all too often what happens is security engineers show up and they say, “Here is my list of requirements,” and they don’t know what they’re applying to.

And all that it takes is for you to be grossly wrong on one point to completely undermine any authority you might’ve had, right?

If you walk in, and I’ll give you a great example, I had this with the FedRAMP program office, wanted to fail us because we didn’t monitor the humidity in our data centers because they had this one requirement – you must monitor humidity. And I’m like, “First of all, it’s not a security requirement.

And second of all, you wrote that because you were going after people who only have one data center. I’m in 2000, I don’t care about humidity in them.” Like, “I assume that the people who own 95% of the space in every data center care about humidity.” “I don’t, have a nice day.” And it completely undermined their authority on other things when they lost that battle because they invested in that battle.

I blew them off, I succeeded.

When a security team walks in and says, “You need to do X,” and they’re like, “Well, I see in X, you demand I do this one stupid thing,” and they win the battle, they’re going to stop listening to you for everything. So, instead, walk in and say, “What are you doing?

How could it go wrong?” Like, get me started. You’ve already thought about this. What are your failure modes? When those happen, what’s your plans? You’ve already done security work, right? It comes in and it respects the work of the developer and puts it onto a positive footing.

They get to teach you and now you have context. So, when you say, “Oh, I see that you are taking user inputs. How are you managing filtering and prompt injection and dealing with that?” Whereas if you walked in and said, “How are you dealing with prompt injection?” and they’re like, “I don’t have an LLM,” you just sound like you don’t know what you’re talking about, which is going to hurt the conversation.

[David Spark] All right, your take.

[Vikas Mahajan] I don’t have a perfect answer for this. I think the best approach here from my thoughts are, yes, you have to understand the application, what it is they’re trying to build or buy or whatever the case may be. And that’s when you have that conversation.

Now, usually in some environments, it starts with the business. What is the business trying to do? What is their goal? What are they trying to accomplish? And from there, we start working into the level-down details. Okay, great. Like you said, you’re getting input for this.

Okay, now have you thought about the failure scenarios where those things happen? The reality in organizations though, is you’ve got lots of projects, you have lots of things getting thrown at you, and it’s hard to sit down and have a lot of those conversations all the time.

You do have to have an answer. The executives of a business are sometimes looking for that, “Hey, just give me your gut check. Does this sound like the right thing to do? Or do we need to go back and look at it?”

And I also bring in third parties here because in some organizations, ours included, we rely on third parties to develop some apps for us. And I have to go back and make sure are the third parties doing their due diligence and writing code securely and doing things the right way.

If we get something thrown over the fence to us, I don’t want the business just taking it and running with it. For me, I like the idea of, instead of bringing the horse to water, which is bring all of these developers and folks into the security world, I think it’s going to be much more effective to bring the water to the horse, bring the security capabilities to the people, embed it in the tools they’re working with, in their Git, whatever it is they’re using, right?

Put those things in their hands and then train them on how to use it and understand what it means. And then provide that added value, which is I’m your trusted advisor. I can help you understand what this thing is telling you to do if you don’t necessarily know.

And we’re here to help you through this process, not here to just be a sort of auditor checklist, make sure this is done, or a top 10 thrown at you kind of a thing.

[David Spark] Excellent.

[David Spark] Well, I’m glad we’re putting a little bit more context around the shift left malignment that has happened. I honestly didn’t think it was getting maligned, but again, also, like I said before, take the zero trust philosophy. It makes sense, but people still slam it too.

Andy?

[Andy Ellis] Yeah, well, the challenge with shift left is, like zero trust, everybody has adopted it. So, every vendor at some point has said, “Oh, well, we’re helping you shift left,” which really was just like how do we get from being completely reactive.

[David Spark] It didn’t start that way. It didn’t start that way. Like, it would start, “We have a zero trust solution,” or “We’re a zero trust answer,” or “We’re a shift left answer,” rather than “No, we’re part of the solution to do that.”

[Andy Ellis] Right. Well, and zero, like I remember being at a conference where after it, I got pitched. It was a student conference and after, all the students came up to sell me their ideas, and every single idea mentioned zero trust. Half of them mentioned blockchain.

[Laughter] I was just like, “Okay, did you basically pick a couple buzzwords and then try to say how this is?” And that’s where the malignment comes from is everyone wants to use the buzzword because they feel the buzzword’s successful, even if they have nothing to do with it.

Closing

40:18.045

[David Spark] All right. Let’s come to an end of this show. Thank you very much, Andy. Thank you very much, Vikas. This has been a great show. Both of you just slammed way too much knowledge in way too much period of time, and I greatly appreciate it.

And we also greatly appreciate our sponsor and that would be Adaptive Security. Remember, it’s the security awareness employees love. You can learn more about them if you go to adaptivesecurity.com, and when you go there, let them know that you heard about them through the CISO Series.

Thank you very much, Vikas. Thank you very much, Andy. And to our audience, as always, we say this – we greatly appreciate your contributions and for listening to the CISO Series Podcast.

[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. We have lots more shows on our website, CISOseries.com. Please join us on Fridays for our live shows – Super Cyber Friday, our virtual meetup, and Cyber Security Headlines Week in Review.

This show thrives on your input. Go to the Participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com.

Thank you for listening to the CISO Series Podcast.