Measuring a CISO’s performance can be tricky. For a while, a company getting breached was a “resume-generating event” for many CISOs. But as security incidents become eventualities rather than possibilities, how can we advance our understanding of a CISO’s performance beyond a “scapegoat-in-waiting?”
This week’s episode is hosted by me, David Spark (@dspark), producer of CISO Series and Andy Ellis (@csoandy), partner, YL Ventures. Joining us is Mike D’Arezzo, executive director of infosec and GRC, Wellstar Health Systems.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, ThreatLocker
Full Transcript
Intro
0:00.000
[Voiceover] What I love about security vendors. Go!
[Mike D’Arezzo] 20 years ago, security vendors would just throw out whatever they could and whatever stuck, stuck. What I like today is most of them are purpose built. Most of them come with an idea that they’re directed to sell, and then they solve real world problems now.
[Voiceover] It’s time to begin the CISO Series Podcast.
[David Spark] Welcome to the CISO Series Podcast. My name is David Spark. I am the producer of the CISO Series. My cohost for today’s episode is Andy Ellis. He is the partner over at YL Ventures. Say hello to the audience, Andy.
[Andy Ellis] [Foreign language 00:00:47]
[David Spark] And the language you were speaking was?
[Andy Ellis] Slovak.
[David Spark] Yeah.
[Andy Ellis] I think I’m now done, and I’m going to go back to only English.
[David Spark] Really?
[Andy Ellis] I think this joke has had its run.
[David Spark] Or you could just not do it. There’s another idea.
[Andy Ellis] But I need to have a formulaic greeting.
[David Spark] Do you?
[Andy Ellis] Yes, I do.
[David Spark] No, you don’t.
[Andy Ellis] Nope. Otherwise I’m stuck trying to make something up and think on my feet, and that never ends well.
[David Spark] Yeah, well, you don’t need a formulaic greeting. Our sponsor for today’s episode, by the way, is ThreatLocker. Zero trust endpoint protection platform. We’ll be talking about that a little bit later in the show. ThreatLocker, an absolutely spectacular sponsor of the CISO Series. For those of you who haven’t been to our website, CISOseries.com, please go there.
Lots more programming you can see. So, we do want to mention something up front. I’m going to bring in our guest now because he is connected to this person. So, our guest on today’s episode is the executive director of InfoSec and GRC over at WellStar Health Systems. None other than Mike D’Arezzo. Mike, thank you so much for joining us today.
[Mike D’Arezzo] Hey. Thanks for having me, Dave. Appreciate it.
[David Spark] We were on a CISO call, and I loved how he was engaging with everybody on the call. And I was like, “This guy would be great on the show.” So, no pressure, Mike.
[Mike D’Arezzo] No pressure.
[David Spark] Now, I do want to bring something up. This is April when this is dropping, but we’re recording this when an unfortunate tragedy happened in the cyber security industry. Our most frequent guest who has been on all of our programming… We adore him. We has been a cheerleader for us. We have been a cheerleader for him.
I mean he was just so supportive of us all the years. He unfortunately passed away two days prior to this recording. I’m speaking of Shawn Bowen, who recently just got hired on as the deputy CISO for the gaming division over at Microsoft. If you don’t know this, I have a newsletter that I posted with my thoughts and my team’s thoughts about it.
And we have a few photos. Andy, you said you actually taught Shawn at one time.
[Andy Ellis] Yeah, he came to an executive ed class I was teaching at Harvard. I love Shawn. Fantastic character. I also originally met him through a mutual friend. Really loved working with him. So, I hope his family is finding comfort as they mourn, and his memory will definitely be for a blessing.
[David Spark] I can’t speak to how phenomenally he supported us. I mean, to think… All the jobs… He was working as a CISO, working for Microsoft, working prior to that at World Fuel Services as well. He was so overwhelmed. But he always was able to make time for us, which was incredible. And by the way, he was the person we called when we had a last minute cancellation.
And almost every single time he would come through for us, which was incredible. Mike, what’s been your connection with Shawn?
[Mike D’Arezzo] So, I met Shawn I believe at an ISC2 conference a number of years ago. He was just in a circle, and we were all talking. Just introducing each other. Very warm, friendly individual. And reached out, as we all do. We kind of bond together in our misery in support of our CISO and security jobs.
And he was just a good human being.
[David Spark] Yeah. Can’t speak more highly of him. So, we have more plans for remembering Shawn over time. But we’re just acknowledging this now. When you’re hearing this, this is going to be a good solid two months after the horrible incident happened. But we just want to thank him. And by the way, if you want to hear more of him, just literally go to CISOseries.com.
Type in his name, Shawn, S-H-A-W-N, as his first name. And then Bowen, B-O-W-E-N. He appeared a lot on our network. We are going to miss him terribly.
Surprising research just in!
4:39.628
[David Spark] A big reason to embrace the Shift Left mentality is that fixing a bug before it hits production will save you a ton of money in the long run. Or does it? Chris Hughs of the Resilience Cyber Podcast points to a recent CISA advisory that found this theory of reduced security vulnerabilities which will save your business’s bottom line in the long run dates from an unofficial IBM research body from the 1980s.
He pointed to a recent Comparitech study that found company’s stock prices returned to pre-breach levels within 53 days of disclosure on average with an average price dip of 1.4%. Now, there are obvious good cyber security reasons to care about fixing bugs early. But I got to ask you, Andy. I’ll start with you.
Is Shift Left stalling out because there isn’t a compelling business case for it? What do you think?
[Andy Ellis] So, I love that people have finally twigged to the fact that…the whole like 10X cost for every waterfall stage later you fix something was never even a study. Like unofficial IBM research body? It was a think piece somebody wrote. I’ve known this my whole career. It was a thing that was very obvious.
It was like how did you find research data that shows exactly 10X per stage? Oh, you didn’t? And yet somehow it has persisted. Because logically it does feel right. And it does but not for bugs. For software defects. Things are architecturally mis-designed get ruinously expensive the longer you leave them there if you’re ever going to fix them.
So, important thing to note. Somethings you’re never going to fix, and that’s okay. But somethings you will need to fix. And the more people who are using it, the harder it is to fix. I also want to look at this Comparitech study because I’ve seen a bunch of others like this. Pre-breach levels isn’t what’s interesting.
And 53 days probably is short enough that it might be. But the question is everybody’s stock is slowly going up over time. Like the market goes up. Do you lose 53 days permanently? That is to say now you’re sort of reset at a slightly lower level than you should have been. Or are you continuing to go up with the market?
I’d like to see more studies sort of showing a trend analysis of the market rather than just looking at my one line.
[David Spark] That is a really good point.
[Andy Ellis] Because just the quantitative easing keeps dumping money into the stock market. So, everybody’s stocks are going up. If you’re not going up then there’s fundamentally a problem with you if money being plentiful isn’t showing up there. So, that’s just looking at the data. High level, I think the biggest challenge of Shift Left is most of the vulnerabilities we tell people to fix don’t matter.
And all you have to do is tell them one that they can look at that doesn’t matter, and they will not listen to the 999 that do. Unfortunately we’re often in a stage of telling them like, “Here is 999 that don’t matter, and there’s one in here that will matter. Good luck finding it.”
[David Spark] All right, Mike, I’m throwing this to you. First of all, I think we all bought in originally to the Shift Left philosophy because it seems like common sense. Doesn’t it? This is just sort of a theory we generally adopt in all parts of our lives. Let’s deal with this problem early in our home.
Let’s fix the pipes now so when we…so we don’t have a water leak later that costs us a fortune in house damage. So, what do you think?
[Mike D’Arezzo] Yeah, so I couldn’t disagree with Andy more. No, I’m just kidding.
[Laughter]
[Mike D’Arezzo] I think…
[David Spark] Oh, by the way, you don’t know how much I enjoy just hearing that phrase. We are, by the way, going to… We’re going to pull that clip…
[Mike D’Arezzo] This was the whole reason David brought me on here. Was just so I can say things like that.
[Crosstalk 00:08:22]
[Andy Ellis] You take that right out of context.
[Laughter]
[David Spark] “I couldn’t disagree with Andy more.” It’s going to go on my sound board. I’m just going to keep playing it over and over again.
[Laughter]
[Mike D’Arezzo] So the reality of it is is when we grew up as kids and we had somebody as a mentor kind of show us, “Hey, you’re going to do this. You can do it right the first time, or you can do it again.” How many times have we heard that? So, kind of the same principle as Shift Left is let’s focus on getting this right, getting it faster, getting it done better, getting it all involved.
So, how do I get the people on the left hand side further in development, smarter, better about creating secure programs? So, I need less of the right hand side. Less of the security vulnerability experts, less of the… Because those people cost money, and those people don’t generate revenue. Right? They don’t.
Security people don’t generate revenue. Get the developers developing more stuff. Right the first time. They can continue on doing more feature requests and things like that. That’s why I think Shift Left made sense. Fake supportive data, alternative facts, whatever you want to call it, be damned. I think we were all like, “Hey, let’s focus on making more changes better in feature request.”
[Andy Ellis] I just need to point out that Akamai has a multibillion-dollar security business that resulted from the belief that security people could bring in revenue.
[Mike D’Arezzo] This is true. This is true. This is not true.
[David Spark] What is it? Is it true or is it not true?
[Andy Ellis] Is it true or not true?
[Crosstalk 00:09:46]
[Mike D’Arezzo] No. I don’t know where you got those numbers from. You know, 70% of all statistics are made up on the spot, Andy.
[David Spark] [Laughs]
[Andy Ellis] 73.5% of statistics are made up on the spot.
[Crosstalk 00:09:55]
[Mike D’Arezzo] You’re right, 73.5.
[Andy Ellis] You have to make it sound plausibly like you didn’t make it up. That was the problem with the original IBM study was they just said like 10X per stage.
[Mike D’Arezzo] Right.
[Andy Ellis] So, it was very obvious this was not actually a study, but somebody’s like whitepaper. I would say PowerPoint, but this predated PowerPoint.
[Laughter]
Can this be measured?
10:14.223
[David Spark] Measuring a CISO’s performance is easy, right? If they didn’t get hacked, they are doing a great job. So, security journalist, JM Porup, points out that the CISO as “scapegoat in waiting” remains distressingly prevalent. But he has a really great encapsulation of what the CISO is actually responsible for.
The CISO, “should be held responsible for identifying risk, for making good recommendations for managing risk, and then executing to meet the risk tolerance of the business within the given budget of time and money.” I will start with you, Mike. Does that sound good to you? And if so, how do we start measuring a CISO’s performance?
[Mike D’Arezzo] So, yeah, keep us from being hacked, that’s the default. And I think a lot of times that’s a true point. Did we get hacked? But a lot of times… Andy knows this. It’s not my fault… I could have every tool, every process in world in place, and we’re still going to get hacked. We’re still going to get breached.
We’re still going to… And some companies hire CISOs just for that fact. You’re going to be the fall person in this case. So, no, I don’t agree that that’s a good statistic. I think better statistics are did you streamline operations by reducing the amount of failures across vulnerabilities and patching, and did you stick to your maintenance periods.
Did you have things like that? onyxia.io has a good product that shows all these things. Gartner has some good tools that kind of showcase what makes an affective CISO. But the hacking thing, it’s just… Unfortunately it’s just going to happen.
[David Spark] All right. Measuring a CISO’s performance. Andy, do you agree with this recommended assessment on how to do it? It’s actually risk management advisor.
[Andy Ellis] I mean there’s a risk management advisor role. But the first thing that’s important to remember is that CISO is not a singular job. It is many, many jobs. And so there’s not one way to grade a CISO because if your CISO isn’t a risk advisor to the business then why are you using risk advisor grading?
So, this is like I wrote the idealized CISO job description. Each one of those should probably have its own way to measure it. I like to measure success rather than failure. Because this really feels like it’s how do you know if your CISO failed. And at the end of the day if your company got breached, it doesn’t matter if you wrote it down.
That’s your failure. Suck it up. That’s part of our job description is helping the business make wiser risk choices. You failed at that. The fact that you wrote it down and they were like, “Okay, fine,” that’s as failure of communication.
[David Spark] Hold on. I’m wondering, Andy. In your career, have you been chewed out before for failing, what was perceived failing?
[Andy Ellis] Absolutely. For both failing and for perceived failing.
[David Spark] [Laughs]
[Andy Ellis] I have been chewed out after somebody four years after they made a risk decision that I had briefed them on making the risk decision, that their team had made the risk… In this case they had chosen to mitigate a risk, but it was expensive. And four years later, all of a sudden the bill is coming due.
And they’re like, “We should have never made this decision.” And I’m like, “You made the decision. I have the notes from the meeting.” And they’re like, “Well, I wasn’t properly informed.” And I’m like, “Well, here’s the briefing we presented you with.” I got chewed out. At the end of the day, they did not like the decision that they made.
And it was me, as the advisor, who had helped guide them through that decision process. And everybody was on board. Their direct reports had agreed with the decision at the time. And now all of a sudden it’s an unacceptable one. And rather than just saying, “Business has changed. We’re changing our decision.” Very important that there was a scapegoat.
I was the scapegoat.
[David Spark] What do you get then padded on the back for? I’ll start with you, Mike.
[Mike D’Arezzo] So, my job and what I do today is more solutions driving. So, as a CISO, I have folks that want to do certain things. And, “Hey, I want to do this.” And we don’t do the, “Let’s protect after the fact.” We’re like, “Hey, I want to be the K-N-O-W police.” The know police is what I tell people.
I don’t want to be the N-O. I just need to know about it. If I know about it, I’ll help you build a solution. I started my career in IT. So, if I can help you build a solution and do that, that’s where I get patted on the back. You enabled this to happen in a secure manner, and everybody is happy. Right?
It’s a win, win, win situation. That’s what I get patted on the back for.
[David Spark] And, Andy, what would you add to that?
[Andy Ellis] So, I love to add near misses. Like my whole job is if I have succeeded getting people to make wiser risk choices. There will be evidence of that wisdom by near misses. “Oh, we had an incident, but it wasn’t that bad because you had this control in place.” Like, “Oh, edge system got breached, but it couldn’t go anywhere because all of the nonhuman identities were well controlled thanks to this project we rolled out.” Like whatever it’s going to be, that’s what I do.
I take those, and I go celebrate the people who did the work. Everybody knows I’m patting myself on the back for getting them to do the work. But now more people want to do work because I will point out that you saved the company.
Sponsor – ThreatLocker
15:11.984
[David Spark] Before I go on any further, I do want to talk about our absolutely spectacular sponsor. You know who I’m talking about. It’s ThreatLocker. As we all know, cyber security isn’t just about fighting fires. It’s about making sure they never start in the first place. Aw. Maybe a think back to Shift Left here.
That’s where ThreatLocker comes in. With ThreatLocker’s deny by default approach, nothing runs on your network unless you say so. It’s like having a digital bouncer guarding your organization, keeping out ransomware, zero day exploits, and sneaky supply chain attacks. Plus you get a full auditorial of every action because visibility is power.
ThreatLocker’s US based support team makes setup seamless, so you can stop worrying about vulnerabilities and start focusing on what matters most. That’s why thousands of companies trust ThreatLocker to keep their business running and secure. You can take control of your business’s cyber security today.
Visit threatlocker.com to learn more. Go check it out.
It’s time to play, “What’s Worse?”
16:21.261
[David Spark] It’s time to play, “What’s Worse?” All right, Mike, I know you’re aware of this game. Yes?
[Mike D’Arezzo] Yep.
[Andy Ellis] But you already used up your phrase, “I couldn’t disagree with Andy more,” so now you’re required to agree with me.
[David Spark] No. Do not listen to him at all.
[Mike D’Arezzo] Yeah. You know, I’ve got a few of them.
[Andy Ellis] Yeah. There’s a budget.
[David Spark] There is no budget. I want you to expend as many of those as you would like, Mike.
[Mike D’Arezzo] All right, you got it.
[David Spark] All right. I will make Andy first, and then you will agree or disagree with him. And this comes from Jerich Beason.
[Andy Ellis] Oh, man. This is going to suck. Jerich, you’re not allowed to put in scenarios anymore. I just already know that. His are always hard.
[David Spark] This one, I… I’m kind of leaning towards one over the other on this one, so I’m fearing it might not be balanced.
[Andy Ellis] Okay. But his usually get a disagreement if I recall correctly.
[David Spark] Yeah. Well, let’s see how this goes.
[Andy Ellis] Right. I may have just pumped him up, and it’s not going to deliver.
[David Spark] We’ll see what happens. Jerich is currently the CISO over at WM. Here. This is what he asks. What’s worse? You’re a new CISO joining an organization post breach. You get one of the following. You inherit a team with no experience, but each person has three certifications in security, none of which are for the cloud environment you secure or for the tools you use.
Or you inherit a team with no experience, but each person has a masters in cyber security. What’s worse?
[Andy Ellis] So, this one is kind of tricky because the correct answer is banned by the rules of, “What’s Worse?” Which is I’m replacing either of these teams. Neither of these teams is fit for duty. That’s probably why you had a breach.
[David Spark] No. This is… You got to pick one of these teams. [Laughs]
[Andy Ellis] But I have to pick one. It says you inherit. But like I could try to tweak it, but we’re not going to tweak it. I’m stuck with this team because…
[David Spark] One or the other.
[Andy Ellis] Okay. I’m going to add in, the company has a hiring freeze, and anybody I let go I’m not allowed to backfill. So, I’m stuck with these people. So, I have a bunch of people with masters degrees and no work experience.
[David Spark] The other one also… Neither team has work experience.
[Andy Ellis] No, no. I have a group with experience and certifications but not experience in the systems I run.
[David Spark] Correct.
[Andy Ellis] Okay. I want that team. Probably I hope… Did they…? They had some experience. Great. I want that…
[David Spark] No, no, no…
[Andy Ellis] They have no experience.
[David Spark] They have experience solely through the certifications. That’s it.
[Andy Ellis] Oh. So, I have two teams.
[David Spark] You inherit a team with no experience.
[Andy Ellis] It’s no experience at all.
[David Spark] But each person has three certifications in security, none of which are for the cloud environment.
[Andy Ellis] Oh, okay. But they got them through some quickie bootcamp, and…
[David Spark] Could be. You don’t know.
[Andy Ellis] But no job experience for either team.
[David Spark] And it’s not for the cloud environment or for the tools you use. So, all their certifications are on something else, doesn’t reflect anything in your environment.
[Andy Ellis] But that certification doesn’t actually include job experience either. That’s what I’m trying to clarify.
[David Spark] Yeah, I know. Yeah, it doesn’t. So, there is no job experience anywhere.
[Andy Ellis] No job experience anywhere. So, either I get people with three random certifications that do not apply, or I get people with a masters degree in cyber security?
[David Spark] Yeah.
[Andy Ellis] I’m taking the team with the masters degrees. Sorry, all the certification people. The three certifications that do not apply to any experience you have are not a thing I’m going to value. I’m going to value that masters degree in cyber security slightly more but only slight more. Just to be clear.
Not one of my favorite things. But I’m going to take those folks.
[David Spark] All right. So, Andy is saying the first scenario is worse, the three pointless certifications.
[Mike D’Arezzo] So, I do want to double…
[David Spark] By the way, in general… Hold on. Before I go to you, Mike. In general, what percentage of certifications are pointless, Andy?
[Andy Ellis] Am I allowed to pick a number higher than 100%?
[Laughter]
[Andy Ellis] But I really, really want to clarify, which is getting through the recruiting pipeline often requires certifications.
[David Spark] I know.
[Andy Ellis] So, I do not want it to sound like I’m just dissing on all certifications. But the experience is what matters.
[David Spark] Yes, I know.
[Andy Ellis] If I thought originally that I had this group with certifications had experience on different systems, in which case I would have taken them in a heartbeat. But they have no experience and certifications. That’s not actually something useful. It’s like, “Oh, so if I put a system in front of you, you might know how to click the buttons,” because that’s what most certifications are teaching you.
But you’ve never even used that system before, let alone the ones we’ve got. So, yeah, that’s actually almost like a negative signal for me. Because I assume none of you have a masters degree. Probably a lot of you don’t even have a bachelors, for what value those are worth. That’s why I went the way I did.
[David Spark] Let’s give Mike a chance to speak.
[Andy Ellis] I’m trying to make sure Mike doesn’t disagree with me by hitting all the arguments before me…
[Crosstalk 00:21:05]
[David Spark] You’re trying to make sure Mike doesn’t disagree with you?
[Mike D’Arezzo] So, I hear what you’re saying. Let me add a counterpoint to that. So, I like what you asked, David, about the question of are these experienced [Inaudible 00:21:17] because a CISSP requires several years experience. Even if you have other certifications. There’s at least a minimum of three or four years experience.
[David Spark] So, they didn’t get that certification. I’ll tell you that much.
[Andy Ellis] On paper, it requires that.
[Mike D’Arezzo] Yeah. It’s not kobayashi maru. Like no win-no win situation.
[Andy Ellis] But it didn’t say the CISSP. These could just be like you went to some security bootcamp, and you got your whatever, whatever, whatever.
[Mike D’Arezzo] It’s an Azure Security…
[Andy Ellis] Right. You got an Azure Security cert.
[Mike D’Arezzo] Architecture.
[Andy Ellis] And I’m using AWS or a GCP shot.
[David Spark] And you got hired by an AWS company.
[Mike D’Arezzo] Right. And the question you didn’t ask is are these CPE driven ones. So, you talk about experience with a masters degree. And they do say masters generates a five-year…the [Inaudible 00:21:58] of a five-year experience within that education. I will say the CPEs, the certified professional education, studies that you’re required to maintain some of these certs require that experience.
And if you don’t have it… You do training, you do this, you do that. And there’s a consistent training.
[David Spark] People can get CPEs listening to this show. I mean, the CPE is kind of vague.
[Andy Ellis] Yeah. Yeah, the CPE… Look, I host webinars all the time. And I will tell you that if you run a webinar at noon Eastern, which is one of the common times to do it, basically you get a lot of people who show up and then walk away from their computer because they just want the CPE credits for having done it.
The CPE system is worthless at this point, in my opinion.
[Mike D’Arezzo] All right. And this show today is brought to you by Certifications-R-Us.
[David Spark] [Laughs]
[Andy Ellis] One of these days I will register and incorporate the formerly certified Security Professional Certification so that you can just get rid of all your other ones and keep ours.
[Mike D’Arezzo] There you go. There you go.
[David Spark] All right, Mike, come on. Stop, Andy. Mike, you need to give your rationale and who you’re picking here. Which of these two scenarios?
[Mike D’Arezzo] Yeah, so my rationale is I actually like the certifications more.
[David Spark] Aw, so we’re disagreeing with Andy here.
[Mike D’Arezzo] You’re a highly educated person, Andy. You established this early on. And I’ve known you I think off and on for a few years.
[Andy Ellis] Yeah. But I don’t have a masters degree. I think the masters is a negative, and the certs is a negative. It’s just which one is more.
[Mike D’Arezzo] It can be, yeah. So, I got my bachelors later on in life. I’m actually pursuing my masters now. And I’m really getting the masters really just kind of to stand out a little bit. My certifications have really kind of exposed me not just to route memorization… “He can memorize a test.” It’s the networking, and meeting people, and exposure of ideas, and everything from other people.
The ISC2 conference where I met Shawn that we talked about at the beginning. I think there’s a hidden quantification there that’s not talked about. And I know not everybody goes to certification conferences to meet people.
[Andy Ellis] And ISC2 is not what it used to be.
[Mike D’Arezzo] It’s not. And shame on you folks out there. You go to a conference, meet the person to the left of you, meet the person to the right of you. Just please if you go to conferences at least meet people. But I…
[Andy Ellis] Go sit at a random table for lunch and talk to people you don’t know.
[Mike D’Arezzo] Bingo. Talk to people you don’t know.
[David Spark] Yeah. Can I tell you? This is my number one pet peeve is when you go to a conference with your five coworkers and you sit and eat with your five coworkers. Stop doing that.
[Andy Ellis] Don’t do that at all.
[Mike D’Arezzo] Don’t do that.
[Andy Ellis] Yeah, Jerich, this was a good one. Because the reality is Mike and I are both sitting here going, “We would take anybody with experience.” I will be honest, I was going to agree with Mike originally because I thought the certifications at least came with experience. Give me a certified Azure engineer from a GCP platform over some random with a masters degree absolutely in a heartbeat.
[Mike D’Arezzo] Yeah. Comp TS, Sec+, you’re like, “Uh… You know, every three years you gotta retest for that.” That’s a hard one. It’s a tough one.
[Andy Ellis] Yeah. But I’m assuming if they have not retested yet, it’s because they have no work experience.
[Mike D’Arezzo] Right.
[Andy Ellis] Like this feels like somebody…these people are what we get a lot of. And I think, David, you… Or maybe it’s somebody else that I saw ranting about this was one of these bootcamps that told you not only would it get you to a certain point, but then you could put on your resume that you were an intern for them so that now you have job experience, too.
[David Spark] I did not post that at all.
[Mike D’Arezzo] Oh.
[David Spark] [Laughs]
[Mike D’Arezzo] No.
[Andy Ellis] Okay. I saw somebody… I think it was in a CISO Slack channel I was in.
[Mike D’Arezzo] That’s terrible.
[Andy Ellis] Right. And this is the problem, is the certifications have basically become a scam. And not all of them are, but we can’t tell them a part anymore.
[Mike D’Arezzo] They were a scam 25 years ago. When people were like, “I’m Microsoft certified,” you’re like, “Great. I need you to give me a subnet based down from these IP addresses.” And they’d just look at you, and you’re like, “You don’t know anything about subnets, do you?”
[Andy Ellis] Yeah. Okay, the Cisco certification… Like the CCIE matters.
[David Spark] All right, this is going off the rails here, gentleman.
[Andy Ellis] Yeah. What’s worse?
[David Spark] Mike, what’s worse? You said the second scenario.
[Mike D’Arezzo] I think the masters degree is worse. I’m going with team cert.
[Andy Ellis] Okay, Jerich, you win.
[David Spark] There you go. We have disagreement. Thank you, Mike. Andy.
[Mike D’Arezzo] Sorry, Andy.
[Andy Ellis] See. Jerich needs to be banned because I think every one he’s brought in has been the disagreement.
[David Spark] No. Quite the opposite. Jerich is awesome.
[Andy Ellis] I love Jerich. But he should be banned.
[David Spark] Last time both of us saw him was in Houston. Jerich is great. Sends great “What’s Worse” scenarios. Keep them coming. Especially when people disagree with Andy.
[Andy Ellis] That was a good one.
[Mike D’Arezzo] That was a good one.
[David Spark] It is.
How is the CISO role evolving?
26:34.704
[David Spark] Do CISOs need to get better at improv? Mike Johnson, our frequent cohost and CISO of Rivian suggested that the power of saying, “Yes, and,” can help make a CISO lean into more creative solutions, particularly around adopting new technology. And this is a tease to you, Mike, when you said the department of K-N-O-W.
Now, he frames this as asking yourself, “How do I secure something?” Versus asking, “Can I?” Now, his point isn’t that it will always be feasible to secure things, but allowing for the possibility can have real benefits to your security program and to the business. Whether you’re figuring out how to bring iPhones into your organization before the advent or MDM or rolling out new gen AI tools.
Actually, I’m going to start with you, Mike, on this one. Have you seen this, “Yes, and,” framing come up with novel solutions to security problems? Because I think that’s how you addressed it earlier in the show.
[Mike D’Arezzo] Yeah, so this is kind of the evolution, is how do I secure this. Right? And we get to see a lot of stuff. And you’re like, “Why would you do that?” You’re like, “All right, tell you what. Maybe we’re asking the wrong question.” And so a lot of times we are the people that change the question.
“I don’t think we need to be looking at that. I think we need to look over here.” So, we had somebody in our healthcare business say, “Hey, I have a Bluetooth thing that I want to plug into a patient’s spine.” Now, the immediate thing to this is, “Why would you do that?” [Laughs] But the second thing is, “What benefits do you get out of that device versus any other device?” Let’s look at other solutions here that are in that space that provide that same information.
And you get those people who wanted that spine crazy Bluetooth device to start going, “Yeah. Why did I want Bluetooth? What other things are there? Oh, there’s a whole laundry… Oh, you could buy this solution, and this generates this data.” I think it’s important to kind of see that. Right? And challenge that person who’s looking at those solutions or things like that to look in other different directions.
And that’s what we are. We’re starting to change how we answer and solve questions. I just happen to transparently tie in security into it.
[David Spark] That’s a good philosophical sort of approach I believe. So, Andy, are you a “do I” or a “can I” questioner?
[Andy Ellis] I will take it actually a step further. And rather than saying “do I” or “can I,” you say, “How would I?”
[David Spark] Well, that’s what he’s saying. “How would I?” It is the same thing.
[Andy Ellis] Right. But it’s not, “Yes, and.” It’s, “What if?” You don’t want to commit a yes or a no. Because that already constrains you. The question becomes how would we… Actually it’s even better than how would I. How would we? Because you’re not the one making the decision. At the end of the day, the only decision a security person makes is whether or not they want to escalate and die on that hill.
What we do is we give you advice. They say, “Okay, you want to do X. Okay. Well, how would we do X to meet our safety goals? Let’s have that conversation.” And maybe at the end of the day, we don’t find a way to make it work. Or maybe we do. So, yes, in general, I agree with the approach. I just don’t like the, “Yes, and.” First of all, there is so many people who have used, “Yes, and” as their way to say no that I think it actually is a buzz for people to hear, “Yes, and,” and they’re like, “Oh, God.
That’s not a yes.”
[David Spark] Well, I’m reminded about… This goes back a long time ago. We had a question that came in about you found a USB drive on the ground. How would you safely look at it? And that kind of falls…
[Andy Ellis] The answer is I’d call Tim April. Sorry. If Tim is listening, he’ll love that I shouted him. If you’re a CISO, your answer should be I have a person on my team I hand it to who will look at it. I don’t look at these things. I’ve got a malware reverse engineer somewhere on my team. I give it to them to go figure out what’s on it.
[David Spark] That wasn’t one of the answers, but that’s a good answer, too, regardless.
[Andy Ellis] Sorry. I’m a former Airforce officer, and one of the things that you learn as a cadet before you become a second lieutenant is the answer to almost everything… If you get asked to like go put up a flagpole… This is a classic one. If you get to your new squadron and they say, “Hey, we need you to put up a flagpole,” the correct answer is go find an NCO and say, “NCO, we need to put up a flagpole.
How much money do you need?” And then you go buy the beer for the enlisted folks who are going to put that up. Now, the funniest part about that is the story is the first assignment I had was with a squadron that had more second lieutenants than any other rank. And so they said, “Second lieutenants, your job is to put up the flagpole as a team building activity.
No, you may not delegate it.”
Is this partnership feasible?
31:18.372
[David Spark] Forget speaking the language of the business. Let’s zero in on how CISOs get along better with the CFO. There can be a lot of barriers to a constructive relationship with a CFO. It’s hard to quantify the return of cyber investments and overall risk reduction strategies. And cyber teams can be seen as project impediments when they discover security issues close to a delivery date, as pointed by David Ghee on CSO Online.
To build up that relationship he suggests reverse mentoring to let the CISO offer cyber security insights and leaning into a CFOs preferred solution approach for new tools, likely focusing on the integrated platforms versus best in breed to make the investment story a little bit clearer. So, those sound like good starters, but, Mike, where have you had success in building a productive relationship with your CFO?
[Mike D’Arezzo] So, I think a lot of it has to do with here are my use cases. I’m a very use case driven person. These are the things I’m trying to accomplish. And I attach to that a numerical figure. I think that’s a $50,000 problem. I think that’s a $100,000 problem. And I go out, and I validate whether there’s truth to that.
Like, “Eh, that’s a $600,000 problem.” And then I come back and say, “Here’s the risk. I thought it was 100. Turns out it’s a 600,000. At least with these two or three vendors that are out there today. Do we want to accept the risk? Do we want to move forward? Here’s what it will eliminate. Oh, by the way, here’s the business operational value we can get out of this solution as well.” So, it’s a… That’s a hard conversation.
I don’t think it’s a, “Is this partnership feasible?” I think it has to be feasible. You have to partner with everyone of those senior executives that needs to help you push in a team. It’s good to have arguments. It’s better to have partnerships with all of them. So, I think it’s something we have to do.
[David Spark] All right, Andy, I am throwing this one to you. What’s your relationship that you have had in the past with CFOs?
[Andy Ellis] So, you should have a great relationship with your CFO. And the first thing you should go do if you do not have an MBA or have never been an accountant is go learn accounting. You should be conversant with double entry bookkeeping, with cash versus accrual, with public company finance. You should know the difference between gap and non-gap and why that matters.
If you’re not yet profitable you should learn the difference between EBITDA, FCF, and EPS, and why those are each different metrics, and why that matters. Like you need this stuff. Why? Because it’ll let you understand how the CFO thinks. So, you can have the right conversations. You should be able to run your own budget and execute on it.
Finance will give you a finance partner. I’ll tell you something, they do not know how you spend money better than you do, so you’d better maintain your own set of books. So that when they screw something up, you can be like, “Hey, what did you just do to me?”
Now you’re in a place where you can talk to the CFO and learn about how they think about risk. Contrary to what most people in the security community have been taught, the CFO does not measure everything as ROI. And in fact the biggest way to find yourself on the outs with the CFO is to pretend that you can measure an ROI that isn’t real.
Because they’ll be like, “Wait a second, you promised me money. Where is the money? ROI means I have more money. What’s going?” Maybe you’re talking about risk reduction, but learn their language. And for those of you who think, “Oh, no, I can quantify everything,” go sit down with your CFO and ask them how they quantify 4X risk.
Sorry, that’s foreign exchange risk. And how do they communicate it? Because they don’t walk in and say, “Our 4X risk is 17.5 million dollars.” They say, “No…” Right now they talk qualitatively about like what the sort of ratios are, and where they might go, and how they’re going to hedge. And in what scenarios what those hedges look like.
And you’ll notice that this is the most quantitative thing they could do, and they do not try to reduce it to a single number. So, why are you trying? That conversation will teach you…
[David Spark] That’s interesting. If the CFO ain’t doing it, you shouldn’t.
[Andy Ellis] Right. We have this mindset of we’re going to quantify risk, and we’re going to get to a return on investment. No. No. You don’t need to. You have to figure out what’s going to convince the CFO, and every CFO is different. But learn to talk their language. Demonstrate you understand their risk.
And honestly, go in, and let’s talk about the actual risks that they deal with. Like business email compromise… They’re the first ones hit by it. So…
[Mike D’Arezzo] Absolutely.
[Andy Ellis] Go talk to them about that.
[David Spark] Let me ask both of you. This came up on a previous episode a number of years ago. That we had… Someone wrote in and said, “My mentor…” And he specifically chose a CFO to be his mentor. It sounds like whatever you’re saying…that for a CISO, having a CFO as a mentor is probably a pretty wise idea?
What do you think?
[Andy Ellis] So, I don’t know that I would say a mentor. I think that we fetishize mentorship a little too much in our industry. You should be able to learn from everybody. My first job was in bookkeeping. So, I didn’t need a CFO as a mentor, but I had CFOs as colleagues. I had friends in the finance world.
Because I could go talk to them. I understood their language. They knew I could speak their language. But absolutely. If you can find one to be a mentor. But the problem with the phrase “mentor” is it really does imply something that most people are not willing to sign up to around how much time they’re going to commit to the relationship.
[Mike D’Arezzo] So, I think any senior level executive that has experience would be a good “mentor” to Andy’s point. I think business… It’s nice to understand a CFO because they hold the purse strings sometimes. It really is how do you understand business as a CISO. As a leader, how do you understand business?
Can you convey that to other leaders above you and higher up? And that’s it. And sometimes the language of finance is the easiest way to get to a lot of board members and senior leadership. “Hey, what’s the risk pneumatically on a dollar amount perspective?” “Well, we pay five million a year for 20 million in cyber insurance.” “Well, is that going to cover everything?” “No.
Our liability is probably 30+ million.” You’re like, “Okay, so we’re at least 10 million short on an eventual large scale cyber event.” And you’re like, “Yeah.”
I mean just in that, you already mentioned, like you’re ten million short just in cyber risk. And maybe we need more cyber liability insurance, or maybe we need a second one, or things like that. You’re starting to communicate in their language. They don’t need to know, “Well, the data protection, science, and everything else is…” You’re going to lose them.
[Laughs] Like, “How much is this going to cost us in a theoretical event? And what’s the probability that that theoretical event is going to happen?” Right? I’ve had mentors before who were VP of sales. Obviously my current boss is a mentor. He’s a fantastic human being. But I’ve never had a CFO that was a mentor.
I’ve had them as respected peers but never as a mentor. So, I don’t see true value other than, “Here’s how you can easily communicate to someone like me, Mike, or like us that are likeminded.” But there’s value in that. You can find value in that.
Closing
38:46.720
[David Spark] Well, I hope our audience found value in this very episode. Whether you learned all these accounting terms that Andy rattled off earlier or not. But thank you very much, Mike. Thank you very much, Andy. I want to thank our audience as well and our sponsor. That would be ThreatLocker. Remember, go to their website.
ThreatLocker.com. They will help you with your zero trust journey. And actually when this episode airs, I will have attended their live show called Zero Trust World, which we are doing a live audience recording there as well. Any last thoughts that you have for today’s episode, Mike?
[Mike D’Arezzo] Number one, thanks for having me. This was really cool. It was great hearing what Andy’s thoughts were and conversing with you all. I think we are evolving. And I think we’re evolving in different ways and different things depending on what vertical you’re in and what your background was.
We’re starting to experience… I don’t know Andy’s background, but my background was in IT. And so I approach everything with a solution driven problem. I think we’re kind of… I’m meeting people now who are like, “I’ve always been in cyber security.” I’m like, “Wow. Okay, so you don’t know IT.” [Laughs] I’ve done Sequel databases, application development.
You name it, I’ve done it. And you’re like, “Yeah, I don’t know anything about networking.” You’re like, “They make people like you? How did you get out of here?” So, I think we’re evolving. And as leaders, we need to evolve with it. And hopefully that just makes us better.
[David Spark] All right. Well, thank you very, very much. Thank you, Andy. And thank to our audience. We greatly appreciate your contributions and for listening to the CISO Series Podcast.
[Voiceover] That wraps up another episode. If you haven’t subscribed to the podcast, please do. We have lots more shows on our website, CISOseries.com. Please join us on Fridays for our live shows, Super Cyber Friday, our Virtual Meetup, and Cyber Security Headlines Week In Review. This show thrives on your input.
Go to the participate menu on our site for plenty of ways to get involved, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at david@CISOseries.com. Thank you for listening to the CISO Series Podcast.