How does the business determine what counts as success for a CISO? What warrants a raise in salary?
Check out this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark, the producer of CISO Series, and Edward Contreras, senior evp and CISO, Frost Bank. Joining them is Jason Richards, vp, information security, CHG Healthcare.
Join the conversation on LinkedIn
Huge thanks to our sponsor, ThreatLocker
Full Transcript
Intro
0:00.000
[David Spark] How does the business determine what counts as success for a CISO? I mean, what warrants a raise in salary?
[Voiceover] You’re listening to Defense in Depth.
[David Spark] Welcome to Defense in Depth. My name is David Spark, I’m the producer of the CISO Series, and joining me as my co-host, one of your favorites, one of mine. It is Eddie Contreras, Senior EVP and CISO over at Frost Bank. Eddie, thank you so much for joining us.
[Edward Contreras] Thanks for having me back, David, yet again.
[David Spark] And it will keep happening. Our sponsor for today’s episode, a fantastic sponsor of the CISO Series. I’m sure you’ve heard them before. It’s ThreatLocker. Allow what you need, block everything else by default, including ransomware and rogue code.
More about just that a little bit later in the show.
But first, let’s get to our topic at hand. So, this actually comes from a Reddit discussion. We love our Reddit discussions. The problem with Reddit discussions is often we don’t know who said what, so it’s hard to quote people by name, but here is the question.
It’s a post by Ross Young, who is the host of CISO Tradecraft and a CISO himself, and asks how is a CISO actually measured when bonus time comes around? And he wanted to know what were the metrics the business should use to measure performance. So, Eddie, I would assume this is something you discuss with your employer during the hiring process, and it’s something near and dear to your heart, and the CISOs listening as well, and security leaders listening.
Do you have metrics that you’re set up to essentially show your performance?
[Edward Contreras] Oh, yeah, absolutely. And David, this is the dreaded money conversation. And I don’t think it just sticks with the role of the CISO. You can replay this for every aspect of your life. Whether you’re in a relationship, you own a business, you are somebody’s subordinate, you have to talk about money.
And for some reason, human nature, it’s just not comfortable, but you do have to have that conversation. And so, the best thing to do is to walk into that office and say, “Here’s what I think I should be measured like,” and that’s the uncomfortable part.
But once you start it, most of the time you’re going to come to an agreement.
[David Spark] Very, very good point. Well, we’re going to have that discussion, and I think it’s quite valuable for everybody whether you’re a CISO or not, because it does address how your performance should be measured and a discussion you should have.
To join us for this discussion, I’m thrilled that he’s joining us. It’s the VP of Information Security for CHG Healthcare. It is Jason Richards. Jason, thank you so much for joining us.
[Jason Richards] It’s great to be here, David.
What would a successful engagement look like?
2:31.903
[David Spark] Now, before I start reading these quotes, I want everyone to know that this comes from Reddit, so I’m actually not going to mention who said what because the usernames are kind of crazy and they do not actually identify the person. So, if you really want to know who said what, we’ll have a link to the post for Reddit on the blog post for this episode.
I’m just going to read quotes without identifying the people. So, first Reddit user said, “I’ve always made the personal effort with staff. If you are friendly and approachable, you go the furthest. Sitting back and complaining about others being ‘idiots’ or feeling overlooked rarely changes outcomes and realistically applies damage to your reputation.
If you’re approachable and positive, others are more likely to advocate for you and include you in important projects.”
“And the honest answer,” another Redditor said, “Public speaking experience. The quickest way to a CISO is to join the public speaking circuit.” So, we are talking about others. You were talking about subordinates here. This is this first section here is talking about that, and it speaks about just being kind and being public, and actually, two things you do quite well, Eddie.
[Edward Contreras] I always tell this to my team, not just my managers or my leaders, never say in private where you’re not willing to say in public. We work in a profession as security professionals that you should assume at some point in time, your emails and your communications will be made public.
If you’re going through a breach, if you’re going through a disclosure, if you’re going through a subpoena, the first thing they’re going to ask is, let me see this correspondence from this team, and really, anybody in that subpoena can be looked at. And so, you should assume that conversation is going to be made public.
So, you always want to act that way. I think it’s great advice to understand that if you can manage your candor, if you can be professional, if you can really just interact with people the way you want to be interacted with, you really have nothing to worry about.
But it’s great advice because some people in our industry, they have very strong opinions and like to share them.
[David Spark] People do just enjoy talking smack, saying negative stuff about others. And in the moment, it can sound fun and they can sit on your side, but man, time goes by and people will remember what you said. Yes, Jason?
[Jason Richards] Yes, that’s true. They’ll remember what you said. And I think an important part of this is as we are being kind that we first seek to understand. So many times we think we’re communicating, and we’re just scratching the surface. But as we’re curious, as we try to understand their point of view, their position, where they’re coming from, we better then are able to communicate our position and why we may have a certain stance and why we may be saying things from a certain point of view.
And we may not see eye to eye every time, but if someone feels heard, if they feel understood, that builds bridges and helps to bring that divide together to where there may be misunderstanding.
[Edward Contreras] Yeah. And if you think always assume innocence, no one comes to work to be mean. They may not be doing what you want them to do, but it may be simply innocence, right? You always have to assume that they have good intentions. And if you think that way, you’re going to approach those conversations differently.
But the fact that we call most of our customers “users,” that’s not always said with the same tone, right? [Laughter] So, you just be cautious.
[David Spark] [Laughter] Yes, I know that.
[Jason Richards] To also add to that, Eddie, is that I think a key attribute of any leader, including a CISO, is humility. We might be wrong in our position. And we need to understand, again, why they have the position and why they might be doing a certain behavior so that we can get to the right outcome and be less wrong together as a collective.
[David Spark] By the way, I’m going to stress this. I like being wrong. You know what I mean? In a sense of when I’m wrong, that means my team is doing more than I can do. If I’m always right, then why the heck do I need the team kind of a thing? [Laughter] I like it when they point out what I’m not getting.
Eddie?
[Edward Contreras] I completely agree. Always hire people smarter than you.
What should we be measuring?
6:52.618
[David Spark] “As an industry, we don’t have widely agreed-upon metrics to measure security effectiveness. So, in reality, this is hugely dependent on the CISO storytelling ability.” Another Redditor said, “Metrics are used like words to tell a story. I think different questions may get to what you’re after.
What value is security bringing to the organization? How is the value brought forth? How are we going to elevate the program next year? And how much of this is due to the CISO and/or their team’s influence? Finding a way to quantify this will give you a bargaining trip when it comes time to talk money.” I think this last quote is really nailing it and saying, look, it’s for you to figure out how these metrics are going to measure your performance the best, how you’re going to be seen and valued within the organization.
And this may be different to everyone. I will just tell you, like in general, we hear the term mean time to remediate and all the variations of that many times on the show as one of the best metrics. What are your thoughts?
[Jason Richards] Yeah, mean time to remediate is a wonderful metric to look at business resilience, and I maybe want to pull this back to what Eddie mentioned starting this off on this conversation about sometimes we have to come in and talk about the outcome we want in anything in life, and sometimes that money conversation is thinking with the end in mind.
But really bringing this back to a first principle that clarity creates metrics, not the other way around, and I don’t think there’ll ever be a one-size-fits-all measurement for a CISO. Just like with any industry, there’s not a one-size-fits-all for a CTO, for a CIO.
It begins with clarity, coming in and understanding what is the goal of the goal of this business? What market are we in? What are we trying to accomplish?
And they may have an idea that they think would be successful before you come into that role, and they may not. If they don’t, that’s not a failure. That’s an opportunity for leadership, to take ownership, and help define what success looks like. I’ve worked in a SaaS organization where mean time to recover, David, to your point, was a key metric for our customers.
I’ve worked in healthcare where confidentiality of patient records is a key position to protect patients, the clients that we serve, and the organization. And so, it’s really getting to the what are the first principles of the organization. Now there are some agnostic tools to help CISOs.
You have your CIS controls, you have NIST frameworks, but you still need to take those and position them to what’s important to the organization. And at the end of the day, metrics, they aren’t the goal. They’re just a tool to help reduce the probability of material impact and bring it to a language that everybody can align on.
[David Spark] That is a good point. Metrics are not the goal. They’re essentially guides as to what’s going on. Eddie, your thoughts on you having to tell others how you should be measured, and I also thought this is dangerous ground. There is literally books written about lying with statistics.
[Laughter] So, what’s your thoughts on this?
[Edward Contreras] It’s kind of like what I’ve said before. If I was to tell you, “Hey, you’re doing great.” What does great mean? Is it a bonus? Is it my monthly pay? It just means I get to keep my job? What does great mean? So, you do have to define that upfront.
And I like the fact that metrics can be both operational and performance based. And that’s going back to the initial question you asked me, David, when we started this conversation. What are the things you agree upon upfront that maybe can influence your bonus or your salary or your pay?
If you can talk about 10% in reduction of cost center expenses, 20% automation of manual workflows, 5% maintain like a 5% attrition rate. Those are metrics. Now those are performance metrics. They’re not operational metrics. Now, operational metrics, exactly what Jason was talking about.
Find something that’s consistent, that’s well-defined, and say, “Here’s my target,” and if you hit that target, okay, now do you have anything else to talk about that says maybe I do deserve more money?
[David Spark] When you’re setting up these metrics, and a quick question for both of you, is are the people in the business so confused about cyber, they’re like, “Well, whatever you say, we’ll go with it,” kind of a thing? [Laughter]
[Edward Contreras] Absolutely not. I’d love to hear Jason’s take on this as well. What I like to do is in the three examples that I gave as far as performance metrics, they have to be agreed upon. When you’re starting the beginning of the year and saying, “Here’s how I want to be evaluated.” If you and your boss come to an agreement and HR says, “Yes, you may submit that,” there’s an agreement in place there.
So, those are metrics that you can essentially measure against. But make sure you’re using smart metrics. Same thing like smart goals. Are they obtainable? Don’t just throw out a number that you’ve never seen achieved anywhere. Make sure they’re realistic.
Jason, I’m curious your thoughts on that.
[Jason Richards] Yeah. And they have to be realistic and point to something. The goal of the goal, how is this helping your organization to win in the market space that you’re in? I kind of think of it in a few different levels. You have your individual contributors, looking, day-to-day signal, what they’re pulling out of it.
You have your trends at your VP or director level you’re looking at. Then at the board and the C-suite, you have a different way to represent that. I know there’s qualitative, there’s quantitative. I’ve seen a CISO who’s had much success using some actuary tables, which some insurers use to underwrite insurance policies.
And they’re using that with the board and the C-suite to show we have this much loss expectancy, and it’s a language which they understand, and it correlates to the maturity of their controls and how they can buy down that risk.
So, there’s different audiences, there’s different layers, but to your point, Eddie, aligning with your direct leader at that time of what are the things that you are going to try to accomplish. And there’s a saying which I like, an organization does well that which the boss measures.
So, if you’re aligned and they’re measuring that, and it now becomes a shared goal between you and your peers. So, take mean time to remediation. If you are a peer to someone in engineering, and you’re trying to keep your systems up, how do you both tie into that goal?
Is it your systems are all configured in a certain way, you’re patching in a certain time, you’re able to bring services back up? Now you’re both shared and linked to that.
And I think that is an outcome that we should also look at is how successful are we making our peers in the organization to reach a shared outcome. Now, I do know that metrics can be gamed, and that is a worry that if you shoot for the sun, you may hit the stars, and you may do something great, you may have a stretch goal.
But there are some organizations that will penalize you if you do not make that stretch goal. So, it has to be a well-trusted relationship with your leader as you create those goals.
Sponsor – ThreatLocker
13:46.854
[David Spark] Before I go on any further, let me tell you about our spectacular sponsor, and that would be ThreatLocker, great supporter of the CISO Series. And here’s something you probably all know, CISOs don’t lose sleep over the malware they see. They can deal with that, right?
They lose sleep over the things they trusted that they really shouldn’t have. Because that’s how modern breaches happen. Not through zero days, through everyday tools, doing things no one realized they could do. See, the attackers, they’re taking advantage of your goodwill and the tools you think you trust.
And this is exactly the problem ThreatLocker eliminates. ThreatLocker enforces default deny at the point of execution. If it’s not approved, it doesn’t run, period. So, your attack surface essentially collapses from everything on the endpoint to only what you say is allowed.
And the real power? ThreatLocker controls how trusted tools behave. PowerShell can’t start scraping credentials, Chrome can’t start launching scripts, your remote monitoring and management, RMM, can’t suddenly turn into an attacker’s remote access platform.
CISOs say the same thing, “This is the first time I felt actual control instead of alert fatigue.”
Now, if you want to shut down entire categories of attacks, not react to them, ThreatLocker built a resource hub just for security leaders. And you’re going to love this. It’s going to be easy to remember. Go to ThreatLocker.com/CISO. Again, ThreatLocker.com/CISO.
Just add that /CISO, it’s going to take you to the hub, and it will also let them know you found out about them through the CISO Series. So, if you want fewer surprises, start there.
What needs to be considered?
15:29.471
[David Spark] All right. First quote from a Redditor, “We actually have 10 to 15 different measurable metrics that get baked into target agreements. Things like scores and maturity assessments, no A-findings in certain audits, no incidents above a certain damage threshold.
It’s actually really transparent over here. Everybody knows the CIO’s goals. He breaks them down into the contributions required by his direct reports, and they break it down further. So, usually you don’t get completely pointless goals because you can trace them back all the way to the high-level business objectives.
You may, however, be asked to ‘contribute’ to things beyond your control or expertise. That’s something to iron out with your manager during performance reviews.”
And here’s another quote. “Bonuses are always baked in when they’re hired, and it’s generally based on company performance while filling out your own performance evaluations and fighting it out with your manager to justify the ‘stretch’ or the ‘outperform’ label.” And the last quote here, “The measurement for bonuses is the same in all my cases, how the business is doing.
It follows naturally that if you are doing worse on a technical metric like mean time to remediate or mean time to detect or failing certification audits, the business won’t be doing as well as a result, but these metrics alone have never materially contributed to my bonus.
Also, let’s be frank. A lot of bonus is relationship and vibe driven within your comp plan and subject to approval by the board.”
So, by the way, they throw in a really good point here is the business is doing well, we all do well. The business is not doing well, it’s not doing well. Let me throw this thing out, and one of the things we haven’t talked about here, and these are metrics, but this may be one of the metrics is performance during an incident.
And this, I guess, is under MTTR and MTTD. You can’t say in any kind of measurement that we’ll never get breached, we’ll never lose this, this will never happen, kind of a thing. Like we will always block and tackle and nothing’ll get through. But one of the things we talk about a lot on our shows is just when it hits the fan, how you perform, that’s like the true measure of a cybersecurity professional.
Eddie, is that something that’s put forward and how are you and other security professionals measured on that?
[Edward Contreras] Yeah, working under duress, that’s a challenge and it’s a skill. And it takes a really good leader to be able to calm the nerves of the room because that can be the very excitable moment. But I like all these quotes, and I’ll start first with this first one, right, around 10 to 15 metrics.
If you have to choose between 10 to 15 metrics to show how important your program is, that can be subjective. For me, it’s really understanding the operating zone, understanding and setting a risk appetite statement. If you don’t know what your comfort zone of your board or your executive team is, then is 10,000 vulnerabilities bad or is it just Patch Tuesday?
You don’t want to send up the alarm just because it’s Patch Tuesday and all of a sudden, they see that number go from 10,000 to 20,000. Well, that’s just normal. That’s monthly events.
So, my thought here is and my guidance to a lot of the listeners is have that conversation with your leadership and say, “What are you comfortable operating in?” And once they say, “You know what? If you have like a five-level measurement and we’d like level three.” Okay, perfect.
Now that I know you’re level three, anytime we get close to level three, I’m going to give you a warning. When I’m in level three, I’m going to give you a plan. And if I exceed level three, you know what? Something’s gone awry. And I think if you understand that operating philosophy, you’re probably going to bring better metrics to that conversation versus just a 10,000 vulnerability number.
[David Spark] That’s a good point. Sort of like attach them to a level rather than having someone else try to figure out how do these six metrics combine to be anything. I like that sort of idea because sometimes, yeah, you have to combine things together.
Jason, what are your thoughts on this in terms of creating levels, metrics, how someone performs under duress, any of the above?
[Jason Richards] So, I’m aligned to what Eddie had laid out there, that our job is a crisis management job, and in order to do that, you need to instill trust. You need to have a level head. And you can also look at what have your decisions been over the last year or year going forward.
That is something that builds trust and relationships with your senior leadership. Going back to those metrics and measurements, I think clarity can be brought in. I think this will come to what Eddie was talking about with some type of level or threshold.
But before you even get to that metric, there’s a conversation to talk about what are we even trying to do? What is the goal of the goal of this metric? And once you have that alignment to say, “We are going to patch within 30 days,” then that is something that you’ve aligned with your leadership because you all feel that that will reduce risk to your business.
Therefore, that is now a metric and a standard that everybody starts to jointly enforce and work towards.
If you don’t have that clarity before you create the metric, we can always suggest them and bring them to the business and get their buy-in on them. But without the clarity first, then we’re just measuring for measuring’s sake, and we’re not sure if it’s really moving the needle in a meaningful way to help the business win in the environment that we’re at.
So, I think clarity, again, precedes metrics.
[David Spark] That’s a really good point of the fact that measuring for the sake of measuring, which you can’t sort of deny the when numbers look better, you want to sort of shout it from the hills, right, Eddie? [Laughter]
[Edward Contreras] Yeah, absolutely. Think about technology lifecycle management. No one wants to have dated software, and when you’re in the green, that’s a badge of honor. It’s like, oh, my goodness, we’ve made it. We’re all in the green, knowing that next month there’s another piece of software that’s going to go end of life.
And so, you do want to shout the winds. You want to praise some of those things. But you also don’t want to dwell on the fact that it’s 1 of, again, 15 metrics in your program. And so, it’s hard to focus on a metric and drive and influence change if they don’t understand the impact of the program in its entirety.
You have to really understand all of it as an aggregate, not just a metric in isolation.
Sometimes it’s really not that difficult.
21:46.107
[David Spark] “Officially, CISOs are good at writing goals and performance targets and giving their boss credit for it. Unofficially, one really visible save for management can count for a lot. If a C-level gets phished, and they have caring and discreet attention available promptly, they’ve got the CISO’s back.
If management sees security working around the clock after a serious zero day is announced, that gets some love.” So, there is the day-to-day work you do, right, Jason? And then there’s, like we talked about, these extreme moments. And when you see someone sort of rise to the top in these single extreme moments, that kind of outweighs everything else and when it comes time for that salary bump, you’re like, “Well, you remember that one time we had this and it looked like it was going to be awful, but I saved everybody’s butt?” that kind of a thing.
Like, doesn’t that sort of just sort of move the needle?
[Jason Richards] I think it does move the needle in a way, but I think these aggregate over time. There may be one of those scenarios that raises everyone’s awareness to who you are, who your team is, and what you’re capable of. But as you show up and you continue to do that in a repeated pattern and behavior, that shows leadership.
And I think what this is really talking to is not so much particular or unique to a CISO position, but it’s a leadership position. You were there to lead, to provide direction, to provide guidance, and to create visibility of the work and the output that your team is doing.
So, while yes, I do think there are those scenarios to where, hey, you may have saved the day, and they do remember that, that is now an opportunity to continue that trajectory, that behavior, and keep showing up as a leader to help shine a light on what your team does and how it helps to move the needle for the business to win in your market space.
[David Spark] What you’re saying, it’s a personal brand-building opportunity. Those are the moments you truly build your brand. Yes, Eddie?
[Edward Contreras] Yeah. Unfortunately, it’s when they’re live that they count the most, but you don’t have to wait for an event to actually practice this. This is great crisis management tabletop exercise. If you can engage your leadership team, walk them through scenarios that are planned, you can start to build that credibility.
And you can get the interaction. It can be bi-directional. It can be engaging. It doesn’t actually have to be based off of an active crisis. And so, don’t wait for the bad time to occur to be able to flex that muscle. This is something that you want to plan for.
And you can build that credibility over time across your peers, and then also with the executive group. So, many opportunities there in crisis management. It’s just a phenomenal way to show people. You hired me for a reason. You hired our team for a reason.
Let’s demonstrate what that reason is.
[David Spark] Excellent point, both of you.
Closing
24:38.219
[David Spark] Well, that brings us to this moment in the show where I’m going to ask you both which quote of all of these, lots of really good quotes, was your favorite and why? And Jason, I will start with you.
[Jason Richards] So, David, my favorite quote was found in the Reddit thread that this topic was taken from by BitSlammer was the handle, love that name, and he said, “Pick 100 companies and get 100 different answers.” The reason why that resonated with me because the topic we’re talking about with the metrics and a way to measure a security program or a CISO is really context to the business that is you’re representing at that time.
And so, that is something for us to always remember, that it is market by market, business by business in which we measure things which matter.
[David Spark] That’s a really good point. And also culture by culture, too. I would also throw out there, too, as well. And I’m assuming you’ve seen a change, although you could have had something happen in a previous job. You’re like, “I’d really like to be measured the way I was measured at this previous job here as well.” All right, Eddie, your favorite quote and why?
[Edward Contreras] I love Jason’s quote, I wish I would have dug that deep. When I look at these quotes, and we should do these Reddit ones more often, this is great. I like this quote. “I’ve always made the personal effort with staff. If you are friendly and approachable, you go the farthest,” and there’s more to the quote, but for me, it really is about just treating people with some respect and some dignity.
In return, you’ll probably receive the same. So, if you want to go far in your career, if you want to be remembered, for good or bad, you have to understand that your words matter and they have a lasting impression.
[David Spark] Yeah, I would just say, like we have a history of carrot-and-stick motivation. I just don’t see how the stick can work at all.
[Edward Contreras] It rarely does. It rarely does. The stick is more of a bravado event. Somebody is winning a conversation, but you’re not going to win the war, right? I mean, this is something that is long term here. And so, if you’re trying to prove a point at what cost, be careful with those times.
[Jason Richards] And to add on to that, I’ve been in, and you’ve probably worked in some of these organizations as well, where their perception of the CISO is come in and do the Gandalf, “None shall pass,” and that is not always the best approach. There are times where you have to come in and give a stronger opinion on a path and maybe enforce a path, but really, the collaborative part of that will get the company further and you’ll build more partnership and help get the company to a spot where they win.
[David Spark] Excellent point. I love that. Well, that brings us to the very tail end of the show. A huge thanks to Jason, of which I’m going to let you have the very last word. But first, huge thanks to our sponsor, and that would be ThreatLocker. You know them as the leading zero trust platform.
Allow what you need. Block everything else by default, including ransomware and rogue code. How do they do it? Well, we’ll talk about it and how amazingly simple it works. Pretty impressive. We’ll talk about that a little bit later in the show. Jason, thank you so much for joining us.
I want you to have the very last word, and one question I’d like to ask our guests is are you hiring over at CHG Healthcare?
[Jason Richards] We are, David. We’re hiring across our product development and product security teams.
[David Spark] Awesome. So, I’m assuming they can reach out to you? Mention that they heard you on the CISO Series as well. And we’ll have a link to your profile on LinkedIn from the blog post for this episode. Well, we loved having you on. Thank you so much, Jason.
And thank you, Eddie, as always, for doing such a great job. And to our audience, I say it… By the way, thanks are not pat. I do appreciate you. I find I’m like apologizing for my thank you’s too much here. But I mean this, this show does not exist without your contribution.
So, thank you for your contributions and for listening to Defense in Depth.
[Voiceover] We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site CISOseries.com where you’ll also see plenty of ways to participate, including recording a question or a comment for the show.
If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com. Thank you for listening to Defense in Depth.