We know third-party risk management is a pain. If nobody likes the universally agreed upon solutions like questionnaires, what are we doing that’s improving the situation?
Check out this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Steve Zalewski. Joining us is our sponsored guest, Nick Muy, CISO, Scrut Automation.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, Scrut Automation
Full Transcript
Intro
[David Spark] We know that third-party risk management is a pain, but what are we doing that actually is improving the situation?
[Voiceover] You’re listening to Defense In Depth.
[David Spark] Welcome to Defense In Depth. My name is David Spark. I am the producer of the CISO series, and joining me as my co-host for this very episode—you’ve heard him before, and guess what? You can’t run away. You’re going to hear him again. It’s none other than Steve Zalewski. Steve, say hello to the audience.
[Steve Zalewski] Hello, audience.
[David Spark] You’re going to hear a lot more of him. Maybe he’ll do it in that sing-song crash. Just wait and see. Our sponsor for today’s episode is Scrut Automation. Stay aware, stay ahead, and stay compliant. More about Scrut later in the show. In fact, they’re responsible for our guest today. Very excited about that.
But first, Steve, I just want to say today’s topic got an explosive response. You can check the news every day and see attacks that can be attributed to third parties. If managing that risk was easy, we wouldn’t see this constant stream of attacks. And we currently have a lot of “solutions,” such as questionnaires that fall into the realm of security theater.
Nobody likes them. I have yet to meet a person who does, and nobody believes they actually work. While we can’t hope for a silver bullet, you asked the question of the LinkedIn community: What is anyone doing to make third-party risk management reasonably better? And you got a ton of responses, didn’t you?
[Steve Zalewski] We almost got more comments than we did likes, and that never happens. A lot of opinions, and most of the responses were not “me too.” They were people thinking about this problem and thinking out of the box, right? Because third-party risk management historically versus where we’re going—I was really happy to see how much thoughtfulness went into the responses.
[David Spark] Yes, because the majority of third-party risk management is a lot of complaining, telling us what does not work. And there was a great positive attitude. Some people in their posts had multiple tips in just one comment. It was quite explosive.
To help us in this very discussion, we’re going to be talking about that and going through your great advice and complimenting it with your advice, Steve, and our guest’s advice. That guest is none other than our sponsored guest from Scrut Automation, the CISO Nick Muy. Nick, thank you so much for joining us.
[Nick Muy] Great to be here.
What should we be measuring?
[David Spark] Mike Wilkes of The Security Agency LLC said, “Continuous monitoring with different profiles for different vendors. Tier 1 means if they go down, you go down or are significantly degraded. But not everyone agrees what continuous means.”
And Viresh Garg of Broadcom actually provided a list of eight different ways to improve third-party relationships. I’m just going to call out two here. Number one, Viresh said, “Active reconnaissance and penetration testing—conduct some active reconnaissance, and based on the vendor’s criticality and relationship, perform penetration testing to identify potential vulnerabilities.”
And also, the rest suggested the Cloud Security Alliance self-assessment: “Require vendors to complete the CSA STAR self-assessment to evaluate their cloud security practices and compliance with industry standards.” So, just some pretty good straight-up advice here. Steve, what do you think?
[Steve Zalewski] So, what should we be measuring? What can we be measuring? And what I really liked is you just saw three different perspectives of what third-party risk management has become versus what it was. Which is if you look at it holistically, it’s: Am I trying to measure third-party risk relative to my tier ones?
Am I trying to answer the question of what are my highest risks for integration? Or is it how do I provide evidence? And so what I like about this is that’s three different ways, all of which are part of the solution.
[David Spark] What’s your take on this, Nick? And again, this seems like some basic good advice. It’s not the silver bullet, not the full answer, but definitely a positive direction.
[Nick Muy] 100 percent agree. I think the great point that Mike makes is, when answering the question of what should we be measuring, he’s really reminding us. It’s who are we measuring? Are we measuring every vendor, or are we measuring the tier one vendors? And Viresh gets it too in his comment about criticality matters, and I think that’s super important.
We can’t forget that.
[David Spark] Yeah, it’s very interesting. Mike calls out the—because we’re recording this soon, not too long after the big CrowdStrike debacle and the question of what do we do when one vendor takes us down? And who would have thought this was the way we were all going to get taken down? It seems so bizarre.
So, I think going back to you, Nick, on this is how do you look at where you can be taken down and how you build resilience on that?
[Nick Muy] Yeah, no, totally. CrowdStrike was definitely a wake-up call for us. We always look at where is this vendor in our product and how customer-facing is it? Is this something our customers are going to directly interact with as part of our product that we’ve integrated into the product? Steve was talking about integration points; for us as a SaaS vendor, that’s super important, right?
We’re always going to be integrating different vendors’ technology and different software packages into our products. And we have to be really serious and proactive in looking at, “Hey, if this breaks down, will the whole app break down? Will our customers be able to use the service that we’re providing them?”
[David Spark] And Steve, that seems like a full-time job in itself. How does it not become a full-time job in itself?
[Steve Zalewski] I’m going to say it’s going to get worse before it gets better. And if we go back to the start, which was “Stay aware, stay ahead, and stay compliant,” and I really like that catchphrase because “stay aware”—we realize there’s four or five different perspectives here of how we have to stay aware of what we’re trying to do.
“Stay ahead” is how am I showing everybody else that you can trust me versus how can I verify that I can trust everybody else? And “stay compliant” is a whole lot of paperwork that we’re having to generate for either case. And that’s why I said it’s going to get worse before it gets better because our ability to do all three of those things, relative to something we can’t see but we have to trust, we’re 20 percent of the journey there.
What do most people think it is and what’s the reality?
[David Spark] Jonathan Waldrop, who’s the CISO over at The Weather Company, said, “The longer I’m in this field, the more I realize that third-party risk management is everywhere except what you control. And the only thing you control is you. The key here is in how you build and frame your response.
You absolutely must plan on the system/service thing being unavailable. How does your team or company carry on without it? We as an industry have long said it’s not ‘if’ but ‘when.’ Build your systems and processes with resiliency in mind and control what you control.”
So, it’s interesting he’s talking about like you should be doing this, you know, at day one when you’re bringing on—not looking at this, you know, posthumously or just after you’ve installed everything.
And Andrew Townley of Archistry Incorporated said, “If you’re not prepared for the impact of disclosure, then don’t share it with a third party. Even with a trickle-down or viral clauses in the contract, it’s really in-party risk management from the get-go. Most of the reason this is a problem is because nobody really understands the potential impact of the exposure, I would agree with that, and therefore it doesn’t factor into the original sharing decision in the first place.”
Who knows if you can create legal anything to control what we really don’t understand here? But I really like this idea, Jonathan. What Jonathan said—and I’m going to toss this to you first, Nick—is just build this in the moment you onboard anything. And have you done that before?
[Nick Muy] I can say as a small company that’s easier for us to do. In previous roles at a big company, that’s a luxury you don’t always have—little harder. I think, as Steve’s saying, it’s going to get worse before it gets better. Building that in requires a lot of buy-in from a lot of stakeholders—your procurement team, the business team that says, “I need this vendor now.” And security is saying, “We have to go look at this and build it in and see how we’re going to integrate it.
What are the controls that are in place?” And all people hear is, “Hey, you’re going to slow me down a bit there.” That’s what it sounds like. I think if it’s truly critical—and you know what Jonathan’s also saying there in his comment is—you really have to understand how your business works. If you don’t understand how your business works, and you’re just—it’s really hard for you to know what kind of risk you are even trying to manage from that third party.
Are you worried about—yeah.
[David Spark] Let me ask you, is there like some kind of an equivalent of a Chaos Monkey that you can do to yourself? Chaos Monkey is essentially this tool that Netflix created to essentially cause problems so that they could build a stronger environment. Is there something like, “Hey, let’s test this out.
Let’s turn this off and see what happens
. Let’s turn this off and see what happens,” but easier said than done because you could cause serious chaos if you truly do these things. So, what do you do to see what your environment can and can’t withstand? And then, when you find that it can’t withstand it, how do you deal with it? Like, what are the ways you look at this if you can’t do a true Chaos Monkey situation?
[Nick Muy] Yeah, no, I mean, if only it were that easy. We could go turn something on that just randomly turns it off in a safe way. I think when we’ve done this before, we’ll go find that we have three different vendors doing the same thing as part of our vendor risk assessments, and we’ll go talk to the different business owners.
It’s always, of course, three different people who bought three of the same thing and say they each need—how special each and every one of them is. And as part of our kind of quarterly reviews, we may say, “Hey, look, we’re going to—I.T. team’s going to turn off access to this one, and the other two are still available.
If we turn it off, what will happen?” We’ll see what are the impacts. We’ll give people some notice, and then we actually go see what happens and how to do this. Naturally, the CFO is very supportive of that, as they don’t like paying for three of the same thing. But also, from a security perspective, we’re really—we’re actually trying to test, like, how is this going to impact us, our operations, our customers?
Of course, I’m not exactly going out there and turning off something me, myself, or the team knows is definitely going to cause an outage. But we will go turn something off that says, “Hey, maybe this is an internal tool. Is this going to break things? Let’s try to do this in a controlled manner.” We used to just do this for, say, engineering tools, but there’s so much technology all our teams are using, not just engineering.
[David Spark] Good point. So, Steve, what about you? What is a good way to stress test an environment or maybe not just stress—just understand, even without you doing—where you just pull out paper and pen and go, “This is a problem. This is a problem.” Just knowing where they are. How do you do that?
[Steve Zalewski] Don’t underestimate the value of a tabletop exercise. And here’s how you can introduce the Chaos Monkey with the tabletop exercise. And I actually was part of this, and it still resonates in my head. Instead of you being with the executive team and putting a scenario on the table and having them work it through, okay, sit in the control room.
Have the executive team in one room and have the I.T. team in the other room, so the I.T. team has to work the actual incident. And tell them that there’s been a compromise, you’ve taken a business system offline, and let them go figure out how to actually do that, okay? And as they’re scrambling to figure out to do something they don’t understand, report that back up to the executive team.
And watch how quickly chaos results in a meltdown on both sides of the house. Okay, that’s the true way to implement a Chaos Monkey. And then, what the message is, historically, right? We’ve tried to control everything inside of our network edge with our data centers, and more and more, I control less, and the things that I don’t control—all those SaaS applications—have all kinds of implications, right?
That’s what we talked about. So, these tabletop exercises, where you put both in the room, both in different rooms, and work it through to get them to understand that I, as a CISO, used to control 90 percent of my environment. And for many of these companies, it’s down to under 10 percent of my environment that I actually control, and the 90 percent…
Sponsor – Scrut Automation
[David Spark] Before I go any further, I do want to tell you about our great sponsor. It’s Nick’s company. That’s Scrut Automation. They’re a leading GRC platform that helps you—and I said it before—stay aware, stay ahead, and stay compliant. Scrut Automation liberates growing enterprises from the morass of compliance debt to proactively manage their strategic risk, enabling organizations to build sustainable GRC strategies that effectively govern and monitor their security programs.
With Scrut’s super flexible GRC platform, security and risk professionals can gain visibility into the risk posture, monitor controls in real-time, and showcase proof of compliance with industry frameworks without stretching the security budget and in alignment with the organization’s business goals.
You want to find out more? Then you gotta go to their website. And it’s easy to find—it’s scrut.io. Let me spell that for you: It’s S-C-R-U-T.io. Go check it out.
What’s the optimal approach?
[David Spark] Marie-Eve Girard of M.E Cie said, “Successful third-party risk management must also consider people, processes, and technology, as only evaluating the technical part could lead to higher risk due to blind spots. The controls should encompass the entire life cycle, from onboarding to offboarding new third parties.
A comprehensive analysis and mitigation plan should include a recovery process for each vendor, as the threats they represent and the mitigation measures required differ.”
And Tony Gonzalez of Innervision Services said, “I have always had strong beliefs that in order to have an effective TPRM process (third-party risk management), it needed to be a lifecycle approach and not just looking at assessment. The process needs to incorporate third-party commitment through contracts that clearly state what is expected from the third party, annual review processes that require evidence of continuous improvement and state of controls, and ownership and engagement of the third-party relationship to govern and manage the process.
Automation plays a big part moving forward to streamlining processes.” I like this whole concept of life cycle, Steve, of—you gotta understand what it’s gonna mean to bring it on, to keep it going, to lose it, and to get rid of it. How healthy do people understand that? And also understanding how the third party is going to act as a partner with you.
[Steve Zalewski] So, one of the themes that we saw in the conversations on LinkedIn, right, about this optimal approach was an appreciation that if you look at governance, risk, and compliance (GRC), which is what we’re talking about here, much of third-party risk management has been focused on compliance.
And how do I generate evidence that allows me to say I did the best I could? But we all don’t believe what we checked because it doesn’t really do much for us, right? It’s an outside-in view. And what I’m proposing here is the optimal approach is that we actually have to think about risk posture management, not compliance.
So, looking at governance for risk, because that is where we’re being pushed now, is to understand: What is my risk profile for what I can and can’t protect versus where’s my evidence of compliance? And the consequence of compliance is a fine. The consequence of risk is going to jail. And so, why I say, “What is the optimal approach?” The optimal approach is we’ve got to look at the risk component now more than the compliance component.
And that is setting us even farther back.
[David Spark] And I would also throw out—I’m going to throw this one to you, Nick—is that, Steve, you make a great point, but the reason we look at compliance is there are actually defined numbers there, as you can see, when it’s a yes-no situation in some cases. I know that compliance is a sort of a drifting situation.
Technically, there is a bar to achieve risk—not so much, and not so clear. So, that’s why it becomes a really muddy… Yes, Nick?
[Nick Muy] I think people like easy answers. Compliance is an easy answer. I think the risk one is hard. From everything I’ve seen firsthand, the risk part requires you as a company to understand your own risk posture. Most companies—it’s actually easier to focus on the vendors than it is to focus on yourself.
I don’t know how many times, both sending questionnaires—having been on both sides of sending out questionnaires and doing risk assessments of vendors—and on the receiving end of questionnaires from customers where they’re asking questions. We’re filling it out, we’re doing our best effort to get them all the information they want, and then it turns out in the discussion, we ask them, I say, “So, what are you really worried about?
What risk do you want to make sure that we as your vendor are helping you manage?” And then that’s exactly the response I get—crickets.
[David Spark] A really good question. I want to double down on that. That is the best question of this episode. And that should be the whole theme of this. Should have been in the title of this episode: What is the damn question you’re—what is the thing you’re worried about? “I’m worried about you’re going to blow up my business.” And that’s pretty much it.
Yes.
[Nick Muy] Yeah, I think it’s—I’m going to—you’re worried about—I’m going to blow up your business because my developers are putting in intellectual property into your product, or I’m worried about you’re going to blow up my business because our production web servers are dependent on when you tell it to be up or down.
I just wish sometimes that the people sending out questionnaires—and guilty of this earlier in my career…
[David Spark] I had to send them out, and I had to receive them. So, by the way, you’re preaching to the choir. We’re all in both camps.
[Nick Muy] Yeah, but it’s like that question is the one that matters. It’s—Is it the data you’re going to lose? Is it some customers? It turns out we’re worried about their employees and
how their employees do or don’t practice the right security behavior and how they access the vendor’s product—not so much the vendor necessarily per se. It was more of an exercise of, how are you going to help me protect the inability of my employees to do the right thing?
[David Spark] Yeah. You know, here, I want to toss this one to you, Steve, because I really like what you just said there. What if the entire third-party risk assessment was, sit down, just tell me, what are you stressed about? What is the thing that worries you? And then we will come back with a list of answers to exactly what you’re stressed about.
And that’s where we’re going to manage a monitor. And that, I think, would make a lot of people happy. What do you think, Steve?
[Steve Zalewski] So, here we are. So, now third-party risk management is not: How am I managing third-party SaaS products? But how am I managing my internal lines of business and the suppliers to that line of business? So, you’ve turned it internally to be able to say, what is it that keeps you awake at night within the lines of business and how they operate to maintain profit?
He’s right on. And so, this is why I found this so fascinating is third-party risk management is not just looking out; it’s looking in, right? And all of a sudden, you’re realizing, yep, because we’re now understanding protecting the company is not the same as securing the company. And we have to think about protection, and we have to talk about the compliance and risk component.
So, I really like this focus because I think what Nick has said is here’s the fourth dimension of third-party risk management.
What else are we missing?
[David Spark] Joseph Longo of GitLab said, “I would love to see a more risk-driven approach.” Ah, we brought this up. “Too many programs paint all vendors with the same broad brush. When a SaaS vendor receives the same questionnaire as a vendor whose product you’ll be hosting on-prem, it just doesn’t make sense to me.
Focus deeper analysis toward your areas of greatest risk and reallocate time towards other efforts that contribute to improving the business.” I think Joseph, even though he didn’t hear your comment, Nick, is doubling down on what you just said right there, isn’t it? It’s—don’t do the same thing for everybody.
What I’m worried about with one vendor is not going to be the same thing I’m worried about with the other vendor. Yes, it should be handled differently.
[Nick Muy] I think Joseph gets it. I think he understands. It’s like you have—you cannot treat them all the same. And if you can’t answer the question about what your greatest risk to your business is, I don’t know how you’re going to assess that in your third parties and in your supply chain.
[David Spark] Steve, what’s your take?
[Steve Zalewski] So, what I like about this comment is third-party risk management is a means to an end. It’s not the end, okay? Understanding risk where I have less and less control over the applications themselves for the company means I better do a business impact analysis, okay? And I really need to understand what the key processes are that I’m trying to protect so that I can then identify all of those business applications relative to that, okay?
And practice on resiliency. Chaos Monkey means we’ve got to practice on understanding that those SaaS applications will periodically go out. And so, which ones do I put the most energy on—either protecting the outage or having a secondary system that I can kick on and the costs associated with that or going tech down?
And so, I think the resiliency component here really with risk is bringing it forward to simply say third-party risk management is a means to an end. Let’s understand the end. Nick said—and there were a lot of comments along that simply said, you’ve got to know what the business needs now because we just can’t provide security uniformly.
Closing
0:00.00
[David Spark] And that brings us to the portion of the show where I ask about your favorite quotes. And I will—obviously, I’m going to echo my favorite quote is yours, Nick. Really? It really boils down to this: What are you worried about? And if we could just figure that out, then we’d solve a lot of problems.
All right, Nick, I will start with you. Of all these wonderful quotes here, not your own, which one was your favorite and why?
[Nick Muy] It’s got to be Joseph’s—Joseph Longo from GitLab. I think his quote just sums it all up. We can’t focus on everything. Focus on what matters and do that, no matter how limited or extensive your resources are. I think you’ll have a better shot of getting the right thing done.
[David Spark] Very good point. Steve, your favorite quote and why?
[Steve Zalewski] So, I like Joseph’s because I think that is the culmination of all of this conversation. It’s how we’re moving forward to get it rentally better. But I’m going to go with Tony Gonzalez at Inner Vision because what he says is, “I have a strong belief to have an effective DPRM process.
It needs to be a life cycle approach and not just looking at assessment, right? The need to incorporate third-party commitment through contracts that clearly state understanding what the business needs.” So, it is a life cycle process, which I think we have to bear in mind. And then we look at Joseph’s and apply that with a risk mentality now, not a compliance mentality.
[David Spark] It’s also like brand building. Brand building is all about touchpoints—all the different touchpoints. So, it’s a combination of what are you worried about and managing all these touchpoints all together. Okay. Good way of approaching third-party risk, I think, altogether. I want to thank Nick’s company—that would be Scrut Automation—for sponsoring this very episode of the podcast.
Remember, Scrut: Stay aware, stay ahead, stay compliant. Just go to their website scrut.io for more about their GRC platform, which will help you, I believe, with this assessment of third-party vendors. I will let you have the very last word here, Nick. And that is, one, I’m assuming people can reach out to you if they’d like to discuss more about third-party risk or any of that stuff.
Heck, anything on GRC. But I believe you’re hiring over at Scrut Automation. Is that true?
[Nick Muy] Yep, we’re hiring a security engineer right now and can’t wait to get that person on the team. Just like everybody else, I think we have a small but mighty team also. And as David said, please reach out. This is something we deal with every day, just internally. We have at least 100 vendors, and we’re doing some interesting stuff ourselves to—how do we actually scale that?
[David Spark] So, you are a customer of your product, dealing with your own third-party risk, and you are helping others with third-party risk. So, you are in a life cycle, and you are helping with life cycles.
[Nick Muy] I am in the middle of it.
[David Spark] You’re in the middle of a lot of it. All right. Then the link to Nick’s LinkedIn profile will be linked on the blog post for this very episode. You can find that. Sorry. It’s Nick Muy—spelled, by the way, M-U-Y is how you spell Nick’s last name, but you pronounce it Muy, not Mui, because Nick is not Spanish.
I don’t believe—I don’t know if any Spanish person pronounces their last name that way anyways, but I’m sure people have pronounced your last name that way.
[Nick Muy] Oh, a hundred percent. I’ve heard it all.
[David Spark] All right.
[Steve Zalewski] Can I have a second-to-last—can I—want to say something, Nick? He gets the last one. Put it in before, right? Which is, before Nick’s, I want to thank our audience. Def Con, all of us. And the number of people that approached me, that emailed me, that called me—it was so refreshing, right, to realize just how engaged our audience is to be part of the solution.
So, thank you, audience.
[David Spark] Oh, yeah. I ran into a lot of—luckily, I got many of them to say nice things on video, and we will be putting out that video very shortly. In fact, we’re aiming to put it out on our sixth anniversary, so you will see that soon—a video of people saying nice things about us. And as I have said many times before, Steve, I have a very high tolerance for compliments.
I can take it.
[Steve Zalewski] Well…
[David Spark] You think I’ve heard enough? I haven’t. I can take more. So, I want to thank our audience as always. We greatly appreciate your contributions and for listening to this show and, heck, all our shows, but for this one, thank you for listening to Defense In Depth.
[Voiceover] We’ve reached the end of Defense In Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site, CISOseries.com, where you’ll also see plenty of ways to participate, including recording a question or a comment for the show.
If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com. Thank you for listening to Defense In Depth.