Can AI improve Third-Party Risk Management (TPRM)

Did you know AI can fill out your security questionnaire for you? Well, if you didn’,t you’ve missed all cybersecurity marketing in the past two years. Getting rid of the questionnair grunt work is huge, but it’s the tip of the iceberg regarding how AI can help us deal with risk. So where are the biggest opportunities?

Check out this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Jason Elrod, CISO, MultiCare Health System. Joining them is Nick Muy, CISO, Scrut Automation.

Got feedback? Join the conversation on LinkedIn.

Huge thanks to our sponsor, Scrut Automation

Full Transcript

Intro

0:00.000

[David Spark] Did you know AI can fill out your security questionnaire for you? Well, if you didn’t, you’ve somehow missed all the cybersecurity marketing for the past two years. Getting rid of the questionnaire grunt work is huge, but it’s just the tip of the iceberg when it comes to how AI can help us deal with risk.

So, where are AI’s biggest opportunities for defenders?

[Voiceover] You’re listening to Defense in Depth.

[David Spark] Welcome to Defense in Depth. My name is David Spark, I am the producer of the CISO Series, and joining me for the very first time as a guest co-host, it’s Jason Elrod, CISO of the MultiCare Health System. Jason, thank you so much for joining us.

[Jason Elrod] Excited to be here.

[David Spark] Thank you, yes, for being here, and I do want to remind everybody that we are available at CISOseries.com, where you can find all of our wonderful programming, especially our new show, Security You Should Know. Our sponsor for today’s episode, who is also responsible for bringing our guest, it’s Scrut Automation.

Stay aware, stay ahead, and stay compliant. We’re going to be talking about that a little bit later in the show, but our topic today is about AI integration. And Jason, for integrating AI into third-party risk management, we’ve started to tackle some of the low-hanging fruit, questionnaires being the obvious one.

We’re nailing it. Congrats, industry. Awesome. But what are some of the other areas we’ll see dominated by AI in third-party risk management? And after that, what are the potential use cases that get you excited? So, I just ask you, where are you in sort of where AI is going to help us?

[Jason Elrod] I think where AI is winning right now is security questionnaires, handled. Automating vendor risk scoring, getting there. So, I think what’s next, we’re probably going to see some great movement around continuous monitoring, where AI can flag vendors, security drift in real time. I think that’s a big miss right now or a big gap that could be solved for it, as well as smart risk prioritization.

Instead of just listing risk, AI can maybe triage them based on real-world impact. And I would go one more, probably threat intelligence integration. AI-driven models can contextualize risk beyond basic compliance because checkbox is not where we need to be.

[David Spark] You have hit all the hits, if you will, because our audience has definitely been talking about a lot of that stuff. And in fact, this is what we put out to the audience. We put out a question to the audience and they fed us back a lot of insight on this topic. And to help us with this conversation is actually our sponsor guest from Scrut Automation.

It is the CISO, none other than Nick Muy. Nick, thank you so much for joining us.

[Nick Muy] Great to be here, David. And great to speak with you, Jason.

How do I go about measuring the risk?

2:44.258

[David Spark] Rashique M. of Capital One said, “In TPRM…” we will use that term instead of third-party risk management, just so you play along, “In TPRM, there are a lot of risk accepts in place by the enterprise.” So, they’re accepting, not excepting, but accepting them. “One potential avenue for innovation would be to leverage AI models to aggregate the third-party risk data that sits in the enterprise system of record and provide threat intelligence on the risk accepts in the enterprise data,” I like the marrying of the two, “The data can be used to change the decision factors for risk accepts and influence mindset shifts across the enterprise regarding which third-party risk the business should accept versus remediate.”

And Phillip Miller, who’s the CISO for Qurple said, “Two broad areas that could supercharge the humans in risk teams, A, collating, categorizing, and organizing the submitted content into a taxonomy that makes sense for your risk-based decision making, and B, searches that can evaluate context versus keyword queries to speed up reviews.” So, Jason, I’m going to start with you.

Rashique and Phillip, I think what’s really kind of interesting is let’s start getting these threats and what our business is trying to do married together. Have you seen any of this or are you starting to see it and you’re getting excited about it? And do you believe that this is going to happen?

[Jason Elrod] I think this is the right direction, moving away from shifting from accepting risk to understanding risk. Risk accepts are common, but no organization, at least any that I’ve encountered, really understands the cumulative impact of all those little risks they’ve accepted over time.

[David Spark] That’s a good point right there.

[Jason Elrod] It’s almost like Mr. Creosote, right? Just one thin mint later and the whole thing is going to blow, or the AI search I did the other day, how many mosquito bites does it take to kill you?

[David Spark] And excuse my ignorance, but are there situations where two small risks put together all of a sudden turn into a gigantic risk?

[Jason Elrod] Of course. AI is prime for aggregating the risk data across different risk factors. What’s often missed is that by risk one and risk two is the process of “and-ing,” which is the combined risk. So, not only do you have risk one, risk two, but you have the cumulative effect of risk three, and if there’s a synergy there, you’re bummed.

[David Spark] You don’t want those helping each other out. Nick, this is a really interesting sort of setup by Rashique and Phillip saying let’s get essentially the threat intelligence married with our risk profiles and see where we’re at. What do you think?

[Nick Muy] First off, just as you were kind of going through Rashique’s insight there, I was like, “Big difference between accept and except.” [Laughter]

[David Spark] Yes, yes.

[Nick Muy] That could be a dangerous mistake to make.

[David Spark] I had to read that a couple of times just to make sure that I was clear to the audience of the difference. Go on.

[Nick Muy] I mean, every security professional can relate to exception requests. I think the hard part that I’ve seen kind of firsthand is having the data about your risk profile is hard, sometimes harder to get than kind of squeezing your vendor to give you information about theirs. So, threat intelligence we can get from, you know, there are great vendors out there that sell threat intelligence as a service.

If your team has analysts on it, they can go research themselves, sometimes having a clean, clear picture that’s up to date. Maybe up to date is the key word. And when I go ask my friends about, “Hey, how well do you keep track of your company’s risk profile, and how do you manage kind of what your risk appetite is?” They’re like, “Oh, we do.” And then I was like, “How recently have you updated it or how frequently does that happen?” “Oh, well, it’s a once-a-year activity or when we get around to it.” I think it would be great to do this.

I think this is the right thing to definitely go do. There’s tons of opportunity, I think, as both the comments and what Jason was saying is like we have to get that information together to help people understand what’s happening instead of trying to focus on volume.

I think there’s a big focus when I’m talking to my peers at other companies and I ask them, “What are your stakeholders measuring your TPRM on?” They say, “Kind of throughput. How many vendors am I assessing?” It’s not because that’s the most important thing, but I think it’s easy to understand. So, I think an executive asking a CISO, “Hey, are you actually assessing all of them?” But all of them aren’t equal, and I think to the points in the comments here, it’s like different vendors have different threats that they’re vulnerable to.

You use them differently in your environment. So, I think those are the important things to kind of watch out for as we use AI to build solutions for this.

[Jason Elrod] I love something you said there. All risks are not created equal. To treat them that way is a gigantic misstep on there. And you’re right. The TPRM needs to be connected to the BIA of that specific risk, which is something I don’t really see going on in a lot of organizations.

What’s the next step?

7:49.998

[David Spark] Eduardo Ortiz of Techtronic Industries said, “Predictive risk analysis incorporates AI to produce models that can forecast potential risks by analyzing historic data, identifying trends, and predicting vendor failures, cybersecurity breaches, compliance violations, etc. We have been thinking about this lately after going through a few with our GRC team with the main purpose of moving from a check-a-box TPRM exercise to one that provides insights about the real posture of the organization that helps us to make a better-informed decision.” So, this is a nice sort of continuation from our last segment here.

Mauricio Ortiz of Merck said, “There are a few areas where AI can help improve the TPRM process. One, risk analysis and assessments, as an AI agent could be designed to analyze vast amounts of data for each vendor to provide key insights about risk scoring and compliance metrics. And two, it also helps to enhance risk prediction with real-time monitoring of each third party to flag unusual activities or disruptions.” So, Nick, I think what’s interesting here, which very nicely sort of continues from our last segment is, all right, if we have this information, we can start making better decisions and kind of like what you said, doing things a little bit more in real time.

So, walk us through how you kind of would see this sort of manifest itself.

[Nick Muy] Yeah. And I really appreciate these comments because, I mean, I think it sounds like everyone agrees we have to move beyond the checkbox. I think that’s what we all want. Getting the historical data sometimes is tricky. If it’s a vendor that you’ve had for multiple years, then you have kind of firsthand historical data.

I think getting it for a new vendor, I’ve found, I think there’s a gap that needs to be closed there – how do we get the historical data to actually predict the future failures in behavior I’m not aware of? There’s not one good place to get that today. I think you can work with what you have. You can get it from the vendor directly.

There’s not exactly a data set all of us have access to, to go reliably do that. I think that’s the ideal. I think we need to find ways to work together across the industry to create that kind of data and make it available.

And then I think once that’s there, I think to what Eduardo’s getting to, is like there’s so much better that we can do to actually, I think, make TPRM kind of risk decisions that are keeping up with the business, that are smarter, that aren’t one-size-fits-all. I don’t think anyone, really, I mean, I can’t seriously believe the one-size-fits-all risk decisions work.

[David Spark] Nobody believes that. We’ve heard it again and again that the risk rating on a vulnerability doesn’t mean anything to the specific individual.

[Nick Muy] Yeah. And especially the rating of a vendor too. This is maybe controversial. The risk rating of a vendor, if we share that data, even between Jason and myself, depending on how we use that vendor, there are different risks for him and I. So, we couldn’t even take the exact same… I couldn’t lift and shift his rating versus our own because we have different businesses.

And that goes back to like, well, what risk is my company worried about? And how are we kind of handling that?

[David Spark] So, let me throw this one to you, Jason. It seems like, as I’m reading this, is that we have the potential to see these sort of big softball-size holes that are of great concern to us once we’re marrying all these elements and AI is able to sort of in real time know what’s going on with us and with all our third parties as well.

Again, I’m just sort of envisioning that this could be done. What’s booing you to believe, yes, this is within our vision right now?

[Jason Elrod] Well, I think Eduardo mentioned something about predictive risk analysis, and I think AI has a really good chance here to move us from a react and respond, classical, to a predict-and-prevent standing here. Most often when classic third-party risk management is, let’s review them at the beginning of the contract and then three years later or whatever the contract length is, let’s review them again for the renewal, what happened in between?

And I think that plays to what Mauricio said about enhancing risk prediction with real-time monitoring.

So, if you have AI out there doing real-time monitoring of various data sources and various factors associated with various third parties, that’s going to give you that real-time risk detection, things that’s continuous in that stream that I never would have noticed, I never would have had the time to look at, and that I think LLMs or generative or even agentic AI are really good at doing.

Like looking at all these disparate data sources, drawing conclusions, looking for frameworks, and then almost getting a little minority report about it, right? What’s the pre-crime? What’s the pre-risk about to happen? Which is great because that’s going to give us a little bit of time to react, to inform, to assist, and to collaborate with our third parties.

Hey, we’re noticing this, we’re kind of seeing this trend. Is it right? Is it wrong? How does this work together? And then back to Nick’s point, we’re going to share that data out, and how does that work?

[David Spark] So, let me just throw this out. How do we know this is even going to be right? This minority pre-crime, which would be great. So, we all agree this is a threat, it’s happening right now, and we’re going to stop it. Like, what was it? Are we going to know?

[Jason Elrod] Hallucinations, bias.

[David Spark] Who knows?

[Jason Elrod] These are all things that pop up with AI, right? Well, they also pop up with people, too. And the point being is it’s not going to be 100%, but what it’s going to be is much better than what we have now. So, if we can say, “Hey,” if we can get the justification for the suggestion or the risk rating, “Hey, I arrived at this risk rating because of A, B, and C seeing 1, 2, 3, that at least gives us a jump.

That’s a force multiplier for our teams. It’s not a force replacer. And I think that’s what we have to understand, that it’s not a replacement for that, but it is a multiplication of the capabilities.

Sponsor – Scrut Automation

13:50.112

[David Spark] Today’s episode, it’s brought to you by Scrut Automation, and they’re the company helping businesses eliminate compliance debt and take control of risk. That’s what we’ve been talking about, doing it all without slowing down, something key you like to hear from your security tool. Now, managing compliance and security shouldn’t be a roadblock to growth.

Scrut simplifies the process with automation, AI-driven insights, and real-time monitoring. So, teams can streamline audits, track security controls, and stay ahead of regulatory requirements without the manual workload.

With Scrut, security and risk professionals gain instant visibility into their compliance posture, automate evidence collection, and manage multiple frameworks effortlessly. AI-powered insights help teams focus on what truly matters, reducing risk, maintaining trust, and scaling with confidence. Do not let compliance slow you down.

Scrut helps you stay secure, audit ready, and focused on growth. You want to do all of that? You got to go to their website. Visit their website, scrut.io, to schedule a demo or just learn more. That’s scrut.io. Check them out.

Would this work?

15:02.512

[David Spark] Andrew Shea of CRFQ said, “There are a lot of opportunities from an AI perspective. In the near term, third-party contract analysis. Based on AI analysis of the contract, what tier should they be put into? What questions should be sent? And longer term, what does continuous monitoring look like for them?

If they are a key application provider that hosts their own data center, and they are in an area like Ukraine with geopolitical issues, then monitoring would be expanded to not only be targeting security issues, but more holistically, increased geopolitical tensions.”

And Kade Hennings of Mimecast said, “Training in AI on the various requirements that your company needs to adhere to for third parties, then having it map submitted documents and publicly available data to find the gaps, using it in a way to suss out actual risks based on user rather than blanket assessing.” All right.

I’m reading these, Jason, and I think we can do this today, in that a lot of this is, “There are a lot of different elements I’m not thinking of. Help me out, AI. What are the risks I’m not thinking about when dealing with this third party?” Yes?

[Jason Elrod] Yeah, I think that plays, we were just talking about the force multiplication. We can do this already, but moving at the speed of people is really, really slow.

[David Spark] Yeah, it’s tough. And also, it’s not just that, it’s what am I not thinking about right now? And the AI will, because it’s got the power to think about everything, or really not think, but because it’s learned, it knows to go out and look for everything.

[Jason Elrod] Yeah. What’s the gap? Mine the gap. And I think AI’s well-placed for that.

[David Spark] Nick, I throw this to you. I feel that this segment here shows the most promise of showing us real value very quickly. Yes?

[Nick Muy] Yeah, no, I mean, I love Andrew’s comment. The contract, as boring as it is, does establish what the vendor is agreeing to be obligated to, what the customer is agreeing to accept with an A, and because of that, I think that kind of sets the baseline for this kind of represents what you’re worried about, what you claim to be worried about, what they claim to be okay with doing for you.

And if you don’t start there, you’re kind of just doing it in a vacuum. And I think starting there, as Andrew said, is not something we have to wait for minority report to go do. We could do that today.

[David Spark] I will throw this out, and we actually did this. One of my employees did this very thing in that we were writing a contract, and it didn’t have any sensitive information in it, but we uploaded the contract to ChatGPT and said, “What’s missing from this contract?” And it gave us some really good advice, like, “Oh, you’re missing this, this, and this,” and we took some of their advice to put it in.

And I was thinking, you could kind of do the same thing. You could feed the contract and go, “What is missing that I should be worried about from this contract?” Could be the same way. Someone writes a contract to say, “I’m good at this. I have no problem with this.” You may be missing [Laughter] what they’re not good at.

Yes?

[Nick Muy] I mean, I think when vendors put their contracts together, they know what they’re good at, and then they don’t say anything else.

[David Spark] Exactly. [Laughter]

[Jason Elrod] I look at a lot of contracts and it’s that point. What’s missing? What’s hiding? Or how are they tilting this in their favor and not ours from an organizational standpoint? And that’s that “what am I missing” content. And back to something you said earlier, Nick. What risks are important to me as an organization?

And does this satisfy those? Because you mentioned risks, posture, risk understanding, risk intelligence is different for every organization. I’m in healthcare. I’ve got a lot of compliance things I’m following on it. But if you’re in finance, you have a different set of compliance versus retail or production or whatever.

So, being able to tilt that towards, “Hey, here’s this contract. By the way, here’s HIPAA and all these other things. Pivot against that. Let me know what I’m missing.” And we can easily do that. I think that’s a good way to enumerate risk that might have been hidden before that AI can actually bring out.

[David Spark] Excellent point.

What else are we missing?

19:08.537

[David Spark] Ahmed A. of Digital Macro Strategy Corporation said, “Automating risk management using AI does not fulfill legal obligations around the risks.” He doesn’t feel so strong about this. “It only morphs it into another form of risk and creates a higher opportunity for false positives to occur.

Most legal systems have caught up to how tick-the-box exercises don’t stack up.” So, Ahmed is not as bullish. And we threw this in here to see your take, and I’ll start with you, Nick. What do you think about someone who’s sort of kind of a naysayer on letting AI run? And I’m going to just say run wild.

Feel free to couch it any way you would like.

[Nick Muy] Totally fair. And I appreciate Ahmed’s healthy skepticism. I think today what I see in a lot of products, and I talk to our own product team about this as an end user and a customer, most of them, when they’re promising what AI is going to do, very, we’ll call it smartly, make sure that the output requires my intervention, usually.

It’s very smart, I guess, what they’re doing, saying Nick accepted the output of this before it finished the automation or the workflow. And so that there’s a lot of work being automated in the risk management, but that the risk management decision was Nick. Nick made that decision. I took that decision because I accepted it.

I didn’t have to, I wasn’t forced to, but I chose to, I guess, of my own free will, supposedly.

So, I think there’s accountability, look, at the end of the day, and I think it’s fair to say, we have to be realistic about the promise, but I also think there’s work not getting done, and we have to solve that too. There’s risks that aren’t being looked at. There are things we’re probably ignoring.

And I think to everyone’s point here, where it’s like, “What are we missing?” We have to find ways to figure that out, and I think sticking with the status quo isn’t going to help us. And I think just being honest and transparent about where does the AI help start and stop. And then I think that’s where we are today, I think, as an industry.

Maybe in the future, I think the promise of a completely autonomous kind of activity maybe is out there, but I also think we have to start somewhere. We have to try to make this work better.

[David Spark] Yeah. Look at it this way. It’s like these driverless cars that are out on the market. Now, I honestly don’t know the stats out there, but I believe they are causing fewer injuries than when humans are driving cars, but because it’s a robot driving it, we have greater concerns. I think it’s the same here.

I don’t think we’re completely giving up control to the machines, but there is a sense of the robot is driving it. Even though the way we’re doing it is way worse, I’m still concerned about the robot driving it. Yes, Jason? I mean, this does come up, doesn’t it?

[Jason Elrod] 100%. We touched on it a little bit, but to Nick’s point, AI doesn’t eliminate risk. At best, it transforms it. And so, don’t think it’s just because you applied AI to it, it’s going to eliminate the risk. And we talked about how AI models introduce their own risks, false positives, bias in training, training data, and possibly regulatory concerns.

All legal obligations still apply. AI-driven assessments don’t replace due diligence. And that’s, again, to Nick’s point. You’ve got to sign off on it. It can help you, it can get you there, but at the end of the day, you have to say, “Yes, Your Honor, I personally thought it was a good idea to now [Inaudible 00:22:38] that thing, check the box for the AI.” So, we have a challenge there.

AI should augment decision making, not replace it. It’s going to be rough because eventually, how do you hold an AI accountable or responsible for a risk acceptance with an A somewhere, or exceptance with an E, how do you hold it accountable? And it comes down to the human. And so, you really got to be there.

Our role is to avoid the black box problem, CISOs. We need transparency into how AI’s making risk decisions so we, when we sign off, can justify them. And I think that’s what’s really important here. And back to the cars. I know why a human might dodge or might go this way or that way, [Inaudible 00:23:16] trolley car.

Which way is the car going to go? I don’t know that in a car that’s automated driving. Even though in 99% of the times, it’s safer than that 1%, would I rather have a human making that decision? Maybe.

Closing

23:28.828

[David Spark] All right. We’ve come to the point of the show. I’m going to start with you, Nick, on this. I’m going to ask you which quote here was your favorite and why?

[Nick Muy] Andrew’s quote about doing the contract analysis. I just felt like this is practical. Having looked at so many vendor contracts before for huge, scary deals, that moment where you’re about to sign a three-year contract, and I think, why don’t we do this more?

[David Spark] And also, let me just throw out, for those of us who are not lawyers, reading legally sometimes scares the crap out of us. [Laughter] It’d be nice to have our hand held a little bit in a situation like this. Yeah. And again, I’ve done something somewhat similar, and I was delighted by the response.

Jason, which quote was your favorite?

[Jason Elrod] I’m going to have to go with Eduardo and the predictive risk analysis. I love the concept of pre-risk, dealing with it when it’s really small before it’s gone into something big with a large impact. So, the better we can do that on a continuous fashion, the better we’re going to be as organizations, as well as with risk management.

[David Spark] Very good takes. Thank you so much, both of you. I love our community with these great insights. And also, like what you said, Nick, we appreciate Ahmed’s good skepticism here. I think we needed that as a closer to sort of explain the balance of this argument and where people should be excited, and also having sort of a handout and saying, “Well, whoa, whoa, not too much too soon,” kind of a thing.

It is very, very healthy. And so, continued on that way, as Jason is doing the wave “come towards me, stay away” look right now on the camera. All right. I want to thank your company, Nick, Scrut Automation for sponsoring this episode. Remember, go to their website, scrut.io. I want you to have the last word here because I know you guys are doing something in this very area that we’ve been talking about today.

So, please talk about it. Remember their slogan – stay aware, stay ahead, stay compliant. Jason, thank you so much for subbing in as guest co-host. I loved it. Now, Nick, you get the final word, correct me if I’m wrong, Scrut is trying to do some work here on AI and TPRM. Explain.

[Nick Muy] So, personally, I very much care about this. And so, the company is doing a lot of work in this space and has applied AI to the TPRM challenge. And I think for our customers who don’t have big teams who need the resources to get this job done and don’t want to let things fall by the wayside, because I think the biggest fear that we hear from our customers is TPRM risk that they’re ignoring or not aware of because they don’t have a team to go follow up and chase with every vendor, so we’ve provided a way to help them do that.

So, throughout the entire majority of the TPRM lifecycle from the beginning, where it’s helping companies classify the vendor and the type of risk they are for that company, and not just taking it kind of off the shelf, one-size-fits-all.

And then following that through. Making sure that if I’m worried about PCI compliance, and that the vendors I’m talking to are going to handle my PCI data, that they’re getting custom-tailored specific assessments that are not the same thing a vendor handling different kinds of data would. And I think we’re really looking at meeting people where they’re at to get them, I think, the help that they need now today, and not just, I mean, not the pre-crime future yet, but help now, help today, and getting the job done.

And we’re using this for all of our own vendors, for my own vendor risk management.

[David Spark] Let me ask you, in all your beta testing that you did, what did you learn from your customers about this TPRM process? What’s one unique thing that you didn’t know going into it?

[Nick Muy] The interesting thing is every company really has a pre-assessment process where they’re really predetermining the potential risk of a company. And that pre-assessment is like a TPRM assessment before the TPRM assessment because almost every single company we talk to is thinking about they can only assess so many vendors, and so they’re all assessing the risk of the vendor before assessing the vendor.

And that pre-assessment process is actually critical because how they do that and what they need to know to make that assessment determines who they actually spend time with because all of them don’t have infinite time. I mean, the time thing, that’s no news to anyone. Why the time thing? Because they all have a different way of assessing who to assess.

And then we really found there’s like TPRM is an assessment within an assessment.

[David Spark] [Laughter] It’s a whole game of inception, if you will.

[Jason Elrod] It’s like a little nesting doll of process.

[David Spark] Thank you so much, Nick. Thank you so much, Jason. Thank you to Scrut as well. And thank you to our audience. We greatly appreciate your contributions and continuing to listen to Defense in Depth.

[Voiceover] We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site CISOseries.com where you’ll also see plenty of ways to participate, including recording a question or a comment for the show.

If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com. Thank you for listening to Defense in Depth.