If you want to get into cybersecurity, there are a multitude of red teaming tools available for little to no cost. So why is it so darn expensive to get any training on the defender side?
Check out this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Steve Zalewski. Joining us is our guest, Ron Gula, president and co-founder, Gula Tech Adventures.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, Query
Full Transcript
[David Spark] If you want to get into cybersecurity, there are a multitude of red teaming tools available for little to no cost. But why is it so darn expensive to get any training on the defender side, the blue team? We’re grooming attackers, not defenders.
[Voiceover] You’re listening to Defense in Depth.
[David Spark] Welcome to Defense in Depth. My name is David Spark, I am the producer of the CISO Series. And joining me for this very episode, you know him, you adore him, you can’t do without him. It’s Steve Zalewski. Steve, say hello to the audience.
[Steve Zalewski] Hello, audience.
[David Spark] That is the sound of Steve’s voice. Our sponsor for today’s episode is Query. Yes, spelled the way you think it sounds. Federated search for security data. You probably aren’t using something like that right now, but you probably want to, especially if you’ve got a SOC where people are staring at single panes of glass.
You’ll be interested to hear what we have to say a little bit later in the show. All right, today’s episode.
Steve, we’re grooming more attackers than we are defenders, noticed Christopher Russell, who’s the CISO of tZERO Group. This is evidenced by the mountain of free education there is for red teaming and really expensive education for the blue teamers. The only way you can get that blue team education or defender education is to either have the money, which it’s very expensive, or work for a company who will provide that, or you can be a customer of the product that you need training access to on whatever the given platform is, and your company will probably have to pay for that as well.
So, Russell argues that blue team training should be free as well as red team training is. Steve, my feeling is this very expensive blue team training could be the firewall that’s preventing us from having all these great cyber experts that we desperately need. Do you think this is what’s preventing it all from happening?
[Steve Zalewski] I think this is definitely part of the problem. What I think, as we’ll talk about here, is the red teaming tools are free, the blue teaming tools are not, and there’s a reason for that.
[David Spark] You will get into that. Yes, there is actually a business model around this as well. Well, the guest who’s going to help walk us through this very discussion is actually someone I’ve worked with before. He used to run a big old company called Tenable, still exists, but now he has this awesome VC firm, of which many of his companies have been wonderful sponsors of the CISO Series.
It is the president and co-founder of Gula Tech Adventures, Ron Gula. Ron, thank you so much for joining us.
[Ron Gula] Hey, David, Steve, glad to be here.
What are the complaints?
2:46.137
[David Spark] Melina Phillips said, “The industry constantly tries to glamorize pentesting, red teaming, while trivializing the work of security operations. If you’re a blue teamer, ‘Then you’re not cool enough.'” Now, again, she put that in quotes herself as well. She doesn’t believe that. Stuart Powell of TSYS said, “There are people worldwide who are time rich but cash poor who would jump at the chance to attain vendor training and certs.
Their employability would improve dramatically.” And Patrick Tsushima of Qualitest said, “Understanding architecture and designing the correct security requirements is a lot more interesting and advanced than rooting boxes all day. You definitely need both.” So, there’s a desirability, Steve, to have a full rounded education.
The demand is definitely there. And I guess if the demand’s there, people will pay for it and that’s why they’re charging. Yes?
[Steve Zalewski] It gets back to the whole offense/defense. And I like the concept of blue team as if it’s my defense, and my SOC is my defense, right? And my ability to determine what an offensive strike looks like. And we’ve built a whole industry around, we have to pay to build our SOC and we pay to educate our teams.
Yet the red team side of the house is, it’s monetized as well, but they have a completely different monetization value proposition. So, I think the challenge there is that difference in perspective more than our interest in wanting to train.
[David Spark] How do you see this monetization model, Ron?
[Ron Gula] So, I don’t know if I agree that there’s this tremendous barrier of getting blue team training. Earlier this year, the Biden administration brought in Google and Microsoft and a lot of other people, you know, (ISC)², and everybody claimed that they’re going to do a million certifications.
Google was going to train a bunch. We’re investors in cyber. We had 300,000 people go through that platform. Now you got to pay for it, but it’s not super expensive. You’re talking about hundreds of dollars to get like a CISSP certification and whatnot. I think the bigger problem is that we don’t appeal our industry to non-cyber people.
We have this overly militaristic kind of nomenclature, “I’m going to go hunt Russia and China on my backbone all by myself,” that sort of thing, and that doesn’t appeal to everybody. I certainly don’t think it’s just paying for blue team courses. That’s what’s keeping people out of this business.
[David Spark] So, the point that you make, Ron, is very good and it’s very recent, and I should mention that the post that Christopher put up was two years old, so a lot has changed per the demand of the administration right now. Ron, what you were about to say.
[Ron Gula] Yeah. So, I mean, being able to break things and download a tool and teach yourself how to do it, that’s a stereotype, people who have time can do that. But if you go back to folks like Ira Winkler, he has the famous Wizard of Oz sort of talk that he does where he says, “Look, you have the ability to defend yourself with everything you have without having to buy anything,” right?
So, you can configure Windows to patch itself. You can configure Chrome. You configure good authentication. You can turn on these things in G Suite. It doesn’t cost a lot of money to learn how to do that basic type of hygiene stuff. The issue is it doesn’t look sexy. It’s not the same thing as dropping a new zero-day or a new exploit on the server.
Blue team just doesn’t. It’s not as sexy as the red team stuff, and it really should be the opposite.
[David Spark] Well, then is it really boiled down to making blue team sexier? Because, I mean, you’re 100% right. You’re not the first one to say that, Ron. Steve, Ron, what do you think? How do we make blue team sexier? I remember, by the way, producing a video, like, how do we make cybersecurity sexy again?
And it’s not easy.
[Steve Zalewski] Well, the other thing about blue team is it’s not within the purview of one individual. Blue team is a team. Your employees are part of the blue team because they have to do their part. It really is a team sport on the blue side, whereas the red side, it really can be one individual who has expertise in one or two tools where he can now try it a thousand times to be successful once.
Whereas from a blue team perspective, we need the entire team, and we have to stop all 1,000, and that’s a much harder proposition.
[Ron Gula] Yeah, and all tools are dual use, right? So, if you’re going to do a network monitoring tool, a vulnerability scanner, UEBA user-based anomaly detection, whatnot, you can use that for offense, you can use that for defense. What I really like about this blue team/red team conversation though is that a lot of modern organizations have a purple team where people will rotate in and out, they will collaborate, they will simulate the adversary, but then they will also kind of try to be more mature about just avoiding the stereotype of the red team breaks in and makes the blue team work the weekend.
We’re way beyond that in this industry.
No one said it would be easy.
8:03.328
[David Spark] Daniel P. of Phoenix Software Limited said, “Resources are not free because people see a huge opportunity to profit from people that seek knowledge and want to join the industry. It’s not just cyber sector, it’s all sectors. Now, the smartest move is to self-teach yourself with all the free and widely available resources out there.
If you can’t find the information for free, then you’re most likely unfit for that industry in the first place.” Strong words. “Being inquisitive is one of the main requirements to be in cybersecurity. There is a plethora of websites offering good training, but for the best, usually you will have to pay.” And Harlan Carvey of Huntress said, “I took responsibility for my own growth and started online and at the library.
From there, I expanded to the bookstore. Develop the foundation first, and then look to vendor training and certifications.” So, both Daniel and Harlan just say, I mean, the information is out there, you just got to hunt for it and be like what a cyber person needs to be. Be inquisitive, search for it.
Yes, Ron?
[Ron Gula] Yeah, absolutely. If you’re going to be in this industry, you have to learn how to teach yourself. And once you teach yourself, you should be mentoring at least one other person. You should be inviting people to this industry because we’re hurting for people. Now, Steve and I, I think we had the benefit of kind of growing up with the internet where technology was rolling out.
Like I got into this before we had Wi-Fi, and I got into this before we had virtualization, before we had mobility, before we had cloud, before we had etc., etc. Now we have AI. So, if you know the basics, you can apply them to these new technologies. The basics never change. Even though we make fun of like the OSI model and the orange book and things like that, you have to learn how things work before you can learn how to protect them and keep people from breaking into them.
[David Spark] Very good point. Steve?
[Steve Zalewski] So, one of the challenges I see is there’s a breadth issue. On blue team, a lot of folks get in running security tools, identity and access management, right? Network endpoint, EDR, data protection edge. So, they understand how to configure the tool and know how to stop a particular type of attack, but they don’t necessarily think like an attacker, right?
And so some of what Ron is saying is, look, you can be a security operations guy, you can be a security architect, but you’ve got to go into the SOC to understand how they think, to realize that while you may be an expert in one domain of security, really what you have to understand is all the domains and the attack paths, and that is more than just curiosity.
That’s time and some kind of natural interest in understanding the breadth of the problem, and then you start to get better. Otherwise, you basically go, “This is my problem and that’s somebody else’s blue team problem,” and that’s the worst possible perspective you can have.
Sponsor – Query.ai
11:08.188
[David Spark] Before I go on any further, I do want to tell you about our brand-new sponsor, and that is Query. Remember I told you, federated search for security data. All right, listen to this. Data overload is a relentlessly growing problem for security teams. I’m preaching to the choir here, I know that, but just hear me out here.
With Query, you can actually break the cycle. As the first open federated search solution for security data, Query provides a new approach to accessing, searching, and understanding the security-relevant data scattered across your security tools, data lakes, cloud surfaces, SIEMs, and other API-accessible systems.
With Query, analysts search once without needing to know a variety of search languages and get back normalized and enriched results using the Open Cybersecurity Schema Framework, or OCSF. Query can be used directly via an API or as a Splunk app to quickly expand your visibility to the security-relevant data you need without more data movement and duplication.
By making better use of the data all around them, security teams are able to effectively reduce mean time to respond. And with more choice on data storage, companies regain control over their systems and avoid vendor lock-in, leading to more efficient SIEM architectures that drive down data-driven costs.
It’s your data. Let Query help you put it to work. Just go to their website. Visit query.ai. It’s query like it sounds. Go there to learn more and get started with a free proof of value deployment. Go check it out.
What needs to be considered?
13:02.889
[David Spark] Glenn McDowell of SAIC said, “I’ve seen tier one SOC analysts struggle to understand what is happening in a threat event due to just having no idea where to start trying to figure out what’s going on without being directly mentored. Now, you can find 100 free samples of how to throw up a reverse shell with Netcat, but I’ve never seen a free resource that will teach you how to take blocked traffic to malware domain event and then walk backwards using your SIEM and EDR to find the Cobalt Strike package that was dropped and the autorun set to provide persistence.” Now, again, these are two-year old comments, so maybe something’s changed in the last two years.
Jonathan Smith of Pioneer Natural Resources Company said, “The path to learning the skills and getting a job is nowhere near as clear-cut as penetration testing.” That, by the way, that’s a good point as referred to the previous segment. “I find myself wondering if it’s better to start out red team and switch over to blue team after I have gotten into cybersecurity or find a way to work on blue team skills in hopes of finding a job.” I think that last sentence is kind of the story for most people.
And lastly, Anthony Browness of Ninety.io said, “I see many commonalities mirrored in the QA industry. The only difference is QA went open source early on with tools like Selenium that became industry standard within a decade, whereas InfoSec seemed to go full-bore money grab, and what are they showing for it a decade later?
Exactly this.” And I think he’s referring to the fact that we need a lot of people. So, I’m going to start with you, Steve. A lot of sort of powerful comments. One is Glenn’s comment of like, “Nobody shows you how to do this very specific path of blue teaming.” And the other person saying, “Well, I guess I’m going to start with red team and hope to get blue team skills or maybe learn blue team and hopefully get hired that way.” And the last person saying, “Hey, other industries have tried to do it.
Why can’t security?” Steve?
[Steve Zalewski] So, there’s another point here, right, which was, you’ve heard the phrase, “The best defense is a good offense.” And so to the degree that you can get into the head of your adversary, right, and know how red teaming works, for you to then take that back, I think is very relevant here.
Because if you just come in as a blue team and you don’t understand how they come through, I’ve seen this before where once you’ve done opened the eyes to see what an attack really looks like, the light bulb goes off and they’re just realizing, “Oh, it’s not just we do multi-factor everywhere and solve the problem.” They’re realizing the different facets of vulnerabilities and the different facets of exploitation.
And it is a non-trivial exercise to look at that and then take a step back and go, “All right, so now I’m in the SOC, how do I understand what attack chains look like? How do I look at the tools that I have? How do I build the defensive perimeter to be able to think about that?” That is a much larger exercise in education and training than, quite honestly, to just be a red team member and be really good at one or two tools and just exploit.
[David Spark] Ron, what do you think of the frustration that I just sort of listed out there? Do you think it’s – let me ask you because you sort of came from a different angle at the beginning – do you think this frustrated student is warranted or they’re just not looking hard enough?
[Ron Gula] Well, I think the frustration is, it’s always going to be there, but if people just want things spoon-fed to them, then they shouldn’t be in this business, right? Because even if you learned everything that was in front of you today, it’s going to change tomorrow. There’s new tech, there’s new things, got to do that.
A couple of things in the quotes though, that kind of struck me. There’s one comment about how we need to open source everything. Look, I mean, we closed source Nessus because it was a huge amount of effort to keep it updated. It was not a community of people who were contributing new checks and stuff like that.
It was a community of users. And most of the open-source projects out there are less, 5% to 95% – 5% development, 95% user, that kind of thing.
But the reality is making this stuff free doesn’t really solve the problem. The adversaries who are going after these organizations are extremely well-funded. Do they have some open source in their stack? Yeah. But are they building really custom implants? Are they building new types of TTPs? Absolutely.
And are they injecting in ways that are even hard for a typical red team to do? Absolutely. So, people need to understand how things get broken into and look for that. And that’s what makes blue team stuff real exciting. The ability to touch these breach and attack simulation tools, the ability to take the most recent threat intel and walk into your board, walk into your CIO and be like, “Hey, look, our Help Desk can be hacked by SMS, and here’s how it’s done.” So, that’s why this thing can be exciting.
[David Spark] Steve, someone eager to get in has said, I’ve done these free penetration tests or the programs, but they are expressing the frustration that they think things are too expensive, although we are seeing in the most recent year that this is very much changing. How do you respond to them when they express their frustration to you?
[Steve Zalewski] Kind of what I talked about, which is best defense is a good offense, which was, well, go use the red team tools. You don’t have to start with, “blue team tools.” Understand the problem and then be part of the solution, right? So, I’m like, by them going into what Ron said, oftentimes the same tool can be used for offense or defense, okay?
It’s literally the same tool, same concepts. It’s just the mindset. So, oftentimes they say, go start with red because it is cheaper for a lot of people, it’s more interesting, and then they realize the challenge of being blue. And I say, so start with that.
The other thing from a frustration standpoint beyond the tools is kind of the perspective about prevent, detect, respond, recover, right? If you look at the NIST framework, how much of the security organizations really focus on prevent? They try to just prevent the problem from happening, and yet blue teams are based on detect, respond, recover.
So, there’s also a certain maturity that many security organizations aspire to get there on the maturity curve, but they’re not. So, that also kind of creates a negative perspective on blue team because it’s we’re not mature enough to have one.
Why does it matter?
19:40.001
[David Spark] Rob Wood who’s CISO over at TrustCISO said, “The rigor required for blue teaming is higher. The penalty for a vulnerability you miss is lower than for an attack you missed. So, an attack is happening now. A vulnerability might be an attack later. I’m not saying either is less important, but they do require different mindsets, behaviors, and demands on diligence and accountability.
Businesses are focused on defense and part of the revenue model is to charge for training. Their products would be an easier choice if the market was awash with skills in their product.” So, that’s a really kind of good point, and I’ll start with you, Steve, on this is, if everyone knew every product, then yeah, we’d be in great shape, but that’s kind of unreasonable, isn’t it?
[Laughter]
[Steve Zalewski] You know, we talk about this, which is there’s something like 4,000 security products out there, 4,000 security companies. Much of it is duplicate, much of it is partial, and so from that standpoint, again, it gets back to overwhelming. I don’t know 4,000. How many do I need? Well, then, I mean, if you think about blue team and red team, which is, well, what are the threat vectors?
Who is attacking me? What does my defensive perimeter look like, right? And therefore, what is most important for me to understand from an attack chain and which tools? I guess where I’m going with this is to be blue team, there’s a higher bar of understanding more of security, and defense generally is harder than offense.
And so for all the blue teamers out here and why does it matter, I think that’s why we’re saying be creative and realize blue team is the higher bar and it is a harder problem, but there’s no reason why you can’t get there with red teaming to start with or leveraging some blue team, but just know by us having this conversation that they’re not equivalent.
[David Spark] By the way, Ron, and maybe you’ve seen this at even previous companies or any of your portfolio companies, are they listing out, “Must have experience with product XYZ”? Because my feeling is there’ll be people ready to learn product XYZ, but they just don’t have that very specific experience, and they may be knocked out or shy away from applying.
Have you run into this issue?
[Ron Gula] So, I have a lot of experience on the Tenable side, of course, and what we see is that a lot of people who are in the military and the government who have Tenable experience, they can get out and go into industry and roll up to a commercial job, even consulting, that sort of that. I see that a lot.
But a couple of comments based on the quote. So, one, almost every vendor charges for training, but in the end gives it away because it’s the kind of thing that keeps your customer happy. You want them to use all the features of your products. Every company I work with, every founder says people only use 10, 20, 30 percent of my products, right?
That goes to glut and having too many products that are out there. But to the red team and blue team stuff, I say, look, if your red team is bored, they should go work in the blue team for a while. And if you ever hear a red team, like, “We always get in,” well, if you always get in, you’re really doing a crappy job of educating the blue team on how you got in.
You should never get in the same way twice, that sort of thing.
[David Spark] This is leaning towards the need for purple teaming.
[Ron Gula] Absolutely. But if your red team is not getting in and you’re using open-source tools, you’re waiting for exploits to drop on the internet, you really need to move towards breach and attack simulation. So, we’ve invested in SCYTHE. That’s a tool where you can create all sorts of really interesting malware implants, so to speak.
And just basically, you can put them out there and say, how long does it take people to find these things? And it changes your vision of security. You move from, “Are we patched? Are we compliant?” to “How long did it take those implants to be found?” or “How long did that insider activity…did it take to be found?” And that really makes the blue team more exciting as well, knowing that they have something on the network that they’re looking for, which is really not any different than what they should be looking for every day – Chinese and Russian malware.
[Steve Zalewski] And I want to chime in on that a little bit to what Ron was saying here, which was vulnerabilities versus exploitabilities. If you know you’re on blue team and you’ve seen the vulnerability scans and you’ve got a thousand vulnerabilities, okay, it’s a little disheartening because you’re in essence going, “Look at all the ways that they can come in,” and therefore it’s whack-a-mole, right?
And no amount of due diligence on your part is really going to get there. But moving to exploitability, right? Understanding the attack chains, and then ultimately looking at material exploitability from a blue team, which is being able to thwart the attacks. It’s not be perfect at the front, but understanding how to thwart, it re-engages them because they’re understanding how they’re out-thinking the red teams.
And then if the red teams come over to the blue team, then you’re having a conversation around the attack chain and what to do. And it’s not about a thousand vulnerabilities. It might be about 40 or 50 material exploitabilities. And now you’re really starting to tighten up your defense. So, another way to look at the problem actually being an opportunity.
[David Spark] Very good point.
Closing
25:19.849
[David Spark] Well, that brings us to the portion of the show where I ask both of you, which quote was your favorite and why? And I’m going to start with you, Ron. Do you have a favorite quote and why?
[Ron Gula] So, the quote from Anthony Browness about the QA relation to this, it really resonates with me because if you think about like an airplane, how do you QA an airplane, right? There’s a huge checklist before it even gets… There’s a checklist before the pilot takes off. There’s even a hundred different instruments and ways to measure things multiple ways.
If you’re not doing that on a blue team, you’re kind of doing it wrong, right? So, you need to have multiple ways to understand what’s going on in real time so you can have the outcomes. You need to treat it like QA. Too many people kind of treat it like, are we secure? Are we not secure? If you can have five or six different ways to measure, am I compliant?
Is it patched? Is that machine supposed to be there? You’re now on your way to automating this problem and really frustrating the red team’s day.
[Steve Zalewski] Here, here.
[David Spark] Steve.
[Steve Zalewski] I’m going to go with Jonathan Smith at Pioneer Natural Resource Company, right? “The path to learning the skills and getting a job is nowhere near as clear-cut as penetration testing,” right? And that was one of the themes that I was talking about, which is blue team means you got to know what the red team does to really be effective at blue team and looking at exploitability.
So, I would say this is blue team takes the higher route. And so the training that you’re getting on blue team specific and red team is realize kind of the scope of what we’re doing here, and now you see why purple teams become very, very relevant. Because if I’m really light on blue team, maybe I can only assign one person in my blue team to do this.
And I’ve got two or three on red team because I’m using outsource capability. Purple now becomes that training opportunity. It’s not just tooling. It’s the process by which we’re using the tooling.
[David Spark] Well, that brings us to the very end of the show. I want to thank our sponsor, and that is Query, federated search for your security data. Just go to their website, query.ai. Get a good look at what they’re doing there. It’s pretty darn awesome. Look, if you’ve got panes of glass in your SOC, you’re only seeing a portion of what’s going on.
Why not look at all of your data without having to ingest all of your data? Check them out, query.ai. And I want to thank Steve and our guest. Ron, we always let our guests, if there’s anything else you want to tell about Gula Tech, I know you work on a ton of nonprofit efforts as well. Give us any final words for today’s show.
[Ron Gula] Yeah, thanks. So, everything we do is over at gula.tech. We’ve got a pretty large YouTube presence, a lot of interesting videos, and we just awarded our $6 million grant from our Gula Tech Foundation, which were for nonprofits who worked on the national cybersecurity strategy pillars one and two.
[David Spark] Wow. That’s great. Well, if people want to find out more, just go to gula.tech. Thank you again, Steve. Thank you very much, Ron. And thank you to our audience. We greatly appreciate your contributions and listening to Defense in Depth.
[Voiceover] We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site CISOseries.com where you’ll also see plenty of ways to participate, including recording a question or a comment for the show.
If you’re interested in sponsoring the podcast, contact David Spark directly at David@CISOseries.com. Thank you for listening to Defense in Depth.