In a recent episode of McAfee’s SOCwise Series, guest security expert Chris Crowley revealed findings of his recent survey of security efforts within SOCs. His questions were designed to gain insight into all things SOC, including how SOCs can accomplish their full potential and how they assess their ability to keep up with security technology.
Hosts Ismael Valenzuela and Michael Leland tapped into Chris’ security operations expertise as he told “A Tale of Two SOCs.”
“Chris has a tremendous experience in security operations,” Ismael said. “I always like people who have experience both in the offensive side and the defensive side. Think red, act blue, right? . . . but I think that’s very important for SOCs. Where does ‘A Tale of Two SOCs’ come from?”
In reference to the Charles Dickens’ classic, Chris explained how survey responses fell into two categories: SOCs that had management support or those that did not.
“It’s not just this idea of does management support us. It’s are we effectively aligned with the organization?” Chris said. “And I think that is manifest in the perception of management support of not management support, right? So, I think when people working in a SOC have the sense that they’re doing good things for the organization, their perceptions is that the management is supporting them.”
In this case, Chris explains “A Tale of Two SOCs” also relates to the compliance SOC versus the real security SOC.
“A lot of it has to do with what are the goals when management set up to fund the SOC, right? Maybe the compliance SOC versus the SOC that’s focused on the security outcomes on defending, right? There are some organizations that are funding for basic compliance,” Chris said. [If the] law says we have to do this, we’re doing that. We’re not really going to invest in your training and your understanding and your comprehension. We’re not going to hire really great analysts. We’re just going to buy the tools that we need to buy. We’re going to buy some people to look at monitors and that’s kind of the end of it.”
One of the easiest and telling methods of assessing where an SOC sees itself in this tale is having conversations with staff. Chris recommends asking staff if they feel aligned with management and do they feel empowered?
“If you feel like you’re being turned into a robot and you pick stuff from here and drop it over there, you’re probably in a place where management doesn’t really support you. Because they’re not using the human being’s capability of synthesis of information and that notion of driving consensus and making things work,” Chris said. “They’re looking more for people who are replaceable to put the bits in the bucket and move through.”
Chris shared other survey takeaways including how SOCs gauge their value, metrics and tools.
SOC INDICATORS AND PERCEIVED VALUE
The survey included hypotheses designed to measure how organizations classify the value of a SOC:
- Budget – The majority of respondents did not list budget as a sign of how their organization value them
- Skilled Staff – Many valued the hiring of skilled workers as a sign of support for their SOC.
- Automation and Orchestration – The SOC teams that believed their organizations already supported through the hiring skilled staff reported their biggest challenge was implementing the automation and orchestration.
“This showed that as SOC teams met the challenge of skilled staffing, they moved on to their next order of task: Let’s make the computers compute well,” Chris said.
SOC METRICS
Ismael asked about the tendency for some SOC management not to report any metrics, and those that simply reported number of incidents not reporting the right metrics. Chris reported that most people said they do provide metrics, but a still–surprising number of people said that they don’t provide metrics at all.
Here’s the breakdown of how respondents answered, “Do you provide metrics to your management?”
- Yes – 69
- No – 24
- We don’t know – 6
That roughly a third of respondents either do not report metrics or don’t know if they report metrics was telling to the survey’s author.
“In which case [metrics] obviously don’t have a central place of importance for your SOC,” Chris said.
Regarding the most frequently used metric – number of incidents – Chris speculated that several SOCs he surveyed are attempting to meet a metric goal of zero incidents, even if it means they’re likely not getting a true reading of their cyber security effectiveness.
“You’re allowed to have zero incidents in the environment. And if you consistently meet that then you’re consistently doing a great job,” Chris said. “Which is insane to me, right? Because we want to have the right number of incidents. If you actually have a cyber security problem … you should want to know about it, okay?
Among the group of respondents who said their most common metric is informational, the desired information from their “zero incidents” metrics doesn’t actually have much bearing on the performance or the value of what the SOC is doing.
“The metrics tend to be focused on what can we easily show as opposed to what truly depicts the value that the SOC has been providing for the org,” Chris said. “And at that point you have something you can show to get more funding and more support right over time.”
Chris suggests better use of metrics can truly depict the value that the SOC is providing the organization and justify the desired support it seeks.
“One which I like, which is not an easy metric to develop is actually loss prevention. If I can actually depict quantitatively, which it will not be precise, there will be some speculation in that,” Chris said. “But if I can depict quantitatively what the SOC did this month, or quarter where our efforts actually prevented or intervened in things which were going wrong and we stopped damage that’s loss prevention, right? That’s what the SOC is there for, right? If I just report, we had 13 incidents there’s not a lot of demonstration of value in that. And so always the metrics tend to be focused on what can we easily show as opposed to what truly depicts the value that the SOC has been providing for the org. “
SOC TOOLS
Michael steered the discussion to the value discussion around incident metrics and their relationship with SOC capacity. How many incidents can you handle? Is it a tools issue or a people issue or a combination of both? Chris’ study also revealed a subset of tools that respondents more frequently leveraged and added value to delivery of higher capacity of incident closure.
One question on the survey asked, “Do you use it?
“Not whether you like it or not, but do you use it? And do you use it in a way where you have full coverage or partial coverage? Because another thing about technology, and this is kind of a dirty secret in technology applications, is a lot of people buy it but actually never get it deployed fully,” Chris said.
His survey allowed respondents to reveal their most-used technologies and to grade tools.
The most common used technologies reported in the survey were:
- SIEM
- Malware Protection Systems
- Next-gen Firewall
- VPN
- Log management
Tools receiving the most A grades:
- EDR
- VPN
- Host-based Malware Protection
- SIEM
- Network Distributed Denial of Service
Tools receiving the most F grades:
- Full Peak App
- Network-Based Application Control
- Artificial Intelligence
- TLS Intercept
Chris pointed out that the reasoning behind the F grades may be less a case of failing and more a case of not meeting their full potential.
“Some of these are newer in this space and some of them just feel like they’re failures for people” Chris said. “Now, whether they’re technology failures or not this is what people are reporting that they don’t like in terms of the tech.”
For more findings read or download Chris Crowley’s 2020 survey here.
Watch this entire episode of SOCwise below.