The next BriefingsDirect inside story examination of security best practices focuses on the building of a global security operations center (SOC) for cyber defense.
Learn here how Zayo Group in Boulder, Colorado built a state-of-the-art SOC as it expanded its international managed security service provider practice.
Hear directly from Mike Vamvakaris, Vice President of Managed Cyber Security at Zayo Group, on the build-out, best practices, and end-results from this impressive project. The moderator is Serge Bertini, Vice President of Sales and General Manager of the Canada Security Division at Hewlett Packard Enterprise (HPE).
Serge Bertini: Mike, you and I have talked many times about the importance of managed security service providers (MSSPs), global SOCs, but for our readers, I want to take them back on the journey that you and I went through to get into the SOC business, and what it took from you to build this up.
So if you could, please describe Zayo’s business and what made you decide to jump into the MSSP field.
Mike Vamvakaris: Thanks for the opportunity. Zayo Group is a global communications and infrastructure provider. We serve more than 365 markets. We have 61 international data centers on-net, off-net, and more than 3,000 employees.
Zayo Canada required a SOC to serve a large government client that required really strict compliance, encryption, and correlational analysis.
Upon further expansion, the SOC we built in Canada became a global SOC, and now it can serve international customers as well. Inside the SOC, you will find things such as US Federal Information Processing Standard (FIPS) 140-2 security standards compliance. We do threat hunting, threat intelligence. We are also doing machine learning, all in a protected facility via five-zone SOC.
This facility was not easy to build; it was a journey, as we have talked about many times in person, Serge.
Bertini: What you guys have built is a state-of-the-art facility. I am seeing how it helps you attract more customers, because not only do you have critical infrastructure in your MSSP, but also you can attract customers whose stringent security and privacy concerns can be met.
Vamvakaris: Zayo is in a unique position now. We have grown the brand aggressively through organic and inorganic activities, and we are able to offer holistic and end-to-end security services to our customers, both via connectivity and non-connectivity.
For example, within our facility, we will have multiple firewalling and distributed denial-of-service (DDoS) technologies -- now all being protected and correlated by our state-of-the-art SOC, as you described. So this is a really exciting and new opportunity that began more than two years ago with what you at HPE have done for us. Now we have the opportunity to turn and pivot what we built here and take that out globally.
Bertini: What made you decide on HPE ArcSight, and what did you see in ArcSight that was able to meet your long-term vision and requirements?
Vamvakaris: That’s a good question. It wasn’t an easy decision. We have talked about this openly and candidly. We did a lot of benchmarking exercises, and obviously selected HPE ArcSight in the end. We looked at everyone, without going into detail. Your listeners will know who they are.
But we needed something that supported multi-tenancy, so the single pane of window view. We are serving multiple customers all over the world, and ArcSight allowed us to scale without applying tremendous amount of capital expenditure (CAPEX) investment and ongoing operational expenditure (OPEX) to support infrastructure and the resources inside the SOC. It was key for me on the business side that the business-case was well supported.
We had a very strict industry regulation in working with a large government customer, to be FIPS-compliant. So out of the box, a lot of the vendors that we were looking at didn’t even meet those requirements.
Another thing I really liked about ArcSight, when we did our benchmarking, is the event log filtration. There really wasn’t anyone else that could actually do the filtration at the throughput and the capacity we needed. So that really lent itself very well. Just making sure that you are getting the salient events and kind of filtering out the noncritical alerts that we still need to be looking at was key for us.
Something that you and I have talked about is the strategic information and operations center (SIOC) service. As a company that knew we needed to build around SOC, to protect our own backbone, and offer those services to our extended connectivity customers, we enlisted SIOC services very early to help us with everything from instant response management, building up the Wiki, even hiring and helping us retain critical skill sets in the SOC.
From an end-to-end perspective, this is why we went with ArcSight and HPE. They offered us a turnkey solution, to really get us something that was running.
The Trifecta: People, Process, Technology
Bertini: In this market, what a lot of our customers see is that their biggest challenge is people. There are a lot of people when it comes to setting up MSSPs. The investment that you made is the big differentiator, because it’s not just the technology, it’s the people and process. When I look at the market and the need in this market, there is a lack of talented people.
How did you build your process and the people? What did you have to do yourself to build the strength of your bench? Later on we can talk a little bit more about Zayo and how HPE can help put all of this together.
Vamvakaris: We were the single tenant, if you will. Ultimately we needed to go international very quickly. So we went from humble beginnings to an international capability. It’s a great story.
For us, you nailed it on the head. SOC, the technology obviously is pertinent, you have to understand your use cases, your policies that you are trying to use and protect your customers with those. We needed something very modular and ArcSight worked for that.
But within the SOC, our customers require things like customized reporting and even customized instant-response plans that are tailored to meet their unique audits or industry regulations. It’s people, process and tools or technology, as they say. I mean, that is the lifeline of your SOC.
One of the things we realized early on, you have to focus on everything from your triage, to instant response, to your kill-chain processes. This is something we have invested significantly in, and this is where we believe we actually add a lot of value to our customers.
Bertini: So it’s not just a logging capability, you guys went way beyond providing just the eyes on the glass to the red team and the tiger team and everything else in between.
Vamvakaris: Let me give you an example. Within the SOC, we have SOC Level 1, all the way to Level 3, and then we have threat hunting. So inside we do threat intelligence. We are now using machine-learning technologies. We have threat hunting, predictive analytics, and we are moving into user behavior analysis
Remember the way I talked about SOC Level 1, Level 2, Level 3, this is a 24x7, 365-day facility. This is a five-zone SOC for enhanced access control, mantraps inside to factor biometric access control. It’s a facility that we are very proud of and that we love showcasing.
Bertini: You are a very modest person, but in the span of two years you have done a lot. You started with probably one of the largest mammoth customers, but one thing that you didn’t really talk about is, you are also drinking your own champagne.
Tell us a little bit more about, Zayo. It’s a large corporation, diverse and global. Tell us about the integration of Zayo into your own SOC, too.
Drinking your own Champagne
Vamvakaris: Customers always ask us about this. We have all kinds of fiber or Ethernet, large super highway customers I call them, massive data connectivity, and Zayo is well-known in the industry for that; obviously one of the leaders.
The interesting part is that we are able to turn and pivot, not only to our customers, but we are also now securing our own assets -- not just the enterprise, but on the backbone.
So you are right, we sip our own champagne. We protect our customers from threats and unauthorized data exfiltration, and we also do that for ourselves. So we are talking about a global multinational backbone environment.
Bertini: That’s pretty neat. What sort of threats are you starting to see in the market and how are you preventing those attacks, or at least how can you be aware in advance of what is coming down the pipe?
Vamvakaris: It’s a perpetual problem. We are invested in what’s called an ethical hacking team, which is the whole white hat/black hat piece.
In practice, we’re trying to -- I won’t say break into networks, but certainly testing the policies, the cyber frameworks that companies think they have, and we go out of our way to make sure that that is actually the case, and we will go back and do an analysis for them.
So where do I see the market going? Well, we see a lot of ransomware; we see a lot of targeted spear phishing. Things are just getting worse, and I always talk about how this is no longer an IT issue, but it’s a business problem.
People now are using very crafty organizational and behavior-style tactics of acquiring identities and mapping them back to individuals in a company. They can have targeted data exfiltration by fooling or tricking users into giving up passwords or access and sign all types of waivers. You hear about this everyday somewhere that someone accidentally clicked on something, and the next thing you know they have wired money across the world to someone.
So we actually see things like that. Obviously we’re very private in terms of where we see them and how we see them, but we protect against those types of scenarios.
Gone are the days where companies are just worried about their customer provided equipment or even cloud firewalls. The analogy I say, Serge, is if you don’t know who is knocking at the door, how are you going to protect yourself, right?
You need to be able to understand who is out there, what they are trying to do, to be able to mitigate that. That’s why I talk about threat hunting and threat intelligence.
Partners in Avoiding Crime
Bertini: I couldn’t agree more with you. To me, what I see is the partnership that we built between Zayo and HPE and that’s a testament of how the business needs to evolve. What we have done is pretty unique in this market, and we truly act as a partner, it’s not a vendor-relationship type of situation.
Can you describe how our SIOC was able to help you get to the next level, because it’s about time-to-market, at the end of the day. Talk about best practices that you have learned, and what you have implemented.
Vamvakaris: We grew out to be an international SOC, and that practice began with one large request for proposal (RFP) customer. So we had a time-to-market issue compressed. We needed to be up and running, and that’s fully turnkey, everything.
When we began this journey, we knew we couldn’t do it ourselves. We selected the technology, we benchmarked that, and we went for the Gartner Magic Quadrant. We were always impressed at HPE ArcSight, over the years, if not a decade, that it’s been in that magic quadrant. That was very impressive for us.
But what really stood out is the HPE SIOC.
We enlisted the SIOC services, essentially the consulting arm of HPE, to help us build out our world-class multizone SOC. That really did help us get to market. In this case, we would have been paying penalties if we weren’t up and running. That did not happen.
The SIOC came in and assessed everything that we talked about earlier, they stress-tested our triage model and instant response plan. They helped us on the kill chain; they helped us with the Wiki. What was really nice and refreshing was that they helped us find talent where our SOC is located. That for me was critical. Frankly, that was a differentiator. No one else was offering those types of services.
Bertini: How is all of this benefitting you at the end of the day? And where do you see the growth in your business coming for the next few years?
Ahead in the Cloud
Vamvakaris: We could not have done this on our own. We are fortunate enough that we have learned so much now in-house.
But we are living in an interconnected world. Like it or not, we are about to automate that world with the Internet of things (IoT), and always-on mobile technologies, and everyone talks about pushing things to the cloud.
The opportunity for us is exciting. I believe in a complete, free, open digital world, which means we are going to need -- for a long time -- to protect the companies as they move their assets to the cloud, and as they continue to do mobile workforce strategies -- and we are excited about that. We get to be a partner in this ecosystem of a new digital era. I think we are just getting started.
The timing then is perfect, it’s exciting, and I think that we are going to see a lot of explosive growth. We have already started to see that, and now I think it’s just going to get even more-and-more exciting as we go on.
Vamvakaris: As anyone out there who has built a SOC knows, you’re only as good as your people, processes, and tools. So we have our tools, we have our processes -- but the people, that cyber security talent is not cheap. The SOC analysts have a tough job. So the more we can automate, and the more we can give them help, the better. A big push now is for AI, which really is machine learning, and automating and creating a baseline of things from which you can create a pattern, if you will, of repeatable incidents, and then understanding that all ahead of time.
We are working with that technology. Obviously HPE ArcSight is the engine to the SOC, for correlational analysis, experience-sampling methods specifically, but outside there are peripherals that tie into that.
It’s not just about having the human capabilities, but it's also augmenting them with the right technologies and tools so they can respond faster, they can get to the issues; they can do a kill chain process quickly. From an OPEX perspective, we can free up the Level 1 and Level 2 talent and move them into the forensic space. That’s really the vision of Zayo.
We are working with technologies including HPE ArcSight to plug into that engine that actually helps us free up the incident-response and move that into forensics. The proactive threat hunting and threat intelligence -- that’s where I see the future for us, and that’s where we’re going.
Bertini: Amazing. Mike, with what you have learned over the last few years, if you had to do this all over again, what would you do differently?
Practice makes perfect
Vamvakaris: I would beg for more time, but I can’t do that. It was tough, it was tough. There were days when we didn’t think we were going to make it. We are very proud and we love showcasing what we built -- it’s an amazing, world-class facility.
But what would I do differently? We probably spent too much time second-guessing ourselves, trying to get everything perfect. Yet it’s never going to be perfect. A SOC is a living, breathing thing -- it's all about the people inside and the processes they use. The technologies work, and getting the right technology, and understanding your use cases and what you are trying to achieve, is key. Not trying to make it perfect and just getting it out there and then being more flexible in making corrections, [that would have been better].
In our case, because it was a large government customer, the regulations that we had to meet, we built that capability the first time, we built this from the ground up properly -- as painful as that was, we can now learn from that.
In hindsight, did we have to have everything perfect? Probably not. Looking back at the compressed schedule, being audited every quarter, that capability has nonetheless put us in a better place for the future.
Bertini: Mike, kudos to you and your team. I have worked with your team for the last two to three years, and what you have done has showed us a miracle. What you built is a top-class MSSP, with some of the most stringent requirements from the government, and it shows.
Now, when you guys talk, when you present to a customer, and when we do joint-calls with the customers -- we are an extension of each other. We at HPE are just feeding you the technology, but how you have implemented it and built it together with your people, process, and technology -- it’s fantastic.
So with that, I really thank you. I'm looking forward to the next few years together, to being successful, and bringing all our customers under your roof.
Vamvakaris: This is the partnership that we talked about. I think that’s probably the most important thing. If you do endeavor to do this, you really do need to bring a partner to the table. HPE helped us scale globally, with cost savings and an accelerated launch. That actually can happen with a world-class partnership. So I also look forward to working with you, and serving both of our customer bases, and bringing this great capability out into the market.