This is the (very) long awaited update to my top 10 mistakes in Security Operations. Unfortunately my Website has migrated between different CMS platforms and different hosting providers and previous comments have been lost in the process.
Vendors sell products, that's what they do.
Every year at RSA, InfoSec Europe, Black Hat and the plethora of other conferences that are starting up, full of vendors pitching the latest security products that are the Silver Bullet to all the ills you're currently suffering, vendors are making concerted efforts to part you from your hard-earnt infosec budgets. Slick demos, based on highly selective inputs and well crafted pitches, all try and convince your organisation to part with it's cash. I know, I was a part of this well-oiled machine for over a decade. It's not like the infosec vendor community is bad, will all had a belief that we were making the World better - and to a certain degree we were - but the reality is that technology, on it's own - is not going to solve anything. It's an old maxim, but you need people, process and technology in order to be successful.
Last year saw the rise of Machine Learning and Analytics as being positioned as delivering a panacea of end-to-end automation, by trawling through your log files in an automated way creating actionable alerts without human intervention. Those of us who've been using analytics in infosec for over a decade acknowledge that analytics can improve the automation of a cyber security operations centre, but it can't replace it. Besides, we've seen these kind of claims before. In 2001, Security Information & Event Management (SIEM) products came to the market and were sold in the same way - aggregate all your logs into this one platform and magic will happen. Fifteen years on and the reality is that most organisations aren't even doing SIEM right today.
As my ex-colleague Anton Chuvakin, now a senior analyst for Gartner in SIEM points out, thinking organisations who have failed to implement log management and SIEM well - and lord knows I can tell you most haven't - can suddenly make something as complicated as big data analytics successful is naive.
The reality is that many organisation's don't have a shortage of security products, or even people, they are unable to deliver quality value-for-money services because they lack:
- Having a risk-based approach that identifies what risks, and the relative impacts of each, that the business needs to address to bring risk down to an acceptable level.
The concept of "security" should really be replaced with that of "risk". In this day-and-age, risk is a part of conducting business and C-level executives are experts in making decisions based on the balance of risk. The challenge is that many in information "security" live within a little technical bubble where the grey area of risk acceptance doesn't gel well with their technology binary view of either something is "secure" or it isn't. Many who've floated to the top in infosec started as firewall or Intrusion Detection System administrators, where either something is permitted to pass or not, or is an attack, or isn't. The concept of accepting certain information risk, because of cost of mitigation exceeds the value mitigated, isn't something that gels well with someone who's thinking is conditioned that way.
C-level executives don't live in such a discrete world and are used to seeing quantitive analysis of the risk and the options for dealing with it, that includes acceptance. I've seen many people in the information security industry bemoan that they've "failed" when they the business does not act as they recommended when they have presented a plan that includes a inherent risk,; the costs and plan for treating the risk with a control; and the remaining residual risk after the control has mitigated the risk. As long as they've done a good job in preparation, they've done their job. At the same time the CISO is pitching for funds for the infosec budget, ehe Sales Manager will be presenting plans for expansion into a new territory; the CIO plans to move to a cloud service provider or Facilities to upgrade the catering area - you're just another Subject Matter Expert presenting plans for an Executive Board to weigh against other activities competing for the limited budgets and attention a business can allocate. In our narrow silo of information security, it's hard to sometime see the broader picture.
I remember one particular instance when I was making a pitch for a £500,000 investment that would mitigate around £1M in risk - a good investment you might think. I thought I'd done a very good job until my request was turned down. In speaking to the CFO later, it turns out that the same £500,000 could have funded 5 additional sales executives who were, on average, bringing in £200,000 in revenue each. With our high customer retention rate would result in many millions of operational cash flow from subscription revenue being available to the business without the need to dig into our roadway from investors. I'd also produced quite a lot of empirical data - imagine if a CISO had come to the table without any firm of quantitative, or even quantitative data, and had made that pitch versus a sales manager who can demonstrate the per capita average value of each salesperson?
Information security needs to articulate risk in the same manner, using the same language and the same metrics, as the other aspects of operational risk - including fraud, safety, disasters, market risk and competitive risk.
Senior executives will make decisions based on a balance of risk in each of these areas, if they can't comprehend the value proposition you're presenting - or understand the technical terms you're using - someone else's budget is going to benefit from your inability show the impact of what you're proposing.
- Properly integrated solutions, based on the products with the appropriate skills and processes - people, process AND technology. Deploying a product isn't the end of a journey, it's the start of it. In fact, the preparations should start long before the courier turns up at the door - planning the integration of the technology with incumbent platforms; ensuring the right skills exist within the business to support the technology; the right processes are in place to maintain the technology; and performance metrics are collected to prove it's business value. The vendor has little or no context of your true business operations - they may support hundred, thousands or even millions of customers in many different verticals.
The vendor may give you a normalised blueprint of what good looks like, but it's the customer's infosec team that are ultimately responsible for turning that into an architecture, integration plan and operational model that will product the organisation not just at implementation time, but also on an ongoing basis as adversaries adapt and new vulnerabilities are found.
All too often vendor's products are deployed and you can go back and check on how often the architecture or content has been updated, or even reviewed, and the last time may actually have been when the professional services team from that vendor first deployed the solution.
- A framework for integrating the different products into a end-to-end framework. Those os us who have lived in the worlds of ISO/IEC 27002, NIST, COBIT or ISF are used to an integrated framework of controls that complement each other and integrate with one-another. If a preventative here fails to stop an attack, it can be detected by a detective control here and responded to using a responsive control there.
Even if an organisation is utilising aa control framework, often the integration and feedback loops between the different controls are missing. For instance, if your Security Operations Centre analysts are detecting attacks at an internal reconnaissance stage, are they feedback back the control failures that allowed the exploit in in the first place?
Integrated frameworks aren't just about a jigsaw of related controls, it's having the right processes and swimlanes around the controls to make them work properly in that integrated fashion.
No-one in infosec should see a demo from a vendor without subjecting what they're seeing to some form of critical analysis. How would be manage this product? How is it updated, but in terms of the platform itself but also it's detection capability? How does it integrate with other products we use? Does it need another UI to manage? Who is going to maintain the platform? How are alerts from the platform managed? What is the process for tuning the platform's logic? What skills does this platform need to manage it? Does the platform introduce any form of privacy considerations?