Survivor's Guide to SIEM in 2024
Survivor's Guide to SIEM in 2024
How an ownership mindset can help you navigate big changes in the SOC landscape
This isn’t another post about the morning of May 15, when IBM Qradar customers learned that the company was exiting the SIEM business. They could start from scratch with Palo Alto Networks’ XSIAM product or something else entirely. You don’t have to go home, but you can’t stay here.
Industry pundits have been discussing the news ad nauseam, so I won’t rehash why these products hit a wall and whether those were savvy business deals. Instead, I present how an ownership mindset can prepare your organization for what comes next.
One: Own Your Pipelines
So far, the biggest progress towards breaking lock-in and taking ownership has been around data pipelines. In the past, data collection tooling was seen as an integral part of SIEM. Splunk has Universal Forwarders, Securonix has Remote Ingestion Nodes, etc. Typical enterprise deployments include thousands of agents installed on servers and networks throughout the organization.
The many-to-one approach became less popular as demand grew for multiple pipeline destinations, including cloud storage and data science platforms. Cribl (founded by former Splunk product leaders) built a huge following by pointing out that SIEM vendors charging by volume might lack the motivation to add robust volume reduction features to their pipeline products. The business case was made to separate the pipeline from the SIEM, and many security organizations successfully took ownership of their pipeline.
Running thousands of vendor-specific agents across servers, clusters, and networks is a huge source of lock-in. We should all feel for Qradar customers, who have significant on-prem environments and depend on agents like “IBM® WinCollect 10” for their pipeline. As I described in my previous post on security data fabric, there are plenty of open-source or independent alternatives available now.
The important thing is that your team can send data from any source to any destination they choose- including multiple parallel destinations. This opens up effective evaluation and migration options as the SIEM landscape twists and turns.
Two: Own Your Data
One thing is having the flexibility to ship logs to your destinations of choice. Owning the data is something else, and it’s often overlooked, especially when it’s all in the cloud. What’s the difference between storing your data in a cloud-based security platform like Exabeam or a cloud-based data platform like Snowflake?
Your level of data ownership depends on the freedom to:
Query the data directly from third-party tools
Use the data for data science and ML model training
Easily export data to an open format such as Parquet or JSON
Set your own retention policies
Create views on the data for easy access by different users and applications
Govern who can access data based on granular role-based access controls
Audit how the data has been accessed and modified
Since most security platforms don’t give you open access to their data backend, your level of data ownership is significantly limited. This is especially true for the XDR vendors that built their SIEM offering around EDR, where data ownership was never on the table. As a result, a replatform project would involve wholesale export and import of many terabytes- an effort that they’re incentivized to discourage.
Cloud data platforms like Snowflake were designed for interoperability, with dozens of tools for ETL, BI, and data science expected to be plugged in. Many Snowflake customers use multiple data visualization products, for example, and can easily try new ones. The flexibility and level of ownership demonstrated below are becoming available to security organizations, promising to enable more “plug-and-play” security operations.
Three: Own Your Detections
Security operations teams spend an incredible amount of energy on detection engineering. Haider Dost wrote a great article about the seven stages his team goes through in their detection development lifecycle. The thought and effort required to work through these stages are necessary to achieve high-fidelity detections in an enterprise environment.
SIEM vendors tend to underplay the investment required to preserve detection investments during migration. The Google Chronicle team, for example, released a post titled Migrate Off That Old SIEM Already! where they paid lip service to this challenge, writing:
Don't migrate all content. Migrating all of your existing detection content, rules, alerts, dashboards, and visualizations to a new SIEM is a lot of work and it's not always necessary. Take the time to evaluate your current detection coverage and only migrate the rules that you need for your new environment.
That misses the point! While some detections might no longer be required, any SIEM migration strategy must account for the hundreds or thousands of threat detections that must be ported to the new platform.
This is an even bigger challenge when moving between platforms with different capabilities, where migration must account for both syntax and functionality. To pick on the SIEM formerly known as Chronicle (they’ve rebranded to SecOps), GCP posted this month on newly added support for statistical search using “Count Distinct.” Detection engineers building rules in Splunk have used aggregate statistics for years. This begs the question: Do the XDR next-gen SIEMs support this function yet? What other functionality is not supported by the newcomers?
The safest approach is investing in detection rules not tied to a particular platform. Standard languages like SQL and Python will always have broad platform support and include analytics functions that cover everything that the detection engineering team needs. Already, several leading SIEM solutions support SQL, and the trend from proprietary, vendor-specific rule languages to general-purpose analytics languages enables detection ownership. This may be the biggest factor in minimizing the impact of an unexpected SIEM migration situation.
Four: Own Your Requirements
When platform vendors spend millions to buy out their competition, they count on customers having fuzzy, loosely defined requirements. The fallout is experienced only later when security operations teams scramble to work around limitations in their new tooling. I’ve seen cases where this happens mid-migration, forcing the whole project to revert after months of effort. The fallout includes breach risk, team burnout, and compliance issues.
With all the shakeups in the SIEM market, it’s prudent to proactively capture detailed SIEM requirements based on what’s working in the current deployment. In large organizations, these requirements span multiple teams and use cases. Documenting the capabilities and performance expectations around custom ingest, activity analytics, incident investigation, health monitoring, and maturity metrics is an extensive project. But an up-front investment in owning your requirements enables fast reactions and confident decision-making if your SIEM of choice changes hands or goes under.
The hits will keep on coming. Unlike some segments of cybersecurity, the SIEM space in 2024 is still far from stabilizing on a handful of safe choices. Your organization can develop an ownership mindset by seeing its pipelines, data, content, and requirements as independent and modular. Owning your future can help you survive and thrive through these crazy times.