What I’ve Learned as a Part-Time Cyber Threat Analyst Using Anomali Enterprise

Big data + big intelligence = big challenges

A few months ago I wrote a post detailing how Anomali Enterprise helped me to identify a malware threat to my home network. Many have since emailed me asking how they can do the same (please keep them coming!).

Since writing that post my router has generated millions of logs that have been ingested by Anomali Enterprise (thankfully still no major threats). As a new “threat analyst” for my families home network I’ve learned a number of things along the way, especially the challenges and frustrations when it comes to performing security investigations.

In the interest of sharing my knowledge to the community I wanted to highlight a few things I’ve come up against, and the what I’ve found most useful.

New threats <= active threats?

Knowing a new threat has been observed is good. Knowing where a threat is in the Kill Chain can is much more useful. By giving you knowledge of where the threat is in the process of achieving it’s objective allows you to not only defend against it, but understand the context of activity of the threat prior to it becoming known.

Threat intelligence products are great at identifying threats as they happen. For example, Anomali ThreatStream integrations with SIEM products — Arcsight, QRadar, or Splunk to name a but few — can identify log data against threat intel on new logs flowing into each product.

However, this only answers the first of the three questions I’d want to ask as an analyst once a threat has been identified;

1. Is our network impacted/compromised? What’s our exposure?
2. How widespread is the impact? How far back does it go?
3. Which specific assets are impacted?

As threats, by their very nature, are reported after-the-fact, there can often be a delay, sometimes weeks, before it is shared. When a threat is identified, it is vitally important to know what its behaviour and what it has potentially breached in the days it was left unreported.

Big data, big numbers

Considering the data from my home network alone from the previous blog posts the calculations required lead to some big numbers:

100,000 logs per day x 1 year of data x 10 indicators = 365,000,000

That’s three hundred sixty-five million calculations that need to be performed for just one investigation!

At enterprise scale the 0’s dramatically increase:

1 billion logs per day x 365 days x 3 years of data = 10 trillion (10,000,000,000,000) matches need to be performed, for one investigation!

Existing security log repositories (I’m using Splunk) are not designed to process queries against such large volumes of historic data. Not only are they limited by the ability to process archived data but often the cost of storing such data means much of it is filtered, and thus impossible to forensically search against.

How Anomali Enterprise helped me (answer questions 2 & 3)

It was not just me suffering some of these pains, our own security team here at Anomali experienced these problems day-in-day-out. In search of a solution we built Anomali Enterprise. Some of the functional and design goals of the product included:

1. The ability to store years of log data online even from highly noisy sources e.g DNS traffic — trillions of logs (without filtering what gets stored due to costs)
2. The ability to analyse these logs against millions of threat indicators in seconds — not minutes, hours, days, or even weeks (both in real-time and retrospectively)
3. The ability for analysts to be more effective, more efficient, and more accurate in detecting and remeditating threats (better worflows for threat intel)

It’s all about time-to-resolution

Analysts want to focus on the most serious threats, not more threats in their already never-ending workload. Anomali Enterprise helps me to do this by automatically comparing threat indicators — domains, URLs, emails, file-hashes etc. — against new and historic data from all devices in my home network. I can see what has been compromised, when it was comprimised and if the threat made any lateral movement. Within an hour of malware being identified (as in the previous post), I can assess the damage, detected affected assets, and take measures to secure them.

Learn more

This post covers Anomali Enterprise’s real-time and forensic capabilities. It can do much more. Check out the Anomali Enterprise product page on our site to find out more about what it can do.

Originally published at www.anomali.com.