Using the Datalake for Incident Detection

Understanding the Datalake

The Armor data lake is a centralized repository for storing Armor collected data. With regards to vulnerabilities, the data lake contains all the data for every report created for an environment and all the historical data from when the reports are run. This can be a lot of data so narrowing down the scope of information is critical to making sense of it all.

Accessing the Datalake

Option 1: Compliance in Nexus

In Nexus, hover over Threat Management
Click Log Search, then the Log Search tab
This opens ChaosSearch in a new window.
Click on the Single Sign On button.
Click Next again on the next page to sign in to ChaosSearch.
Once the page loads the following will show:
Note that there are two filters already being applied based on which control was open when Advanced Query was selected. The ruleId and ReportId.
To see the complete report, click on the X next to the rule.Id and now the filter is only using the ReportId to get data.
1. Keeping the rule.Id can also be useful for comparing changes over time (using a wider date range) for that rule.
Changing the date range allows for viewing a single or multiple runs of the report depending on the goal.

Data Presentation

Data consists of documents stored in the datalake. Each document contains all the data related to that particular rule and resource. Below are examples of the table and JSON views:

Table Example

JSON Example

The schema for these documents is based on Elastic Common Schema, please refer to the below links for the details and explanation of the fields:

Vulnerability schema - https://www.elastic.co/guide/en/ecs/1.5/ecs-vulnerability.html

Custom Fields:

src_ip - the event's source IP
src_port - the event's source port
parsed.trendmicro.name - the name of the event signature
dst_ip - the event's destination IP
dst_port - the event's destination port
parsed.trendmicro.severity - the severity of the event

Helpful Fields for Searching the Datalake

Field	Filter By
hostname	the hostname of the machine on which the event was sent
data_type	the type of the data being searched for, trend-hids in this instance

Adding a Filter

To add additional filters, click on the Add Filter Button.

Then set the field to one of the helpful fields above, select the operator, put in the value and hit save. The data is now filtered on a specific reportId, rPolicy or other field selected.

Viewing Datalake Aggregations

Please refer to Reports for custom aggregations, visualizations and custom reports.