Rule Tuning with Watchlists
Overview
Required Reading
There are a number of ways to achieve tuning where specific data sets are concerned, this document will assume you’ve already read through the documentation on creating a watchlist
If you are into deploying your own changes, we will assume that you’ve read through where to store new watchlists for automated deployments.
We recommend you read through the above referenced documentation if you wish to tune rules specific to your environment. Tuning made outside of the repos may not be recoverable unless it’s stored in your repo.
Use Cases
Allow Lists
If an incident or alert continuously triggers on known good activity or that which is not considered a risk then you can have a watchlist applied that can be queried by the rule to prevent triggering by omitting results that match.
For example if you have a database instance or server that should only receive connections from a set of known good IPs, then you might create a watchlist for those known good IPs called “db_exclusions” with a single column with the header “dpips” of those IPs, one per line. Then you can reference this list within a rule that might be triggering false positive alerts or incidents every time a connection is made from these IPs to tell it to exclude those IPs from triggering results.
While you may have a firewall rule or network security group preventing unauthorized connections, a layered approach to security by implementing this type of rule would be ideal should the firewall fail open, or a misconfiguration of network rules leaves your database open to would-be attackers.
Block Lists
If an incident or alert is required to trigger on specific activity or particular users, IPs, and/or systems that are considered risky then you can have a watchlist applied that can be queried by the rule to trigger it upon matching.
For example if you have a set of criteria that could pose a problem which consists of certain commands being run by any user that is not an authorized admin, then you might want to implement a watchlist with a set of hosts where this command could be run in one column and a set of authorized admins in another column, or even another watchlist. Then these could be used as a monitoring mechanism for for anytime a command in the list is used by anyone who doesn’t match the authorized admins then an alert or incident will be triggered.
Groups or Sets
If you need certain rules to apply to a set of similar critera, you can create a watchlist for a set of hosts - like databases, or users (administrators), IP address space (public or private), file types, directory structure, specific commands, and much more provided that data point is surfaced in the results from the rule logic.
Event Enrichment
If you want to add information into incidents you can do it through a watchlist. For example, if an event doesn’t surface a hostname or it’s compliance status or any other interesting factors, you could make a watchlist with the IP in one column and use it to look up the additional information in other columns associated with that IP - such as a hostname, compliance status, OS, role, department, or managing entity. There is no limit to the types of data you could tie into an event from a watchlist.
One Watchlist or Many?
You can have a single watchlist with multiple columns, or multiple watchlists with a single column, or a combination of both. In Azure the limit of the number of watchlists is 10 million.
Watchlist Storage
- A watchlist can be a csv file that you upload, these are limited to 3.8MB in size
- You can use an Azure storage account for your watchlist data, the limit is 500MB
- It is not recommended to use the watchlist to store large data volumes, it’s purpose is only for reference data sets
Watchlist Standards
When you upload your watchlist you will need to provide some key information, this is covered in the create a watchlist documentation. However, we want to make sure you understand how this information is used.
When the Watchlist Wizards asks you for the Name, Description, and Alias on the General tab, ensure you are using concise characterization of the data.
Name
- The Name should be specific to the purpose of the data
- The value must be between 3 and 64 characters long
- It should start with an alpha character and the rest may contain alphanumeric, hyphen and underscore characters
- The value must not be empty
Alias
- The Alias can be the same as the Name or different
- For example “AuthorizedUsers” as a Name could be “authorized_users” as an Alias, or they can both be “AuthorizedUsers”.
- The Alias is what will be used in the query to reference the dataset.
- Whatever you choose, ensure you standardize on that convention, or it will create confusion later.
- The value must be between 3 and 64 characters long
- It should start with an alpha character and the rest may contain alphanumeric, hyphen and underscore characters.
- The value must not be empty.
Create a Watchlist in 5 Steps:
Example CSV:
- Save the watchlist in comma delimited format from a plain text file, from excel, or export from Google Docs as a csv.
- Using the Watchlist Wizard in Sentinel. From Sentinel, choose Watchlists in the lower left navigation pane and select “+ Add new”.
- Give it a “Name”, “Description”, and “Alias” and click “Next: Source”
- Adding the CSV to the Watchlist Wizard. Simply drag the csv onto the shaded area, or open a file explorer window to browse to it and select it. Click “Next: Review and Create”
- Review and create the watchlist. Select the searchkey in the highlighted drop down. This should be the column name you saved in the CSV. If your CSV has multiple columns, choose the column you intend your query to search to do lookups on. Click “Create”
Using a Watchlist in a Query:
An example query with a watchlist to prevent triggering on certain IPs:
let threshold = 15;
let allowlist = (_GetWatchlist('ip-allowlist') | project ipv4);
Syslog
| where ProcessName =~ "sshd"
| where SyslogMessage contains "Failed password for invalid user"
| parse kind=relaxed SyslogMessage with * "invalid user" user " from " ip " port" port " ssh2"
| project user, ip, port, SyslogMessage, EventTime
| summarize EventTimes = make_list(EventTime), PerHourCount = count() by ip, bin(EventTime, 4h), user
| where PerHourCount > threshold
| mvexpand EventTimes
| extend EventTimes = tostring(EventTimes)
| summarize StartTimeUtc = min(EventTimes), EndTimeUtc = max(EventTimes), UserList = makeset(user), sum(PerHourCount) by IPAddress = ip
| extend UserList = tostring(UserList)
| extend timestamp = StartTimeUtc, IPCustomEntity = IPAddress, AccountCustomEntity = UserList
| where IPAddress !in (allowlist)If the allowlist line in the query had been written with the “Name” instead:
let allowlist = (_GetWatchlist('IP-Allowlist') | project ipv4);it will result in the following error:
'project' operator: Failed to resolve scalar expression named 'ipv4'.The column name is what is referenced in the query to match against as the searchkey that you pick in the “Source” tab of the Watchlist Wizard.
Notice the placement of the where clause that does a negative match on the allow list (watchlist). We place it near the end so that the final results after all the processing can filter out our allowed IPs.
Changes to make the query trigger on a set of IPs:
If we wanted to make a block list then all that would need to change is the name and alias from “allowlist”, and of course a set of IPs to block, and be sure to remove the “!” from the where clause to positively match our watchlist.
| where IPAddress in (blocklist)An example query with a watchlist to enrich incidents:
In this example we are looking for accounts without MFA enabled, if the mfa in use is not provided by Azure but some other provider or method that isn’t tracked well within a login event from a device, then we’ll need to have a logic app that queries for users with active MFA if we don’t have a way to consume login events from the MFA provider. That logic app could then populate a watchlist that consists of active and inactive accounts and whether MFA is enabled, as well as other identifying data, such as department, managing party, etc.
For an example on updating watchlists with logic apps, please see Microsoft’s example Update-Watchlist-With-NamedLocation
You would trigger an incident off of an alert that seems suspicious, maybe a copier account was used to log into a resource. The query of the incident logic should include the watchlist parameters below:
let userdata = (_GetWatchlist('user-enrichment') | project TargetAccount);
SecurityAlert
| where AccountName in (userdata)
| lookup kind=inner _GetWatchlist('user-enrichment') on $left.AccountName == $right.SearchKey
| where MFA == "no"The resulting hits can surface the data from the watchlist in the incident to help your SOC team members understand the risk and who to reach out to in order to start the incident response process. This is just one of many ways to enrich incident data.
We hope this helps increase your knowledge on tuning rules using watchlists.