Cloudflare Zone-scoped Logs

Topics Discussed

You can use this document to send Cloudflare Zone-scoped logs to Armor's Security Information & Event Management (SIEM) using Logpush.

Cloudflare Logpush is able to push logs of Cloudflare's datasets to AWS S3 in batches.

Logpush is available to customers on Cloudflare's Enterprise plan.

Logpush will generally deliver batches of logs to within 5 minutes, depending on the volume of logs. Each batch of logs will have no more than 100,000 events — so the more events there are for a zone the more frequent the logs will be pushed.

In order to provide ingestion of Cloudflare Zone-scoped HTTP requests and Firewall events datasets you will need to add Logpush jobs to your zone(s) with our destination_conf.

Pre-deployment Considerations

In order to enable this log type, you must have:

• An AWS account

• A Cloudflare zone with an Enterprise plan

• Access to manage Cloudflare Zone logging

• Access to create a new Logpush job to the designated AWS S3 bucket

Setup

1. Determine the AWS Account that provides IAM Roles

2. Add a policy that provides access to the S3 bucket in our destination_conf

3. Attach the policy to a role that will be setting up Logpush

4. Identify the Cloudflare Zone that you would like to forward logs for

5. Identify which datasets (out of http_requests and firewall_events) you would like to push to Armor

6. Create a Cloudflare API Token with permissions to Logs Write

7. Assume the AWS Role that provides access to our S3 bucket

8. Set the environment variable CLOUDFLARE_API_TOKEN to the value of your Cloudflare API Token and run the cloudflare-logpush.sh script to configure Cloudflare to forward logs for the dataset

9. Download the cloudflare-logpush.sh file

   http_requests example

   BASH

   export CLOUDFLARE_API_TOKEN=abcd PARTNER_ID=65535 TENANT_ID=65536 DATASET=http_requests DOMAIN_NAME=example.com ZONE_ID=ABCDEF0123456789;
   ./cloudflare-logpush.sh $PARTNER_ID $TENANT_ID "$DATASET" "$DOMAIN_NAME" "$ZONE_ID"

   
   

   firewall_events example

   BASH

   export CLOUDFLARE_API_TOKEN=abcd PARTNER_ID=65535 TENANT_ID=65536 DATASET=firewall_events DOMAIN_NAME=example.com ZONE_ID=ABCDEF0123456789;
   ./cloudflare-logpush.sh $PARTNER_ID $TENANT_ID "$DATASET" "$DOMAIN_NAME" "$ZONE_ID"

10. Only setup the datasets that you require

11. You may be limited in the number of Logpush Jobs that you can configure. Work with your Cloudflare representative if you run into limitations with the number of Logpush jobs.

Supporting Vendor Documentation

https://www.cloudflare.com/plans/enterprise/

• https://developers.cloudflare.com/logs/about/

• https://developers.cloudflare.com/logs/get-started/enable-destinations/aws-s3/

• https://developers.cloudflare.com/logs/tutorials/parsing-json-log-data/

• https://developers.cloudflare.com/logs/reference/log-fields/zone/firewall_events/

• https://developers.cloudflare.com/logs/reference/log-fields/zone/http_requests/

• https://developers.cloudflare.com/api/tokens/create

• https://developers.cloudflare.com/api/tokens/create/permissions#zone-permissions

Configuration Details

Your partner account will be assigned an AWS S3 bucket that is used by Armor. This S3 bucket will provide policy permitting both Cloudflare and the AWS principal(s) of your choice access. Cloudflare will be permitted to create objects within the bucket, within a specified prefix. Your AWS principal will be permitted to list and get objects within the specified prefix.

When approved for access to this log source we will request the AWS Principal that you would like to provide access to. This will typically be an AWS Organizational account, a dedicated AWS Account that provides IAM users, or an AWS Account that provides your own integration services.

S3 Bucket: prod-xdr-cloudflare-${partner_id}

S3 Prefix: ${partner_id}

Cloudflare Logpush Destination_conf

When providing the destination_conf to the Cloudflare Logpush job, Armor requires a specific path to be configured in order to properly associate the ingested data with your customer and your partner account. The destination_conf will consist of the Armor-leveraged S3 bucket, your partner id, your customer's tenant id, the Cloudflare dataset name, and the date.

In addition to the bucket name we also specify the bucket region and configure object encryption.

Example destination_conf: s3://prod-xdr-cloudflare-65535/65535/65536/http_requests/{DATE}

Example: s3://prod-xdr-cloudflare-65535/65535/65536/http_requests/20220101/object.txt

Where:

• PartnerID = 65535

• PartnerCustomerID = 65536

• Cloudflare dataset = http_requests

• DATE = 20220101

• Cloudflare object = object.txt

Cloudflare Logpush Logpull_options

When configuring the Logpush job we enable all fields available for the dataset and request timestamps to be in the rfc3339 format.

Example logpull_options: fields=ClientIP, ClientRequestMethod, ClientSrcPort, EdgeResponseStatus, EdgeStartTimestamp, WAFAction, ZoneID, ZoneName&timestamps=rfc3339

Log Fields

Dataset http_requests

Dataset firewall_events