Splunk to Ingest Wasabi Bucket Logs

How do I use Splunk to ingest Wasabi Bucket Logs?

Splunk Enterprise can be used to ingest Wasabi bucket logs to see S3 events affecting your bucket's data, such as when an object is uploaded, deleted, or downloaded.  This requires the use of Wasabi Simple Notification Service (SNS) event notifications being sent to an AWS SNS topic, an AWS Simple Queue Service (SQS) queue subscription to that SNS topic, and Splunk polling that SQS queue.  

Prerequisites

• An active Wasabi Cloud Storage account
• Access to the Wasabi Console as the account's root user or a sub-user with WasabiFullAccess permissions
• AWS console access
• An AWS account with SNS and SQS
• An AWS IAM user with SNS and SQS permissions
• AWS access and secret keys 
• Splunk Enterprise software.  This solutions was tested on version 9.4.0.
• Splunk Add-On for AWS.

High-Level Configuration Steps

1.  Create an AWS SNS topic.
2.  Create an AWS SQS Dead-letter queue.
3.  Create an AWS SQS queue and subscribe to the SNS topic.
4.  Configure Splunk to poll the AWS SQS queue.
5.  Configure your AWS account in the Wasabi console with AWS access and secret keys.
6.  Configure your Wasabi bucket to send SNS events to the AWS SNS topic.
7.  Test the Wasabi event types and observe them in Splunk.

Create an AWS SNS Topic

1.  Login to your AWS account via the AWS console.
2.  In the top right of the console, select the region you wish to create the SNS topic (and subsequent SQS queues) in.  For our testing, we selected us-east-1.  
3.  Ensure your AWS IAM user or group has the necessary SNS and SQS permissions.  See Change permissions for an IAM userfor details.  This solution was tested with the following permissions.
   1.  AmazonSNSFullAccess
   2.  AmazonSQSFullAccess
4. Navigate to Amazon SNS > Topics > Create topic. Select the Standard type and give the SNS topic a name.
5.  Leave all the other settings at their defaults.  Scroll down and click on Create topic.

Create an AWS SQS Dead-letter Queue

A dead-letter queue is where undeliverable messages will be sent.  

1. Find the ARN of the IAM user or group being used.   The ARN can be found under "Identity and Access Management (IAM)" > Users.  Search for the name or your IAM user, click the user name.  Copy the ARN and save it in secure location.
2. Navigate to Amazon SQS > Queues > Create queue.
3.  Select the Standard type and give the queue a name.
4. Change Encryption to Disabled.  This was set to disabled for uniformity and reduced complexity during our testing. 
5. Under Access Policy, select Basic as the type.  For "Define who can send messages to the queue" and "Define who can receive messages from the queue," select "Only the specified AWS accounts, IAM users and roles."  Paste the ARN copied above into both of these fields.     
6. Leave "Dead-letter queue" disabled.  Scroll down and click Create queue.

Create an AWS SQS Queue

1.  Navigate to Amazon SQS > Queues > Create queue.
2.  Select the Standard type and give the queue a name.
3.  Under Configuration, change the Visibility timeout to 10 minutes.
4.  Change Encryption to Disabled.  This was changed to disabled for uniformity during our testing due to the complexity of SNS requiring a custom KMS key, whereas SQS allows a Server Side Encryption (SSE) key to be used.
   See Setting up Amazon SNS topic encryption with encrypted Amazon SQS queue subscription if you wish to enable encryption in your SNS topic, and then in your SQS queue, set Encryption to Enabled and select either encryption key type depending on your organization's policies.  
5. Under Access Policy, select Basic as the type. For "Define who can send messages to the queue" and "Define who can receive messages from the queue," select "Only the specified AWS accounts, IAM users and roles." Paste the ARN copied above into both of these fields.  
   
6. Enable the Dead-letter queue and select the previously created Dead-letter queue from the drop-down menu.   
7. Scroll down and click on Create queue.
8.  Select the Subscription region associated with your SNS topic and click Subscribe to Amazon SNS topic.
9.  Select the previously created SNS topic from the drop-down menu and click Save.

Configure Splunk to Poll the AWS SQS Queue

1.  Navigate to the Splunk Add-On for AWS app in your Splunk console.  On our Splunk Linux server, this was located at localhost:8000.  
2.  Click Configuration, then Account, then Add.
3.  Enter the following information for your AWS account:
   1.  A name for the account
   2.  AWS access key
   3.  AWS secret key
   4.  The region category.  This is typically "Global."  Click Add.
4.  Your AWS account will now be listed in the Account section of the Configuration.  Click Inputs.
5.  Click Create New Input, then click Custom Data Type from the drop-down menu.
6.  Click SQS.
7.  Enter the following information about your SQS queue, then click Add.
   1.  A name for the queue.
   2.  Select your AWS account from the drop-down menu.
   3.  Select the AWS region your SQS queue is located in from the drop-down menu.
   4.  Select the SQS queue from the menu.
8.  Your new input will now appear on the Inputs screen.  

Configure Your AWS Account in the Wasabi Console

1.  Login to the Wasabi Console as the root user.
2.  Navigate to Settings and click on Event Notifications.  Click Configure New Credentials.
3.  Select the AWS region your SNS topic is located in from the drop-down menu.  Enter your AWS access and secret keys, then click Add Credentials.
4.  Your AWS credentials entry will now show in the Wasabi Console.

Configure Your Wasabi Bucket to send SNS Events

1.  Navigate to Buckets and click the name of the bucket you wish to monitor. 
2.  Click on the Settings gear wheel on the right.
3. Under Properties, click Event Notifications. 
4.  Click Create Event Notification.
5.  Give the event a name and click Next.
6.  Enable the events you wish to receive notifications about and click Next.
7.  Select the AWS region and previously created SNS topic in that region and click Create.

Test Wasabi Event Types and Observe Them in Splunk

1.  In the Splunk Add-on for AWS console, click Search.
2.  Perform actions such as a test upload, download, and deletion of an object to your Wasabi bucket using an S3 client such as Cyberduck, Wasabi Explorer, the Wasabi Console, etc.  Your S3 events will show up on the Splunk Search screen.  You may need to place your cursor in the Search text box and hit enter.  Some example fields of interest are circled in the following diagram.
3.  You can also search for "Wasabi" in the search field to narrow down the search results.