Wasabi has been validated with DataLakeHouse.io, a leading cloud data synchronization and business intelligence orchestration platform that enables data teams to build single-source-of-truth repositories and data models with fast time-to-value solutions.
1. Requirements
Active Wasabi Cloud Storage Account
Wasabi Bucket - See our Bucket Creation Guide
Access & Secret Key Pair - See our Access Keys Guide
DataLakeHouse.io Latest Release (>=v0.1)
DataLakeHouse.io Enterprise Edition
2. Reference Architecture
3. Configuration
3.1 Create an account at DataLakeHouse.io and make sure to verify your email address to activate your account.
3.2 Log in to your DataLakeHouse.io account at the portal login.
3.3 On the Dashboard click + Add a Source.
3.4 Search for Wasabi and then click Add New Source.
3.5 Follow the instructions and enter the required information about your Wasabi Connection:
Name - Alias for this connection, unique from any other connections you created or will create
Target Schema Prefix - Prefix for the schema at the target you will sync to
Bucket - Enter the bucket name where your files are stored
Region - Select the region where your bucket is stored
Folder Path - is a path on the root bucket from where the desired files will be retrieved
Access Key - Enter your Wasabi Access Key credentials
Secret Key - Enter your Wasabi Secret Key credentials
3.6 Click the Save & Test button once all the data has been entered.
3.7 Next, you will create a Cloud Data Warehouse Target. Click Targets under Connections.
3.8 Click + Add New Target under the desired Data Warehouse vendor.
Note - Snowflake Setup/Config Instructions below:
3.9 Click the Save & Test button after inputting all the required information.
3.10 Click Sync Bridges.
3.11 Click + New Sync Bridge.
3.12 Input the required information for your Sync Bridge Settings:
Sync Bridge Name - Enter a unique name for your Sync Bridge
Select Connections - Select your Source and Target Connections that you created from each Dropdown
Sync Time Zone - Reflects when your data should load. All times are ultimately converted to UTC.
Sync Frequency - Select a Sync Frequency at which your data will synchronize. The lowest frequency for non-Enterprise Plan customers is 15 min, depending on the amount of data needing to be replicated, which is source-dependent. By selecting 12 or 24 hours, you will be prompted with the option to set the start time of the sync, but this is optional.
If the Apply Start Time? appears (optional), by checking the checkbox, you will be able to see a Sync Start Time dropdown
If a Manual Sync option is available for your plan, the data will still be synchronized on a 24-hour period from when you save the Sync Bridge.
3.13 Click Save Sync Bridge, and the Sync will start at the next scheduled time.
3.14 Your Sync Bridges page will now reflect your newly created Bridge and should look similar to the following.
3.15 While in the Sync Bridges page, click the Actions menu to the right of where your Sync Bridge listing appears. Click Run Sync Bridge Right Now.
Note - This will start the process to synchronize your Wasabi bucket files into the cloud data warehouse connection
3.16 Once the Sync Bridge process is complete, you can visit your Cloud Data Warehouse Database to see your data in the respective database tables ready for consumption.