How do I use Diskover with Wasabi?
Wasabi has been validated for use with Diskover Data. Diskover is a web-based data management solution providing a global view of data fragmented across cloud and on-premise storage, with integrated search, analytics, workflows, and actions. To learn more about the Diskover-Wasabi solution, please refer to our joint solution brief.
1. Reference Architecture
2. Prerequisites
Diskover Professional, Enterprise, Media Edition, and Life Science Edition support native S3 scanning/indexing
A license/annual subscription for Diskover Professional, Enterprise, Media Edition, or Life Science Edition is needed to unlock S3 scanner features and communicate with Wasabi
An active Wasabi account
Wasabi Bucket - See our Bucket Creation Guide
Access & Secret Key Pair - See our Access Keys Guide
3. Configuration
The indexing of Wasabi cloud storage can be configured both via a command line terminal and the Diskover web user interface. The command line is typically used for one-time and/or first-time scans, while the Diskover web user interface is the preferred method to set up indexing tasks at scheduled intervals.
Indexing via a terminal session
3.1. In a terminal session, add credentials to the default location for Wasabi S3 credentials:
cd /root/.aws
vi credentialsExample:
3.2. To use a Wasabi S3 endpoint, set the AWS_PROFILE and the S3_ENDPOINT_URL
environment variables before running the crawl. Type the following commands to export the variables via a terminal session.
export AWS_PROFILE=wasabi-eu
export S3_ENDPOINT_URL=https://
cd /opt/diskover
python3 diskover.py --altscanner scandir_s3 s3://Note: For information on the Wasabi service URLs, please refer to our Wasabi Service URLs article .
Indexing via the Diskover web user interface
3.3. Log in to the Diskover web interface. To add a Wasabi S3 endpoint via the Diskover web task panel, select "gear icon > Task Panel".
3.4. In the Task List panel, select "New Index Task" to start a new index. To modify an existing index, click Info in line with an existing index, then "Edit task".
3.5. Go down to "Environment Vars" and insert your endpoint in the dialog box for the task, for example:
AWS_PROFILE=wasabi-west,S3_ENDPOINT_URL=https://s3.us-west-1.wasabisys.com
Note: This config example discusses the use of Wasabi's us-west-1 storage region. To use other Wasabi storage regions, please refer to our Wasabi Service URLs article .
3.6. To further configure indexing scheduled tasks, please follow these instructions.
Note: For further Diskover installation instructions, click here or access their complete online resources catalog .
4. Operations Summary
Diskover is a data management platform allowing you to “connect” all your storage repositories, either they are located on-prem or in the cloud. Diskover is non-proprietary; we do not store your files; we simply index their metadata. Therefore, end-users have access to read-only index of files and not the files themselves, hence assuring the safety of your source files and production network.
Diskover offers integrated:
To learn more, please refer to Diskover user guide.