Dell Powerscale SmartSync With Wasabi Dell ObjectScale

Dell PowerScale SmartSync is a data replication and mobility solution integrated into Dell’s OneFS operating system. It facilitates efficient and secure data movement between PowerScale clusters and Wasabi cloud object storage. SmartSync is beneficial for organizations aiming to implement hybrid cloud strategies, enhance disaster recovery, or streamline data archiving processes.

Prerequisites

• An active Wasabi with Dell Object Storage Account

• A Wasabi bucket created in the appropriate Wasabi OBS Storage Vault along with a user with available Access/Secret Keys.

• Minimum PowerScale software release 9.11 and above. Check with your Dell representative on the recommended release for SmartSync.

• PowerScale’s external network is publicly routable.

• Port 9021 (HTTPS) for S3 API.

• SyncIQ license for your PowerScale OneFS.

• A Certificate Authority configured on the Cluster.

Reference Architecture Diagram

SmartSync Configuration

To use the SmartSync feature, you will need to activate the SmartSync Daemon, then configure a Certificate Authority on the PowerScale cluster, following the steps documented in the Dell PowerScale SmartSync KB Configuration Encryption section.

If a Certificate Authority is not correctly installed, you will get an error as described in the Troubleshooting Unconfigured CA section.

Enabling the SmartSync (Datamover) Service

After your Certificate Authority is correctly configured, as per Dell’s documentation, enable the Datamover service on your PowerScale cluster.

isi services -a isi_dm_d enable

Creating a SmartSync Certificate Authority for Wasabi

Create an ECS_S3 Account type pointing to your Wasabi ObjectScale bucket.

• Account Type - ECS_S3

• URI - https://s3-nns-us-east-1-dell-obs.wasabisys.com:9021 or https://s3-nns.eu-west-1-dell-obs.wasabisys.com:9021

• Auth Mode - CLOUD

• Access ID - Wasabi Access Key

• Secret Key - Wasabi Secret Key

isi dm accounts create ECS_S3 https://s3-nns.us-east-1-dell-obs.wasabisys.com:9021/(BUCKET NAME) --name (UNIQUE ACCOUNT NAME) --auth-mode CLOUD --access-id (ACCESS KEY) --secret-key (SECRET KEY)

isi dm accounts view (ACCOUNT NUMBER)

Creating Policies

1. Create a Dataset Creation Policy.

   isi dm policies create (UNIQUE POLICY NAME) --priority='NORMAL' --enabled='true' --policy-type='CREATION' --creation-account-id=(LOCAL ACCOUNT ID) --creation-base-path=(SOURCE PATH) --creation-dataset-retention-period=600 --creation-dataset-reserve=2 --creation-dataset-expiry-action=DELETE

   

2. Create a Repeat-Copy Policy.

   isi dm policies create (UNIQUE POLICY NAME) --policy-type='REPEAT_COPY' --priority='NORMAL' --enabled='true' --repeat-copy-source-base-path=(SOURCE PATH) --repeat-copy-base-account-id=(LOCAL ACCOUNT ID) --repeat-copy-base-source-accont-id=(LOCAL ACCOUNT ID) --repeat-copy-base-target-account-id=(CLOUD ACCOUNT ID) --repeat-copy-base-target-base-path=(BUCKET NAME) --repeat-copy-base-target-dataset-type='FILE_ON_OBJECT_BACKUP' --repeat-copy-base-dataset-retention-period=1800 --repeat-copy-base-dataset-reserve=5 --repeat-copy-base-dataset-expiry-action=DELETE

   

3. Enable the Dataset Creation policy to run now to create a dataset/snapshot.

   isi dm policies modify (POLICY NUMBER) --run-now=true
   isi dm jobs list

   

4. Enable the Repeat Copy policy to run now to start a backup job to your Wasabi bucket.

   isi dm policies modify (POLICY NUMBER) --run-now=true
   isi dm jobs list

   

   isi dm jobs list

   

5. You can view the job by running “isi dm jobs view xx” and verifying that the data was transferred.

   isi dm jobs view (JOB NUMBER)

   

   

   When using the Repeat Copy job, the data is transferred to Wasabi ObjectScale storage in proprietary format. Using Cyberduck, S3 Browser, or any other application, you can view your OBS bucket and see that the data is now successfully in your bucket. The data, however, will not be in a readable format, and you will need to use SmartSync to view or restore the data.

   

Running Copy Job Incrementally

After running the initial baseline copy/backup job with the initial dataset, any new jobs will be incremental, comparing the data against the original snapshot in the series and uploading only changed files.

isi dm policies modify (POLICY NUMBER) --run-now=true
isi dm policies modify (POLICY NUMBER) --run-now=true
isi dm jobs list

Run “isi dm reports list” to show all the jobs run and their job type.

isi dm reports list

Restoring SmartSync

Restore data from Cloud to the Powerscale Cluster with either Partial or Full Restoration. Partial Restoration restores a subset of the directory structure specified under the source path. Full restoration is the process of restoring the entire dataset.

Check and view the dataset id for restoration.

isi_dm browse
list-accounts
connect-account (Cloud Account ID)
list-datasets
connect-dataset (Dataset ID)

Creating a Partial Restoration Policy

isi dm policies create (UNIQUE POLICY NAME) --policy-type='COPY' --enabled='true' --priority='NORMAL' --copy-base-target-account-id=(LOCAL ACCOUNT ID) --copy-base-source-account-id=(SOURCE ACCOUNT ID) --copy-source-base-path=(SOURCE PATH) --copy-create-dataset-on-target='false' --copy-base-target-dataset-type='FILE' --copy-base-target-base-path=(TARGET PATH ON CLUSTER) --copy-dataset-id=(DATASET ID FOR RESTORATION) --copy-source-subpaths=(SUB PATH FOR RESTORATION)

1. Run the Partial Restoration Policy.

   isi dm policies list
   isi dm policies modify (RESTORATION POLICY ID) --run-now=true

2. View the Data Restoration Job Status.

   isi dm jobs list

   

3. Once complete, you can validate a successful restore by browsing to that local source path to see that the file(s) are now restored.

Creating a Full Restoration Policy

If you want to restore a full dataset, you will follow the same steps as above for Partial Restore, just removing the Subpath command.

isi dm policies create (UNIQUE POLICY NAME) --policy-type'COPY' --enabled='true' --priority='NORMAL' --copy-base-target-account-id=(LOCAL ACCOUNT ID) --copy-base-source-account-id=(SOURCE ACCOUNT ID) --copy-source-base-path=(SOURCE PATH) --copy-create-dataset-on-target=(TRUE OR FALSE) --copy-base-target-dataset-type='FILE' --copy-base-target-base-path=(TARGET PATH ON CLUSTER) --copy-dataset-id=(DATASET ID FOR RESTORATION) 

1. Run the Partial Restoration Policy.

   isi dm policies list
   isi dm policies modify (RESTORATION POLICY ID) --run-now=true

2. View the Data Restoration Job Status.

   isi dm jobs list

Restoring Remote Browse Datasets

Remote Browsing of datasets allows you to view them and restore the data object by object manually.

1. Run “isi_dm browse” to enter the dataset commands.

2. Run “connect-accounts xxxx” where you input your ECS_S3 account number.

3. Run “list-datasets” to view all datasets created.

4. Run “connect-dataset x” and specify the number associated with the dataset you want to view.

5. Run “ls” to list all the files in that dataset.

   isi_dm browse
   $ connect-account (CLOUD ACCOUNT NUMBER)
   $ list-datasets
   $ connect-dataset (DATASET ID)
   $ ls

   

   For testing purposes, if you were to delete a file from one of your shares.

   

6. Now run the “download” command while connected to the dataset and restore it to the share.

7. Specify the file you want to download, and the path where you want to restore it.

   

   When you go back to your share, you will see the file back in place.

   

SmartSync Expiration Policy

Expiration Policies are needed to clean up expired datasets locally on the cluster (Source DM Account) and on the Target platform (Target DM Account).

isi dm policies create [Expiration Policy Name] --policy-type='EXPIRATION' --enabled='true' --priority='NORMAL' --copy-base-target-account-id=[Local account id] –-expiration-account-id [Source DM Account or Target DM Account] ––recurrence="cron expression" --start-time="YYYY-MM-DD HH:MM:SS"