SoDA With Wasabi

How do I use SoDA with Wasabi?

Wasabi has been validated for use with SoDASoDA’s data management software is storage agnostic, providing unprecedented insight into a company’s storage ecosystem. To learn more about the Wasabi - SoDA solution, please refer to our joint solution brief. To use this product with Wasabi, please follow the instructions below.

Reference Architecture

Screen_Shot_2022-03-29_at_9.31.19_PM.png

Prerequisites

  • SoDA v 22.1.1

  • An active Wasabi account

Configuration

Adding Wasabi Storage

1. Log in to the SoDA User Interface.

2. Click Storage in the left-hand menu.

Screen_Shot_2022-03-29_at_9.37.02_PM.png

3. Click Price Books.

Screen_Shot_2022-03-29_at_9.38.12_PM.png

4. Click the Add Price Book button.

Screen_Shot_2022-03-29_at_9.39.06_PM.png

5. Fill out the Add Price Book page as shown below, checking for any updated pricing from Wasabi. Click the Save button to finalize the updates.

Screen_Shot_2022-03-29_at_9.39.59_PM.png

6. Click Storage.

Screen_Shot_2022-03-29_at_9.41.21_PM.png

7. Click Add Storage.

Screen_Shot_2022-03-29_at_9.42.45_PM.png

8. Fill out the page as shown below. If you want to use a prefix (or folder), you can specify one. Use your own SoDA Storage Name and the bucket you created in Wasabi.

Screen_Shot_2022-03-29_at_9.43.59_PM.png

9. Click Save to finalize your updates. After clicking Save, the corresponding Storage card view appears.

Screen_Shot_2022-03-29_at_9.45.00_PM.png

10. Click Agents in the left-hand menu.

Screen_Shot_2022-03-29_at_9.45.45_PM.png

11. Click Accessors.

Screen_Shot_2022-03-29_at_9.46.40_PM.png

12. Click Add Accessor.

Screen_Shot_2022-03-29_at_9.47.19_PM.png

13. Enter the fields as shown below.

  • Provide a name for your access method.

  • Select Accessor Type Custom S3.

  • Enter the appropriate URL for your Wasabi region in the Api URL field. 

  • Use the Access Key ID and Secret Access Key for your bucket that Wasabi supplied to you.

  • Click the Configure Accessor button to finalize your updates.

Note: Each Agent can have a different access key if desired.

Screen_Shot_2022-03-29_at_9.48.43_PM.png

Note: This config example discusses the use of Wasabi's us-west-1 storage region. To use other Wasabi storage regions, please use the appropriate Wasabi service URL as described in this article.

14. Your newly-created Accessor is then listed.

Screen_Shot_2022-03-29_at_9.50.11_PM.png

15. Click Agents.

Screen_Shot_2022-03-29_at_9.50.59_PM.png

16. Click on any Agent that you want to access the Wasabi bucket.

Screen_Shot_2022-03-29_at_9.51.40_PM.png

17. Click the Attach Storage button.

Screen_Shot_2022-03-29_at_9.52.30_PM.png

18. Select the Wasabi storage you created, and your accessor, as shown below.

Screen_Shot_2022-03-29_at_9.53.18_PM.png

19. Click the Attach button, confirm that it is now attached, as shown below, and close the window.

Screen_Shot_2022-03-29_at_9.54.08_PM.png

20. Click Storage in the left-hand menu.

Screen_Shot_2022-03-29_at_9.56.05_PM.png

21. Click on your Wasabi storage.

Screen_Shot_2022-03-29_at_9.56.54_PM.png

22. Click on the 3 dots in the top-right to open the additional options. Click Enable.

Screen_Shot_2022-03-29_at_9.57.59_PM.png

23. Your Wasabi bucket is now online, as indicated by the blue icon.

Screen_Shot_2022-03-29_at_9.58.48_PM.png

Adding Local Storage

Before creating file storage, SoDA offers the ability to create a price book on file-based storage. Follow the instructions in the Wasabi Price Book but select a local price book. Once finished follow the instructions below to add local storage.

24. Click Storage in the left-hand menu.

Screen_Shot_2022-04-01_at_12.32.27_PM.png

25. Click the Add Storage button.

Screen_Shot_2022-04-01_at_12.33.21_PM.png

26. Give your Storage a name and select File as the Storage Type. If you configured a local price book, select it here and click Save to finalize the updates.

Screen_Shot_2022-04-01_at_12.34.17_PM.png

27. Your new storage appears, as shown below.

Screen_Shot_2022-04-01_at_12.35.23_PM.png

28. Click Agents in the left-hand menu.

Screen_Shot_2022-04-01_at_12.36.06_PM.png

29. Click Accessors.

Screen_Shot_2022-04-01_at_12.36.55_PM.png

30. Click the Add Accessor button.

Screen_Shot_2022-04-01_at_12.37.42_PM.png

31. Enter the fields as shown below:

  1. Give your access method a name.

    • Select Local as the Accessor Type.

    • Enter the appropriate mount path in the Path field

    • For Linux, use the root path.

    • For Windows, use the Volume letter and folder “V:\Files”.

Screen_Shot_2022-04-01_at_12.38.58_PM.png

32. Click the Configure Accessor button to create the Accessor, and your new Accessor appears in the list as shown below.

Screen_Shot_2022-04-01_at_12.39.58_PM.png

33. Click Agents.

Screen_Shot_2022-04-01_at_12.40.41_PM.png

34. Click on any Agent you want to access the local file mount.

Screen_Shot_2022-04-01_at_12.41.28_PM.png

35. Click the Attach Storage button.

Screen_Shot_2022-04-01_at_12.42.41_PM.png

36. Select the File Storage you created, and your Accessor, as shown below, and click the Attach button.

Screen_Shot_2022-04-01_at_12.43.20_PM.png

37. When the Attach Agent to Storage screen closes, you see the newly attached Agent listed as attached in the previous Agent display, as shown below and close the window.

Screen_Shot_2022-04-01_at_12.44.10_PM.png

38. Click Storage in the left-hand menu.

Screen_Shot_2022-04-01_at_12.45.22_PM.png

39. Click on your “My Local Storage” storage card.

Screen_Shot_2022-04-01_at_12.46.19_PM.png

40. Click on the 3 dots in the top-right to open additional options. Click Enable.

Screen_Shot_2022-04-01_at_12.47.54_PM.png

41. Your file-based storage is now online, as indicated by the blue icon.

Screen_Shot_2022-04-01_at_12.48.52_PM.png

Moving Data into Wasabi

You can move data in and out of Wasabi using SoDA by establishing a Policy or by using a one-time Quick Transfer. To do so using a Quick Transfer, follow the procedure below, or proceed to the Moving Data via Policy procedure to do so that way.

Moving Data via Quick Transfer

42. Click Jobs in the left-hand menu.

image.png

43. Click the Quick Transfer button.

Screen_Shot_2022-04-05_at_10.14.39_PM.png

44. On the Quick Transfer Page, do the following:

  • Select your Source as “My Local Storage”.

  • Select your Target as “My Wasabi Bucket”.

  • Select Copy/Move/Sync in the drop-down between the Source and Target fields to indicate how to transfer the data.

    • Copy and Move use conflict handling of Overwrite/Rename/Skip for identical files already on the Target.

    • Sync makes a Smart Duplicate of the Source to the Target, including propagating deletes.

  • Click on the right of the Source to select the folders to transfer.

  • Click on the right of the Target to select folders to create or to put data into.

  • Select Filters to filter the data set as desired.

    • Once ready, click the Dry Run button, which simulates the transfer and calculates the cost and time of the transfer, without actually executing the transfer.

Screen_Shot_2022-04-05_at_10.17.48_PM.png

45. The results of the dry run display, as shown below.

Screen_Shot_2022-04-05_at_10.18.31_PM.png

46. Determine if the dry run is successful. If so, proceed to the next step, otherwise, reconfigure your Quick Transfer by returning to step 44 above.

47. When satisfied with the dry run results, click the Transfer Now button at the top-right of the page to begin the transfer.

Screen_Shot_2022-04-05_at_10.19.58_PM.png

48. The Jobs page displays, where you can see the current job process, as shown below.

Screen_Shot_2022-04-05_at_10.20.43_PM.png

49. Click on the job record to display more information.

Screen_Shot_2022-04-05_at_10.21.41_PM.png

50. Log in to the Wasabi UI to verify data is there.

Screen_Shot_2022-04-05_at_10.22.25_PM.png

Moving Data via Policy

51. Click Policies in the left-hand menu.

Screen_Shot_2022-04-06_at_9.12.52_AM.png

52. Click the Add Policy button.

Screen_Shot_2022-04-06_at_9.13.43_AM.png

53. On the Policy page, update the following elements:

  • Give the Policy a name.

    • Select your Source as “My Local Storage”.

    • Select your Target as “My Wasabi Bucket”.

    • Select Copy/Move/Sync in the drop-down between the Source and Target fields to indicate how to transfer the data.

      • Copy and Move use conflict handling of Overwrite/Rename/Skip for identical files already on the Target.

      • Sync makes a Smart Duplicate of the Source to the Target, including propagating deletes.

    • Click on the right of the Source to select the folders to transfer.

    • Click on the right of the Target to select folders to create or to put data into.

    • Select Filters to filter the Data Set as desired.

54. Once ready, click the Dry Run button, which simulates the transfer and calculates the cost and time of the transfer before it happens, without actually executing the transfer.

55. Determine if the dry run is successful. If so, proceed to the next step, otherwise, reconfigure your Policy by returning to step 53 above.

56. Click the Deploy button.

Screen_Shot_2022-04-06_at_9.22.20_AM.png

57. The Deploy Policy window opens. Fill out the fields accordingly and click the Deploy button.

  1. Select the desired start time.

    • Select the cadence at which the policy should run.

    • Activate the policy.

Screen_Shot_2022-04-06_at_9.24.03_AM.png

58. The Deploy Policy window closes, and you can see the new policy configured on the Policies page.

Screen_Shot_2022-04-06_at_9.24.52_AM.png