SoDA With Wasabi
    • 20 Jun 2024
    • 5 Minutes to read
    • PDF

    SoDA With Wasabi

    • PDF

    Article summary

    How do I use SoDA with Wasabi?

    Wasabi has been validated for use with SoDASoDA’s data management software is storage agnostic, providing unprecedented insight into a company’s storage ecosystem. To learn more about the Wasabi - SoDA solution, please refer to our joint solution brief. To use this product with Wasabi, please follow the instructions below.

    Reference Architecture

    Screen_Shot_2022-03-29_at_9.31.19_PM.png

    Prerequisites

    • SoDA v 22.1.1

    • An active Wasabi account

    Configuration

    Adding Wasabi Storage

    1. Log in to the SoDA User Interface.

    2. Click Storage in the left-hand menu.

    Screen_Shot_2022-03-29_at_9.37.02_PM.png

    3. Click Price Books.

    Screen_Shot_2022-03-29_at_9.38.12_PM.png

    4. Click the Add Price Book button.

    Screen_Shot_2022-03-29_at_9.39.06_PM.png

    5. Fill out the Add Price Book page as shown below, checking for any updated pricing from Wasabi. Click the Save button to finalize the updates.

    Screen_Shot_2022-03-29_at_9.39.59_PM.png

    6. Click Storage.

    Screen_Shot_2022-03-29_at_9.41.21_PM.png

    7. Click Add Storage.

    Screen_Shot_2022-03-29_at_9.42.45_PM.png

    8. Fill out the page as shown below. If you want to use a prefix (or folder), you can specify one. Use your own SoDA Storage Name and the bucket you created in Wasabi.

    Screen_Shot_2022-03-29_at_9.43.59_PM.png

    9. Click Save to finalize your updates. After clicking Save, the corresponding Storage card view appears.

    Screen_Shot_2022-03-29_at_9.45.00_PM.png

    10. Click Agents in the left-hand menu.

    Screen_Shot_2022-03-29_at_9.45.45_PM.png

    11. Click Accessors.

    Screen_Shot_2022-03-29_at_9.46.40_PM.png

    12. Click Add Accessor.

    Screen_Shot_2022-03-29_at_9.47.19_PM.png

    13. Enter the fields as shown below.

    • Provide a name for your access method.

    • Select Accessor Type Custom S3.

    • Enter the appropriate URL for your Wasabi region in the Api URL field. 

    • Use the Access Key ID and Secret Access Key for your bucket that Wasabi supplied to you.

    • Click the Configure Accessor button to finalize your updates.

    Note: Each Agent can have a different access key if desired.

    Screen_Shot_2022-03-29_at_9.48.43_PM.png

    Note: This config example discusses the use of Wasabi's us-west-1 storage region. To use other Wasabi storage regions, please use the appropriate Wasabi service URL as described in this article.

    14. Your newly-created Accessor is then listed.

    Screen_Shot_2022-03-29_at_9.50.11_PM.png

    15. Click Agents.

    Screen_Shot_2022-03-29_at_9.50.59_PM.png

    16. Click on any Agent that you want to access the Wasabi bucket.

    Screen_Shot_2022-03-29_at_9.51.40_PM.png

    17. Click the Attach Storage button.

    Screen_Shot_2022-03-29_at_9.52.30_PM.png

    18. Select the Wasabi storage you created, and your accessor, as shown below.

    Screen_Shot_2022-03-29_at_9.53.18_PM.png

    19. Click the Attach button, confirm that it is now attached, as shown below, and close the window.

    Screen_Shot_2022-03-29_at_9.54.08_PM.png

    20. Click Storage in the left-hand menu.

    Screen_Shot_2022-03-29_at_9.56.05_PM.png

    21. Click on your Wasabi storage.

    Screen_Shot_2022-03-29_at_9.56.54_PM.png

    22. Click on the 3 dots in the top-right to open the additional options. Click Enable.

    Screen_Shot_2022-03-29_at_9.57.59_PM.png

    23. Your Wasabi bucket is now online, as indicated by the blue icon.

    Screen_Shot_2022-03-29_at_9.58.48_PM.png

    Adding Local Storage

    Before creating file storage, SoDA offers the ability to create a price book on file-based storage. Follow the instructions in the Wasabi Price Book but select a local price book. Once finished follow the instructions below to add local storage.

    24. Click Storage in the left-hand menu.

    Screen_Shot_2022-04-01_at_12.32.27_PM.png

    25. Click the Add Storage button.

    Screen_Shot_2022-04-01_at_12.33.21_PM.png

    26. Give your Storage a name and select File as the Storage Type. If you configured a local price book, select it here and click Save to finalize the updates.

    Screen_Shot_2022-04-01_at_12.34.17_PM.png

    27. Your new storage appears, as shown below.

    Screen_Shot_2022-04-01_at_12.35.23_PM.png

    28. Click Agents in the left-hand menu.

    Screen_Shot_2022-04-01_at_12.36.06_PM.png

    29. Click Accessors.

    Screen_Shot_2022-04-01_at_12.36.55_PM.png

    30. Click the Add Accessor button.

    Screen_Shot_2022-04-01_at_12.37.42_PM.png

    31. Enter the fields as shown below:

    1. Give your access method a name.

      • Select Local as the Accessor Type.

      • Enter the appropriate mount path in the Path field

      • For Linux, use the root path.

      • For Windows, use the Volume letter and folder “V:\Files”.

    Screen_Shot_2022-04-01_at_12.38.58_PM.png

    32. Click the Configure Accessor button to create the Accessor, and your new Accessor appears in the list as shown below.

    Screen_Shot_2022-04-01_at_12.39.58_PM.png

    33. Click Agents.

    Screen_Shot_2022-04-01_at_12.40.41_PM.png

    34. Click on any Agent you want to access the local file mount.

    Screen_Shot_2022-04-01_at_12.41.28_PM.png

    35. Click the Attach Storage button.

    Screen_Shot_2022-04-01_at_12.42.41_PM.png

    36. Select the File Storage you created, and your Accessor, as shown below, and click the Attach button.

    Screen_Shot_2022-04-01_at_12.43.20_PM.png

    37. When the Attach Agent to Storage screen closes, you see the newly attached Agent listed as attached in the previous Agent display, as shown below and close the window.

    Screen_Shot_2022-04-01_at_12.44.10_PM.png

    38. Click Storage in the left-hand menu.

    Screen_Shot_2022-04-01_at_12.45.22_PM.png

    39. Click on your “My Local Storage” storage card.

    Screen_Shot_2022-04-01_at_12.46.19_PM.png

    40. Click on the 3 dots in the top-right to open additional options. Click Enable.

    Screen_Shot_2022-04-01_at_12.47.54_PM.png

    41. Your file-based storage is now online, as indicated by the blue icon.

    Screen_Shot_2022-04-01_at_12.48.52_PM.png

    Moving Data into Wasabi

    You can move data in and out of Wasabi using SoDA by establishing a Policy or by using a one-time Quick Transfer. To do so using a Quick Transfer, follow the procedure below, or proceed to the Moving Data via Policy procedure to do so that way.

    Moving Data via Quick Transfer

    42. Click Jobs in the left-hand menu.

    image.png

    43. Click the Quick Transfer button.

    Screen_Shot_2022-04-05_at_10.14.39_PM.png

    44. On the Quick Transfer Page, do the following:

    • Select your Source as “My Local Storage”.

    • Select your Target as “My Wasabi Bucket”.

    • Select Copy/Move/Sync in the drop-down between the Source and Target fields to indicate how to transfer the data.

      • Copy and Move use conflict handling of Overwrite/Rename/Skip for identical files already on the Target.

      • Sync makes a Smart Duplicate of the Source to the Target, including propagating deletes.

    • Click on the right of the Source to select the folders to transfer.

    • Click on the right of the Target to select folders to create or to put data into.

    • Select Filters to filter the data set as desired.

      • Once ready, click the Dry Run button, which simulates the transfer and calculates the cost and time of the transfer, without actually executing the transfer.

    Screen_Shot_2022-04-05_at_10.17.48_PM.png

    45. The results of the dry run display, as shown below.

    Screen_Shot_2022-04-05_at_10.18.31_PM.png

    46. Determine if the dry run is successful. If so, proceed to the next step, otherwise, reconfigure your Quick Transfer by returning to step 44 above.

    47. When satisfied with the dry run results, click the Transfer Now button at the top-right of the page to begin the transfer.

    Screen_Shot_2022-04-05_at_10.19.58_PM.png

    48. The Jobs page displays, where you can see the current job process, as shown below.

    Screen_Shot_2022-04-05_at_10.20.43_PM.png

    49. Click on the job record to display more information.

    Screen_Shot_2022-04-05_at_10.21.41_PM.png

    50. Log in to the Wasabi UI to verify data is there.

    Screen_Shot_2022-04-05_at_10.22.25_PM.png

    Moving Data via Policy

    51. Click Policies in the left-hand menu.

    Screen_Shot_2022-04-06_at_9.12.52_AM.png

    52. Click the Add Policy button.

    Screen_Shot_2022-04-06_at_9.13.43_AM.png

    53. On the Policy page, update the following elements:

    • Give the Policy a name.

      • Select your Source as “My Local Storage”.

      • Select your Target as “My Wasabi Bucket”.

      • Select Copy/Move/Sync in the drop-down between the Source and Target fields to indicate how to transfer the data.

        • Copy and Move use conflict handling of Overwrite/Rename/Skip for identical files already on the Target.

        • Sync makes a Smart Duplicate of the Source to the Target, including propagating deletes.

      • Click on the right of the Source to select the folders to transfer.

      • Click on the right of the Target to select folders to create or to put data into.

      • Select Filters to filter the Data Set as desired.

    54. Once ready, click the Dry Run button, which simulates the transfer and calculates the cost and time of the transfer before it happens, without actually executing the transfer.

    55. Determine if the dry run is successful. If so, proceed to the next step, otherwise, reconfigure your Policy by returning to step 53 above.

    56. Click the Deploy button.

    Screen_Shot_2022-04-06_at_9.22.20_AM.png

    57. The Deploy Policy window opens. Fill out the fields accordingly and click the Deploy button.

    1. Select the desired start time.

      • Select the cadence at which the policy should run.

      • Activate the policy.

    Screen_Shot_2022-04-06_at_9.24.03_AM.png

    58. The Deploy Policy window closes, and you can see the new policy configured on the Policies page.

    Screen_Shot_2022-04-06_at_9.24.52_AM.png