Contents x
      Flexify.io With Wasabi
      • 03 Oct 2024
      • 6 Minutes to read
      • Print
      • PDF
      Contents

      Flexify.io With Wasabi

      • Updated on 03 Oct 2024
      • 6 Minutes to read
      • Print
      • PDF
      Article summary

      How do I use Flexify.io with Wasabi?

      Flexify.io (Flexify) is multi-cloud storage and zero-downtime migration solution that allows migrating data to Wasabi from other clouds or on-premise file storage, migrating data between Wasabi buckets and regions, and duplicating or distributing data across multiple clouds or buckets. Flexify.IO is available both as a cloud-hosted service and free host-yourself Community Edition. To use Flexify.IO with Wasabi, please follow the instructions below.

      How do I use Flexify.IO to migrate data from another cloud to Wasabi?

      1. Create an account with Flexify.IO (faster) or install Flexify.IO Community Edition (free).

      2. In Flexify.IO Management Console switch to the Data tab and click Add Storage.

      3. Select Wasabi from the list of providers, copy-paste Wasabi access keys and click Add Storage. You can get Wasabi access keys here. 

      mceclip1.png

      1. Click Add Storage again and select your previous storage provider. Flexify.IO supports Amazon S3, Microsoft Azure, Google Cloud Storage DigitalOcean Spaces and many other cloud storages. 

      2. Enter Secret Key and Access key that you can get on your previous provider’s portal and add the storage.

      3. Click Transfer Data.

      4. Chose mode (Copy or Move), buckets you want to migrate from, and Wasabi as destination. This will automatically use or create buckets with the original names. Alternatively, you can select an any existing Wasabi bucket.

      mceclip2.png

      1. Optionally, click Advanced settings and fine tune incremental migration options, conflict resolution, performance settings, regions, or define a pattern to migrate only some of the objects.

      2. Click Start Migration and monitor the migration progress. If you use a cloud-hosted service, you will get an email when migration competes. 

      mceclip4.png

       

      How do I use Flexify.IO to migrate data from on-premise or self-hosted file storage to Wasabi?

      1. If not already installed, download and install Docker on your file server.

      2. Run Minio container pointing it to your files (/mnt/storage directory in this example).

      $ docker run -p 9000:9000 \
  -e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" \
  -e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \
  -v /mnt/storage:/data \
  minio/minio server /data

      Hint: you may consider pointing Minio to a directory one level above your data directory in order to present all the files and directories as a single bucket.

      1. Open http://:9000, sign in with the keys you used to start Minio and check that your data is visible in Minio Browser.

      2. Create an account with Flexify.IO or install Flexify.IO Community Edition.

      3. In Flexify.IO Management Console switch to the Data tab, click Add Storage and select Minio from the providers list.

      4. Specify server’s IP address as Endpoint and copy-paste keys you specified when starting Minio. Make sure to uncheck “Encrypt transfer with SSL” checkbox (or configure SSL on the server).

      mceclip5.png

      1. Add the storage and click Add Storage again to add your Wasabi account.

      2. Select Wasabi from the list, copy-paste keys from Wasabi’s Access Keys page and add the storage.      

      mceclip9.png

      1. Click Transfer Data to open the Migrate data page.

      2. Select Minio in the From field and Wasabi in the To field. Alternatively, you can expand storage accounts and select individual bucket to avoid bucket names conflicting with other Wasabi buckets.

      mceclip0.png

      1. Optionally, click Advanced settings and fine-tune the migration.

      2. Click Start Migration and monitor the migration progress.

      mceclip3.png

       

      How do I use Flexify.IO to migrate data between Wasabi buckets or regions?

      1. Create an account with Flexify.IO (faster) or install IO Community Edition (free).

      2. In the Flexify.IO Management Console switch to the Data tab and click Add Storage.

      3. Select Wasabi from the list of providers, copy-paste keys from Wasabi portal and add the storage.

      mceclip4.png

      1. Click at Wasabi storage account to see the list of Wasabi buckets along with and their regions.

      mceclip5.png

      1. Click Transfer Data.

      2. Select mode (Copy or Move), source bucket and destination bucket.

      mceclip6.png

      1. Optionally click Advanced settings and fine-tune the migration.

      2. Click Start Migration and monitor the progress.

      mceclip7.png

       

      How do I use Flexify.IO to duplicate or distribute data across multiple buckets or build multi-cloud storage?

      Flexify.IO provides a virtual stateless S3-compatible endpoint that combines data from multiple storages into a single unified namespace.

      This walkthrough demonstrates how to transparently duplicate data between Amazon S3 and Wasabi, with extra data from Azure Blob Storage, all accessible via a single S3 endpoint.

      1. Create an account with Flexify.IO (faster) or install Flexify.IO Community Edition (free).

      2. In the Flexify.IO Management Console switch to the Data tab.

      3. Click Add Storage, select Amazon S3, copy-paste S3 keys and add the storage.

      4. Click Add Storage again, select Wasabi, copy-paste Wasabi keys and add the storage.

      5. Click Add Storage one more time, select Microsoft Azure. copy-paste Azure’s storage account name and secret key and add the storage.

      6. By now you should have three storage accounts in the list. Click on a storage account to see a list of buckets.

      mceclip8.png

      1. Click on the Endpoint card, select Amazon S3 storage account from the list and click Attach. 

      mceclip9.png

      1. Repeat the operation for Wasabi storage account.

      2. Repeat the operation for Azure storage account, but this time make sure to uncheck “Put new data to this storage account”. 

      mceclip10.png

      1. Click on Settings heading to see endpoint address and keys. 

      mceclip11.png

      1. Configure your application to use Flexify.IO endpoint and keys.

      For example, if you use AWS CLI, you can type:

      $ aws configure set aws_access_key_id h8I8TiKWaOq87OSaXyb
      $ aws configure set aws_secret_access_key snFuLMl2rm9uU4SeNo8oeO46RP0Rbs
      $ aws s3 --endpoint-url https://s3.flexify.io ls

      Note that the list of buckets containes bucket from all three attached clouds.

      1. Create a new bucket. Make sure to choose a unuqie bucket name that does not exist in any of the attached clouds.

      $ aws s3api --endpoint-url https://s3.flexify.io \
      create-bucket --bucket flexify-demo

      Hint: To map bucket names or use non-unique names concider configured a Virtual Bucket by clicking New Virtual Bucket. 

      1. Put an object to the new bucket.

      $ echo 'demo' |
      aws s3 --endpoint-url https://s3.flexify.io \
      cp - s3://flexify-demo/demo.txt

      1. Verify that the bucket and the object are duplicated to both Wasabi and Amazon S3 (but not to Azure).     

      Screen_Shot_2020-03-27_at_5.10.21_PM.png

      1. In the Azure Portal manually create Flexify-demo container and upload any object there.

      mceclip15.png

      1. List objects though Flexify.IO. Note that the result contains both objects from Wasabi/Amazon and Azure.

      $ aws s3 --endpoint-url https://s3.flexify.io ls s3://flexify-demo
      2019-08-02 09:33:34 5 demo.txt
      2019-08-02 09:48:03 8470 logo.png

      1. Download object though Flexify.IO.

      $ aws s3 --endpoint-url https://s3.flexify.io cp s3://flexify-demo/logo.png.
      download: s3://flexify-demo/logo.png to ./logo.png

      We have configured a multi-cloud virtual storage that combines and reads data from Wasabi, Amazon S3 and Azure, and duplicates new data between Wasabi and Amazon S3.

      How do I avoid downtime when migrating data with Flexify.IO?

      Downtime is necessary when data being migrated is actively changing at the same time. There are two ways to avoid downtime during migration with Flexify.IO:

      Minimized Downtime

      1. Migrate all data “as is”, even though it’s changing in the source.

      2. Pause updates, for example by stopping the application.

      3. Run the same migration again in incremental mode. All Flexify.IO migrations are incremental by default.

      4. Reconfigure the application to point to the migration destination and restart the application.

      5. Delete data from the source.

      The incremental migration analyzes the differences and only copies changes, thus significantly reducing migration duration and required downtime.

      Zero Downtime

      1. Configure IO virtual endpoint to combine and duplicate data in both the source and the destination.

      2. Re-configure the application to point to the Flexify.IO virtual endpoint.

      3. Run the migration normally in either copy or move mode.

      4. Re-configure the application to point to the destination.

      5. Disable Flexify.IO virtual endpoint.

      6. If you used copy migration mode, delete data from the source.

      At any time, your application will see data in both the source and the destination and will not need to be brought offline. You can configure updates to be duplicated to both the source and the destination, or to be saved to the destination only.

      How to install Flexify.IO Community Edition?

      Flexify.IO Community Edition is a free host-yourself tool limited to a single machine. It’s recommended for smaller projects and proof-of-concepts.

      1. Provision a machine with at least 2 GB RAM. Flexify.IO will not start if you have less than 2 GB.

      2. Install Docker.

      3. Start Flexify.IO Community Edition.

        # docker run -d -p 80:80 -p 443:443 flexifyio/ce 

      4. Open http:// (or http://10.0.75.1 if you use Docker on Windows), wait for Flexify.IO CE to start, create an account and configure Flexify.IO.

      For increased bandwidth capacity and reliability, consider using Flexify.IO Standard/Enterprise Edition or Flexify.IO cloud-hosted service.

      What's Next
      Copyright © 2024 by Wasabi Technologies LLC
      75 Arlington Street, Suite 810, Boston, MA 02116 USA
      All Rights Reserved
      Visit us at https://wasabi.com.
      WASABI and the WASABI logo are trademarks of Wasabi Technologies LLC and may not be used without permission of Wasabi Technologies. All other names are used for identification purposes only and are trademarks or registered trademarks of their respective companies.

      Information presented herein is subject to change without notice. Companies, names, and data used in examples are fictitious unless otherwise noted. No part of this information may be reproduced or transmitted in any form by means electronic or mechanical, for any purpose, without express written permission of Wasabi Technologies.