Migrating Data From Google Drive to Wasabi Using Rclone

How to Migrate Data from Google Drive to Wasabi using Rclone?

The Rclone software package is a command-line tool which allows a variety of modes of operation for managing data in two or more locations. Rclone has been certified for use with Wasabi. This article describes how to use Rclone to migrate data from Google Drive to Wasabi. For more information about Rclone, please visit the Rclone project page.

Table of Contents

1. Prerequisites 

• Active Wasabi Cloud Storage Account

• Wasabi Bucket - See our Bucket Creation Guide

• Access & Secret Key Pair - See our Access Keys Guide

• Rclone Executable Version 1.63.1

2. Configure Google Drive as a Remote Location

2.1 Open your OS specific terminal and navigate to the folder location where the Rclone executable is stored. 

2.2 Type in rclone config to create a new Remote

rclone config

2.3 Type in/Select "n" for a new remote connection

No remotes found, make a new one? 
n) New remote 
r) Rename remote 
c) Copy remote 
s) Set configuration password 
q) Quit config
n/r/c/s/q> n

2.4 Enter a name for your google drive remote connection

Enter name for new remote.
name> google drive

2.5 Type in/Select Option "18" for Google Drive

Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
1 / 1Fichier
\ (fichier)
2 / Akamai NetStorage
\ (netstorage)
3 / Alias for an existing remote
\ (alias)
4 / Amazon Drive
\ (amazon cloud drive)
5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, ArvanCloud, Ceph, China Mobile, Cloudflare, GCS, DigitalOcean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, IONOS Cloud, Liara, Lyve Cloud, Minio, Netease, Petabox, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS, Qiniu and Wasabi
\ (s3)
6 / Backblaze B2
\ (b2)
7 / Better checksums for other remotes
\ (hasher)
8 / Box
\ (box)
9 / Cache a remote
\ (cache)
10 / Citrix Sharefile
\ (sharefile)
11 / Combine several remotes into one
\ (combine)
12 / Compress a remote
\ (compress)
13 / Dropbox
\ (dropbox)
14 / Encrypt/Decrypt a remote
\ (crypt)
15 / Enterprise File Fabric
\ (filefabric)
16 / FTP
\ (ftp)
17 / Google Cloud Storage (this is not Google Drive)
\ (google cloud storage)
18 / Google Drive
Storage> 18

2.5 Leave Blank and Hit Enter

Option client_id.
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a value. Press Enter to leave empty.
client_id>

2.6 Leave Blank and Hit Enter

Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret>

2.7 Choose your scope and hit enter

Option scope.
Scope that rclone should use when requesting access from drive.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
1 / Full access all files, excluding Application Data Folder.
\ (drive)
2 / Read-only access to file metadata and file contents.
\ (drive.readonly)
/ Access to files created by rclone only.
3 | These are visible in the drive website.
| File authorization is revoked when the user deauthorizes the app.
\ (drive.file)
/ Allows read and write access to the Application Data folder.
4 | This is not visible in the drive website.
\ (drive.appfolder)
/ Allows read-only access to file metadata but
5 | does not allow any access to read or download file content.
\ (drive.metadata.readonly)
scope> 1

2.8 Leave Blank and Hit Enter

Option service_account_file.
Service Account Credentials JSON file path.
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a value. Press Enter to leave empty.
service_account_file>

2.9 Type in/Select "n" for No

Edit advanced config?
y) Yes
n) No (default)
y/n> n

2.10 Type in "y" for Yes to authenticate to your google drive using a web browser

Use web browser to automatically authenticate rclone with remote?
* Say Y if the machine running rclone has a web browser you can use
* Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.

y) Yes (default)
n) No
y/n>y

2023/07/27 13:16:20 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=hy8ONFPQCykB30YldCCQfA
2023/07/27 13:16:20 NOTICE: Log in and authorize rclone for access
2023/07/27 13:16:20 NOTICE: Waiting for code...

2.11 A web browser window will pop up allowing you to enter in your credentials for your Google Drive Account

2.12 After signing in you will get a Success Message and then you can go back to command prompt to continue with setup.

2.13 Type in/Select "n" for no in regards to configuring as a Shared Drive.

2023/07/27 13:24:14 NOTICE: Got code
Configure this as a Shared Drive (Team Drive)?

y) Yes
n) No (default)
y/n> n

2.14 Configuration of Google Drive is complete. Review the information for accuracy and then type in/select "y" for yes to save the configuration and you are done.

Configuration complete.
Options:
- type: drive
- scope: drive
- token: {"access_token":"ya29.a0AbVbY6NTDkLCH7EgnFk1J2fKDtmXsqIlSff9lQnejKJweBLU9NteOyJtPxsqUTNAqMu-cJIuqwsgQOfO_SH7zbsKGoql0w4v0H3L80VDRsoK4U48LVXppsfgNb0-wJh3_YdAnTOcZQrqeLkaguSJAupy3IIhaCgYKAZQSARMSFQFWKvPlsfpcY6uxG-vsoPowhX3p-Q0163","token_type":"Bearer","refresh_token":"1//01c-s3KbSkfWzCgYIARAAGAESNwF-L9IrkIUDhEWLzb8MUJefx5INb8WAgj6tGm4QH7tvCQ-O3T-AnrztykMfFsv6KM0t9Inv7co","expiry":"2023-07-27T14:24:13.4851761-07:00"}
- team_drive:
Keep this "google drive" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>y

3. Configure Wasabi as a Remote Location

3.1 Enter "n" for a new remote connection

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n

3.2 Type in a name corresponding to Wasabi for your Remote location

Enter name for new remote.
name> wasabi

3.3 Type in/select "5" to choose Amazon S3 compliant provider

Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
1 / 1Fichier
\ (fichier)
2 / Akamai NetStorage
\ (netstorage)
3 / Alias for an existing remote
\ (alias)
4 / Amazon Drive
\ (amazon cloud drive)
5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, ArvanCloud, Ceph, China Mobile, Cloudflare, GCS, DigitalOcean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, IONOS Cloud, Liara, Lyve Cloud, Minio, Netease, Petabox, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS, Qiniu and Wasabi
\ (s3)
Storage> 5

3.4 Type in/select "25" to choose Wasabi as the S3 provider from the list

Option provider.
Choose your S3 provider.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
1 / Amazon Web Services (AWS) S3
\ (AWS)
2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun
\ (Alibaba)
3 / Arvan Cloud Object Storage (AOS)
\ (ArvanCloud)
4 / Ceph Object Storage
\ (Ceph)
5 / China Mobile Ecloud Elastic Object Storage (EOS)
\ (ChinaMobile)
6 / Cloudflare R2 Storage
\ (Cloudflare)
7 / DigitalOcean Spaces
\ (DigitalOcean)
8 / Dreamhost DreamObjects
\ (Dreamhost)
9 / Google Cloud Storage
\ (GCS)
10 / Huawei Object Storage Service
\ (HuaweiOBS)
11 / IBM COS S3
\ (IBMCOS)
12 / IDrive e2
\ (IDrive)
13 / IONOS Cloud
\ (IONOS)
14 / Seagate Lyve Cloud
\ (LyveCloud)
15 / Liara Object Storage
\ (Liara)
16 / Minio Object Storage
\ (Minio)
17 / Netease Object Storage (NOS)
\ (Netease)
18 / Petabox Object Storage
\ (Petabox)
19 / RackCorp Object Storage
\ (RackCorp)
20 / Scaleway Object Storage
\ (Scaleway)
21 / SeaweedFS S3
\ (SeaweedFS)
22 / StackPath Object Storage
\ (StackPath)
23 / Storj (S3 Compatible Gateway)
\ (Storj)
24 / Tencent Cloud Object Storage (COS)
\ (TencentCOS)
25 / Wasabi Object Storage
\ (Wasabi)
26 / Qiniu Object Storage (Kodo)
\ (Qiniu)
27 / Any other S3 compatible provider
\ (Other)
provider> 25

3.5 Choose Option 1 to type in your Wasabi Credentials in the following Steps

Option env_auth.
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
Only applies if access_key_id and secret_access_key is blank.
Choose a number from below, or type in your own boolean value (true or false).
Press Enter for the default (false).
1 / Enter AWS credentials in the next step.
\ (false)
2 / Get AWS credentials from the environment (env vars or IAM).
\ (true)
env_auth> 1

3.6 Type in your Wasabi Access Key

Option access_key_id.
AWS Access Key ID.
Leave blank for anonymous access or runtime credentials.
Enter a value. Press Enter to leave empty.
access_key_id> xxxxxxxxxxx

3.7 Type if your Wasabi Secret Key

Option secret_access_key.
AWS Secret Access Key (password).
Leave blank for anonymous access or runtime credentials.
Enter a value. Press Enter to leave empty.
secret_access_key> xxxxxxxxxxxxxxxxxxx

3.8 Choose Option 1 for your Region selection

Option region.
Region to connect to.
Leave blank if you are using an S3 clone and you don't have a region.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
/ Use this if unsure.
1 | Will use v4 signatures and an empty region.
\ ()
/ Use this only if v4 signatures don't work.
2 | E.g. pre Jewel/v10 CEPH.
\ (other-v2-signature)
region> 1

3.9 Choose your endpoint for Wasabi where your bucket is located.

Option endpoint.
Endpoint for S3 API.
Required when using an S3 clone.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
1 / Wasabi US East 1 (N. Virginia)
\ (s3.wasabisys.com)
2 / Wasabi US East 2 (N. Virginia)
\ (s3.us-east-2.wasabisys.com)
3 / Wasabi US Central 1 (Texas)
\ (s3.us-central-1.wasabisys.com)
4 / Wasabi US West 1 (Oregon)
\ (s3.us-west-1.wasabisys.com)
5 / Wasabi CA Central 1 (Toronto)
\ (s3.ca-central-1.wasabisys.com)
6 / Wasabi EU Central 1 (Amsterdam)
\ (s3.eu-central-1.wasabisys.com)
7 / Wasabi EU Central 2 (Frankfurt)
\ (s3.eu-central-2.wasabisys.com)
8 / Wasabi EU West 1 (London)
\ (s3.eu-west-1.wasabisys.com)
9 / Wasabi EU West 2 (Paris)
\ (s3.eu-west-2.wasabisys.com)
10 / Wasabi AP Northeast 1 (Tokyo) endpoint
\ (s3.ap-northeast-1.wasabisys.com)
11 / Wasabi AP Northeast 2 (Osaka) endpoint
\ (s3.ap-northeast-2.wasabisys.com)
12 / Wasabi AP Southeast 1 (Singapore)
\ (s3.ap-southeast-1.wasabisys.com)
13 / Wasabi AP Southeast 2 (Sydney)
\ (s3.ap-southeast-2.wasabisys.com)
endpoint>2

3.10 For the Local Constraint option leave blank and hit enter.

Option location_constraint.
Location constraint - must be set to match the Region.
Leave blank if not sure. Used when creating buckets only.
Enter a value. Press Enter to leave empty.
location_constraint>

3.11 Choose the ACL option you wish to use and enter the corresponding number and hit enter.

Option acl.
Canned ACL used when creating buckets and storing or copying objects.
This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
Note that this ACL is applied when server-side copying objects as S3
doesn't copy the ACL from the source but rather writes a fresh one.
If the acl is an empty string then no X-Amz-Acl: header is added and
the default (private) will be used.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
/ Owner gets FULL_CONTROL.
1 | No one else has access rights (default).
\ (private)
/ Owner gets FULL_CONTROL.
2 | The AllUsers group gets READ access.
\ (public-read)
/ Owner gets FULL_CONTROL.
3 | The AllUsers group gets READ and WRITE access.
| Granting this on a bucket is generally not recommended.
\ (public-read-write)
/ Owner gets FULL_CONTROL.
4 | The AuthenticatedUsers group gets READ access.
\ (authenticated-read)
/ Object owner gets FULL_CONTROL.
5 | Bucket owner gets READ access.
| If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
\ (bucket-owner-read)
/ Both the object owner and the bucket owner get FULL_CONTROL over the object.
6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
\ (bucket-owner-full-control)
acl> 1

3.12 Type in/Select "n" for No

Edit advanced config?
y) Yes
n) No (default)
y/n> n

3.13 Configuration of your Remote Location for Wasabi is now complete. Review the information for accuracy and type in/select "y" for yes and you are done. 

Configuration complete.
Options:
- type: s3
- provider: Wasabi
- access_key_id: 916ACWKFVJ9RPC17UF5H
- secret_access_key: czpZZtYj1OXv2D1epYDYX8lbHRl7A1UlpWkaOyVO
- endpoint: s3.us-east-2.wasabisys.com
- acl: private
Keep this "wasabi" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>y

3.14 You should not see both your Google Drive and Wasabi Remote Accounts listed.

Current remotes:

Name Type
==== ====
google drive drive
wasabi-us-east-2 s3

3.15 Type in/Select "q" to quit from the Remote Configuration and go back to the root level

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

c:\rclone-v1.63.1>

4. Copy Data

Now that you have both Google Drive and Wasabi Configured as remotes, you can copy the data over to Wasabi. The basic Rclone command to copy data from Google Drive to Wasabi looks like this:

rclone copy "remote name for google drive":source folder "remote name for wasabi":destination folder

Note: If you have spaces in either your Remote Drive name or Source/Destination paths you will need to use double quotes or else you will get an error. For example if your drive is google drive you need to put "google drive" or if your path is Rclone\Data\Test Folder then you need to use "Rclone\Data\Test Folder"

Example: 

The -P will give you a print out of the transfer as it is taking place.

rclone -P copy "google drive":Data wasabi-us-east-2:rclone\Data
Transferred: 334.316 MiB / 334.316 MiB, 100%, 8.086 MiB/s, ETA 0s
Transferred: 2 / 2, 100%
Elapsed time: 32.0s

5. Move Data

Now that you have both Google Drive and Wasabi Configured as remotes, you can move the data to Wasabi. The "rclone move" command will copy files from the source (Google Drive) to the destination (Wasabi) and then remove the source files from Google Drive if the transfer is successful. The basic syntax of the 'rclone move' command looks like this:

rclone move "remote name for google drive": "remote name for wasabi":

Note: If you have spaces in either your Remote Drive name or Source/Destination paths you will need to use double quotes or else you will get an error. For example if your drive is google drive you need to put "google drive" or if your path is Rclone\Data\Test Folder then you need to use "Rclone\Data\Test Folder"

Example:

rclone -P move "google drive":Data wasabi-us-east-2:rclone\Data
Transferred: 1.308 MiB / 1.308 MiB, 100%, 0 B/s, ETA -
Checks: 2 / 2, 100%
Deleted: 1 (files), 0 (dirs)
Renamed: 1
Transferred: 1 / 1, 100%
Elapsed time: 5.5s