Virteche
Guideschevron_rightStorage

Storage · 8 min read · Updated

How to Copy an S3 Bucket to Another Provider with rclone

rclone copies a bucket from one S3-compatible provider to another: set up a remote for each side, run rclone copy source:bucket destination:bucket, check the result with rclone check, then point your apps at the new endpoint and copy once more to pick up the last changes. It works the same from AWS S3, Backblaze B2, Wasabi, Cloudflare R2 and DigitalOcean Spaces. The data passes through the machine running rclone, so run it on a server, not over a home connection, for a large bucket.

1. Install rclone

The packaged version in Ubuntu and Debian is often old. rclone's own install script fetches the current release; on Windows and macOS, use the download from rclone.org.

sudo -v ; curl https://rclone.org/install.sh | sudo bash
rclone version

2. Set Up the Source

Use the line for where your bucket is now, with a key that can read it:

# AWS S3
rclone config create src s3 provider=AWS region=us-east-1 \
  access_key_id=KEY secret_access_key=SECRET

# Backblaze B2 (rclone's native B2 support)
rclone config create src b2 account=KEY_ID key=APPLICATION_KEY

# Wasabi (use your bucket's region in the endpoint)
rclone config create src s3 provider=Wasabi endpoint=s3.us-east-1.wasabisys.com \
  access_key_id=KEY secret_access_key=SECRET

# Cloudflare R2
rclone config create src s3 provider=Cloudflare region=auto \
  endpoint=https://ACCOUNT_ID.r2.cloudflarestorage.com \
  access_key_id=KEY secret_access_key=SECRET

# DigitalOcean Spaces (use your Space's region)
rclone config create src s3 provider=DigitalOcean endpoint=nyc3.digitaloceanspaces.com \
  access_key_id=KEY secret_access_key=SECRET
rclone lsd src:              # lists your buckets
rclone size src:my-bucket    # objects and total size

3. Set Up the Destination

For Virteche object storage, create the bucket in your dashboard first, then add the remote with the key and secret shown there. no_check_bucket stops rclone trying to create the bucket itself:

rclone config create virteche s3 provider=Other endpoint=https://s3.us-west.virteche.com region=us-west \
  access_key_id=KEY secret_access_key=SECRET force_path_style=true no_check_bucket=true

Any other S3-compatible destination works the same way with its own endpoint and region. Using object storage has the settings for the AWS CLI, restic and the SDKs too.

4. Copy

Try it with --dry-run first to see what would be copied, then run it for real:

rclone copy src:my-bucket virteche:my-bucket --dry-run
rclone copy src:my-bucket virteche:my-bucket \
  --metadata --fast-list --transfers 16 --checkers 32 --progress

For a bucket that takes hours, run it inside tmux or screen so it survives a dropped SSH connection. If it stops, run the same command again: files already copied are skipped.

5. Check the Copy

rclone check src:my-bucket virteche:my-bucket --one-way --fast-list
rclone size virteche:my-bucket

rclone check compares sizes and, where both sides have the same kind of checksum, the checksums. When it reports no common hash (B2 to S3, for example), it has compared sizes only. Adding --download compares the actual contents, at the cost of reading everything again from both sides.

6. Switch Over

  1. Change your apps and backup tools to the new endpoint, region and keys. Most need path-style addressing turned on for Virteche object storage.
  2. Run the same rclone copy again to pick up anything written in the meantime.
  3. Keep the old bucket for a week or two, until nothing has asked for it, then delete it.

Common Questions

  • Will moving the data cost anything at my old provider?

    Possibly. Reading data out of a bucket (egress) is billed by some providers and free or partly free at others, and each list and read request can be billed too. Check your current provider's pricing for the amount you are moving before you start; --fast-list keeps the request count down.

  • Does rclone copy old versions, permissions and lifecycle rules?

    No. It copies the current version of each object, with its content type and, with --metadata, its custom metadata. Bucket policies, ACLs, lifecycle rules, object versions and object lock settings stay behind, so set up whatever the new provider offers in their place.

  • Should I use rclone sync or rclone copy?

    copy for the move: it only adds and updates, and never deletes anything at the destination. sync also deletes files that are gone from the source, which is right for keeping a mirror up to date and wrong if you point it the wrong way round.

  • My files were public. Can they stay public?

    Virteche buckets are private: every request needs your keys or a pre-signed link. If your site loads images straight from a public bucket, serve them through your own server or app instead, or give out pre-signed links.

S3-compatible object storage from 1 TB at a flat monthly price, with transfer included.

See object storage

Keep Reading

All guides