How to use Storadera with Rclone

From Rclone’s homepage:
“Rclone is a command line program to manage files on cloud storage.”
From Storadera side we can say that Rclone is a really powerful tool that can help you use cloud storage in many ways.

Install

Rclone is available for Windows, Linux, Mac and more.
Get installation instructions from:
https://rclone.org/downloads/

Configure

At a first glance, following configuration may seem long but there are actually only a handful of steps.
We will show detailed overview of all the steps.
First, enter configuration command

$ rclone config

Create new config, enter “n”

No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n

Enter name: storadera

name> storadera

Next step shows long list of storage protocols. Choose storage type: 4

Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
....
 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean,
Dreamhost, IBM COS, Minio, etc)
 \ "s3"
.....
Storage> 4

Next step shows list of storage providers. Choose: 11

** See help for s3 backend at: https://rclone.org/s3/ **
Choose your S3 provider.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
.....
11 / Any other S3 compatible provider
 \ "Other"
provider> 11

For simplicity we configure credentials right here to configuration file. Choose: 1

Get AWS credentials from runtime (environment variables or EC2/ECS meta data if
no env vars).
Only applies if access_key_id and secret_access_key is blank.
Enter a boolean value (true or false). Press Enter for the default ("false").
Choose a number from below, or type in your own value
 1 / Enter AWS credentials in the next step
 \ "false"
 2 / Get AWS credentials from the environment (env vars or IAM)
 \ "true"
env_auth> 1

Enter your Access key and Secret access key

AWS Access Key ID.
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("").
access_key_id> YOUR_ACCESS_KEY
AWS Secret Access Key (password)
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("").
secret_access_key> YOUR_SECRET_ACCESS_KEY

Leave region blank. Just press enter.

Region to connect to.
Leave blank if you are using an S3 clone and you don't have a region.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Use this if unsure. Will use v4 signatures and an empty region.
 \ ""
2 / Use this only if v4 signatures don't work, eg pre Jewel/v10 CEPH.
 \ "other-v2-signature"
region>

Enter the service url: https://eu-east-1.s3.storadera.com

Endpoint for S3 API.
Required when using an S3 clone.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
endpoint> https://eu-east-1.s3.storadera.com

No need for location constraint, leave empty

Location constraint - must be set to match the Region.
Leave blank if not sure. Used when creating buckets only.
Enter a string value. Press Enter for the default ("").
location_constraint>

Security settings. Take safest option and make all data private. Choose: 1

Canned ACL used when creating buckets and storing or copying objects.
This ACL is used for creating objects and if bucket_acl isn't set, for creating
buckets too.
For more info visit
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
Note that this ACL is applied when server side copying objects as S3
doesn't copy the ACL from the source but rather writes a fresh one.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Owner gets FULL_CONTROL. No one else has access rights (default).
 \ "private"
2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
 \ "public-read"
 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
3 | Granting this on a bucket is generally not recommended.
 \ "public-read-write"
4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access.
 \ "authenticated-read"
 / Object owner gets FULL_CONTROL. Bucket owner gets READ access.
5 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores
it.
 \ "bucket-owner-read"
 / Both the object owner and the bucket owner get FULL_CONTROL over the
object.
6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores
it.
 \ "bucket-owner-full-control"
acl> 1

No need for advance config. Choose: n

Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n> n

Confirmation screen to check if all is good. Choose: y

Remote config
--------------------
[storadera]
type = s3
provider = Other
env_auth = false
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUUR_SECRET_ACCESS_KEY
endpoint = https://eu-east-1.s3.storadera.com
acl = private
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y

Main configuration menu. To exit, choose: q

Current remotes:
Name Type
==== ====
storadera s3
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

Configuration file

After prior configurations the configuration file should look like that:

[storadera]
type = s3
provider = Other
env_auth = false
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUR_SECRET_ACCESS_KEY
endpoint = https://eu-east-1.s3.storadera.com
acl = private

Test if it works (the colon is important there!)

$ rclone ls storadera:

Copy one file “sample.txt” to bucket “my-bucket”

rclone copy sample_folder storadera:my-bucket

Mount a bucket as local folder

rclone mount storadera:my-bucket rmount --vfs-cache-mode full

NB! Do check the documentation for production usage!

Rclone is very powerful tool and has many possibilities of usage. The examples above are only for giving a hint how to start using it.
For full list of commands and usage, please consult Rclone homepage
https://rclone.org/docs/#subcommands




Privacy Policy | Terms and Conditions | Copyright 2021 | Storadera Ltd.