S3 Compatible Cloud Storage
History
S3 stands for Simple Storage Service. It was one of first three cloud services AWS released back in 2006. Over time, people discovered that S3 is a cost effective way to store the data.
Present
S3 API has become de facto standard for non-structured data. It means that S3 has become default choice for unstructured data storage.
There is more choice of services as well. Besides AWS there are many new S3 service providers, Storadera is an example.
In addition to public cloud services, companies can self-host S3 compatible storage if they want or need it. Owning infrastructure is more complex than using a public cloud, make sure you have good reasons to do it. Cloud service can be better for the environment as well!
For self-hosting, companies can use open source or commercial software.
Other big cloud providers have developed competing standards: Google Cloud Storage and Microsoft Azure Blob Storage. These are used typically by clients who use their other cloud services as well.
Multiverse of S3 pricing complexity
Be aware that word “simple” does not apply for AWS pricing. They have (at the time of writing) 7 different storage tiers, every tier having multiple pages for price-list.
The most obvious part of price is storage. Most providers want to make it fuzzy and present price per GB, / month sometimes price per GB / hour. This produces numbers with many decimal places and it is hard to comprehend. When comparing, always convert price to TB / month.
The second important component is download cost (sometimes called egress fee). It can very easily be the biggest part of the cost.
Upload is typically free but there are exceptions!
Then there are micro-charges for listing your files or getting other information about your own data.
Dome pricing tiers or S3 compatible storage providers charge for minimum retention period. As an example, if there is 90 day retention period and you delete your data before this, you still pay for remaining days.
And there can also be minimum file size, for example 128KB. It may sound small but if you upload 12KB files then your storage cost will be 10x of the sum your file sizes.
Be aware that pricing in different regions can differ as well!
Storadera offers the simplest pricing model, with only storage cost that is typically 5x less than AWS. And no extra fees. It is that simple. Sign up to try.
Architecture
Technical bits
S3 cloud storage services are typically a distributed systems.
S3 API is a REST api.
Fun fact, this S3 “simple storage service” is not so simple any more, it has close to 100 different API-s to offer all the modern features you may need.
Data structure
As an owner of data, you first choose region where you like your data to live. It can be Germany in Europe, US east coast, Japan etc.
Second step is creating a bucket. Bucket can be considered a main level folder. Bucket name must be unique over all the buckets in this region because it can be used as part of url to access your data. It is a clever choice by AWS back in the days because it allows routing your http request directly to the servers that hold the data.
Third peace is key – it is actual filename and it can contain slashes to mimic folders.
Access and permissions
Access to data is protected by KEY and SECRET_KEY. This pair can be generated in self service portals and need to be entered to your application.
NB! Keep you SECRET_KEY a secrete, never share!
Applications
As said, S3 is the default protocol for using cloud storage in your applications and really wide variety of applications are available. It is out of the scope of this page to list them all but you can count that you can cover use-cases for backup, archive, data lake and so on.
You can see our small curated list of S3 compatible application set up manuals
Why it is called object storage?
To be clear, S3 storage can be called object storage.
Object storage opposes to filesystem. Object storage solution is meant for big amounts of unstructured data.
Objects in storage can have wide amount of metadata attached to it.
Storadera
Storadera offers fast and secure S3 compatible storage, including strong encryption for your data in transit and at rest.
Our pricing is the simplest and most affordable S3 compatible cloud storage on the market for only 6 EUR / TB / month. No extra fees. No download cost.
Storadera just makes sense!