How Do I Use an AWS S3 Bucket? Amazon S3 Six Storage Types (Standard/Tiered/Cold Archive) Pricing and Selection Guide
In cloud computing and offshore business, if there is a service that all technical teams can't get around, it must be
Amazon S3(Simple Storage Service)
.
Whether you are storing App's user avatar, highly concurrent static web pages, game release patches, or dozens of T database history backups, S3 is the "golden universal trash can" that silently resists everything at the bottom all the year round and will never explode ".
However, many people in the first contact with S3, often think that it is too simple-"is it not a net disk? Create a new bucket to throw files on the line?"
This is the beginning of a huge bill.
S3 is ostensibly a service, but in fact it is hidden in the stomach.
6 completely different storage types (Storage Classes)
. For the same 10 TB of data, if you choose the wrong type, you may have to pay hundreds of dollars a month. However, if you choose the right one, the price can be cut directly.
90%
!
Today's tutorial does not talk about empty official slang, but uses the most down-to-earth language to take you to completely strip S3
Core usage, computing power price differences for the six storage types, and money-saving options.
.
The first stage: big vernacular dismantling, S3 bucket in the end how to use?
Before selecting the model, let's streamline the core operating logic of S3 in three sentences. S3 world does not have the traditional server "folder path" concept, it is essentially a.
Key-Value Object Storage System
.
Bucket: It is equivalent to your exclusive "root disk". The name of the bucket must be unique among all Amazon cloud accounts worldwide (for example, if you name my-test-bucket, others can no longer use it).
Object: The file you upload. Each file has a unique key (equivalent to a file name or a pseudo-path, such as images/user01.jpg).
AWS Amazon Cloud Account Purchase
Brainless expansion: You don't need to specify "I want to buy 500 GB in this bucket" in advance like buying a cloud hard disk ". The capacity of S3 is unlimited. You can put 100 PB of money for one file and Tencent/Amazon.
Phase II: Core hard-core duel, full resolution of the six storage types of S3
AWS based on data
Access frequency (how many times a month you read it) and retrieval speed (how fast it can spit out when you want it)
, the storage is divided into six major sects:
1. S3 Standard Edition (S3 Standard)-Default All-round Flash
Features: millisecond latency, high concurrency, high reliability (data is automatically stored in at least 3 different computer rooms).
Price: The most expensive. Take the eastern United States (Virginia) for example, about $0.02
3/GB/month.
Big White: This is the default option when you upload a file. If your files are frequently accessed every day (such as pictures of web pages, short videos that are being broadcast), don't hesitate to lock the standard version.
2. S3 Infrequent Access Edition (S3 Standard-IA)-Save Money "Semi-Retired" Veterans
Features: It is still a millisecond delay. Data is also backed up in 3 computer rooms, but the storage unit price is directly cut by half (about $0.0125/GB/month).
Cost: Storage becomes cheaper, but there is a "retrieval fee" (data download fee) for each file read.
Purchase an AWS Amazon Cloud account
Big vernacular: suitable for storing data that "no one reads it at ordinary times, but it must be opened immediately once it is read. For example, financial statements and historical compliance logs from six months ago.
3. S3 Single Zone Infrequent Access Edition (S3 One Zone-IA)-Extreme Money-Saving "Gamblers"
Features: The price is 20% cheaper than ordinary IA (about $0.01/GB/month).
Cost: As the name implies, it is only stored in 1 computer room (usable area). If that computer room is hit by an earthquake fire, your data will be completely wiped out.
Big white: Never use it to save core assets! It is only suitable for storing unimportant, non-core secondary caches that can be regenerated by code even if lost (such as intermediate slices of video transcoding).
4. S3 Intelligent Layering (S3 Intelligent-Tiering)-The artifact of lazy people and architects.
Features: The official comes with the automation plug-in. AWS will be watching your files in the background. If a file is not touched for 30 consecutive days, it will automatically downgrade you to IA (save money); If one day this file suddenly explodes and is read at high frequency, it will automatically upgrade you back to the standard version in seconds (eliminating the high retrieval fee).
Cost: Amazon charges an extremely small "monitoring fee" every month (only a few cents for tens of thousands of files based on the number of files).
Big vernacular: If you take over an old system and have no idea about the access rules of these files, choosing intelligent layering is the safest anti-riot lightning scheme.
5. S3 Glacier Instant Retrieval (S3 Glacier Instant Retrieval)-Frozen Cold Data
Features: The price is extremely low (about $0.004/GB/month), and fortunately, it is also millisecond when it is taken out, so there is no need to wait.
Great vernacular: suitable for quarterly/annual archives that are read only once or twice a year, but require "seconds on" as long as the call is made.
6. S3 Glacier Flexible/Deep Cold Archive (S3 Glacier Flexible/Deep Archive)-Data Ultimate Crematorium
Features: Cheap to outrageous! Deep cold archiving only costs $0.00099/GB/month. Save 1 TB of data, one
Only US $1 (less than RMB 7) a month.
Cost: Not seconds. If you want to use this file, you must first submit a "unfreeze (Restore)" application in the console. Flexible archiving requires a wait of minutes to hours; deep cold archiving requires a dry wait of 12 hours. Moreover, the cost of reading (thawing) is very expensive.
The vernacular: "dead data" for those compliance requirements ". For example, medical records backup, bank backup, according to the law must keep more than 5 years of the original log.
Phase 3: A table to see S3 price actuarial (take Eastern Virginia as an example)
We take the team to store
10 TB(10240 GB)
Take the data as an example to see how exaggerated the meter is when choosing the wrong type:
Storage type
Monthly Rent Price per GB
10 TB data pure storage monthly bill
Retrieval fee (per GB read)
File fetching speed
S3 Standard (Standard)
$0.023
$235.5
$0 (free)
Millisecond on
S3 Standard-IA (infrequent)
$0.0125
$128.0
$0.01
Millisecond on
S3 Intelligent-Tiering
dynamic change
Automatically help you save money in the middle
$0
Millisecond on
S3 Glacier Instant
$0.004
$40.9
$0.03
Millisecond on
S3 Glacier Deep
$0.00099
$10.1
$0.05
Wait 12 hours
🚨Small case of accounting: if this 10 TB is the company's historical surveillance video, no one will watch it once a year. If you throw it in the standard version, it will cost 235.5*12=2826 US dollars a year. If dropped in deep cold filing: only 10.1*12=$121.2 a year. In a word, moving the mouse to change the type saves nearly 20,000 yuan per year.
Stage 4: Advanced Master-Two Life-saving Tips to Prevent S3 Billing Thunder
In a production environment, veterans never manually change the storage type of millions of files. We all use the following two official strategies to weld our wallets:
1. Configure life cycle rules (Lifecycle Rules)-automatic aging mechanism
You can configure an automated pipeline on the bucket management page:
Business logic: The golden period of the file is only one month.
Policy configuration: * File upload day 1: in S3 Standard (high frequency seconds on). More than 30 days: Automatic flow to S3 Standard-IA (storage fee halved). More than 90 days: Auto-sink to S3 Glacier Deep Archive (enter very low cost
frozen state). More than 365 days: automatically completely deleted (expired cleanup).
In this way, the data of the entire bucket is maintained in a virtuous circle of dynamic balance, the old data automatically becomes cold and cheap, and the bill is always in the most reasonable state.
2. Beware of "shard/multi-version" vampires (AbortIncompleteMultipartUpload)
Many R & D teams use "multipart upload" to transfer large files of several G's. If the upload fails due to network disconnection, those "incomplete slices" that have been uploaded in half will be stuck in your S3 bucket.
Invisible pits: These fragments are invisible in your console file list, but AWS charges you for storage every day!
Self-defense trick: A rule must be added to the life cycle: "Clean up expired part uploads (Lifecycle rule to abort incomplete multipart uploads)", and set the fragments that have not been transmitted for more than 7 days to be automatically crushed. This rule can save many teams hundreds of dollars.
Summary and Selection Formula
Amazon S3 is not a simple network disk, it is a combination of high availability and cost control art. There is no need to struggle when selecting the type. Remember these four self-defense tips:
High-frequency access standards: static pictures of web pages and core thermal data of App, with the most stable Standard speed.
Purchase an AWS Amazon Cloud account
Unknown rules, selection and stratification: too lazy to analyze the hot and cold data, directly open the Intelligent-Tiering and let the official escrow.
Regular audits of lower glaciers: historical logs, semi-annual compliance reports, thrown into Standard-IA or Glacier Instant.
The dead cost lock is very cold: backup disaster recovery, dead data that have not been seen for several years, and decisively choose Deep Archive to lock into the frozen crematorium.
