Amazon Web Services or AWS has a range of storage systems to choose from and it often confuses the users to select the right one for their requirements. In an organization, the cloud architects, involved in that project analyze the business needs and choose the proper storage option accordingly. But if you an individual or a small team of web developers and don’t have much knowledge about AWS architecture, then you can go through this article to get an idea.
AWS Storage options
There are five types of storages options are available in Amazon Web Services:
- Amazon Elastic Block Storage (Amazon EBS)
- Amazon Elastic File System (Amazon EFS)
- Amazon Simple Storage Service (Amazon S3)
- Amazon Glacier
- Amazon Storage Gateway
For Web developers, only first three options are important. So, if you are planning to build or move your web app or website to Amazon Web Services, then you have three storage options to choose from: Amazon EBS, Amazon EFS and Amazon S3.
In this section we will discuss the differences between Amazon S3, Amazon EFS and Amazon EBS storage options. Before that you need to know these two terms: Throughput and IOPS.
Throughput is data transfer speed in megabytes per second.
IOPS is Input/Output operations per second or in simple words maximum number of reads and writes to non-contiguous storage locations per second.
Amazon S3
Amazon S3 is highly scalable, highly available, redundant stand-alone storage service. It’s not like traditional file systems- you can’t just mount it as NFS. Amazon S3 is an object store with simple key, value store design. It stores data as objects or files. The objects are stored in one or more Buckets. Each object is assigned a name (Key), you can use that Key to access the object from anywhere, even directly through internet. The content that is stored in the object is called Value. The maximum size of an object can be of 5TB.
So, Amazon S3 can be a good option to store static contents like images, videos for your website. You can also use Amazon’s cache service- AWS CloudFront to deliver your website static contents instead of delivering those from the bucket. If you are planning to migrate your existing static contents to AWS S3, make sure to perform data deduplication before the migration to save some storage cost.
Amazon EBS:
Unlike Amazon S3, to use Amazon EBS you need to have Amazon EC2 (Elastic Computer Cloud). Like traditional file systems it stores data in data-blocks and can be attached to different EC2 instances only. Amazon EBS cannot be scaled easily. If storage volume space is exhausted, then you can attach another volume (specific limit of attaching EBS volumes) or you can increase the size of existing EBS volume. To increase the size of an existing volume, the EBS volume is required to be detached from the EC2 instance – that means stopping the EC2 instance.
So, for Amazon EBS capacity planning is very important and volume addition or expansion should be planned properly to overcome any unplanned business outage.
So, if you host a very large file uploading website where you don’t have any prediction or actual estimation of how much data will be uploaded by the users, using Amazon EBS to hold those uploads is not a good idea, in that case Amazon S3 will be a better choice.
Whereas if you need Block level storage for specific requirements, (for e.g. Oracle ASM uses its own file system and it requires block level storage) you need to use Amazon EBS.
There are four types of volumes in Amazon EBS:
- General Purpose SSD Volumes (gp2) has baseline performance of 3 IOPS/GB and a possibility to burst up to 10,000 IOPS and throughput of 160MBps, that makes it suitable for system boot volumes and small-medium size databases. You can use gp2 for mongoDB, Couchbase etc. NoSQL databases where you have option of horizontal scaling.
- Provisioned IOPS SSD (io1) is designed for much higher workload. It supports up to 64,000 IOPS and 1,000 MB/s of throughput. It’s ideal for high transaction RDBMS databases like Oracle, MSSQL etc.
- Throughput optimized HDD (stl) and Cold HDD (sc1) both are low-cost magnetic storage; they have good throughput (stl: 500 MBps and sc1: 250 MBps) but poor IOPS. So stl and sc1 are optimized for large, sequential I/O but not efficient to handle small, random I/O. It’s not recommended to use stl or sc1 for the application/databases that demands higher IOPS.
Amazon EFS
Amazon EFS or Elastic File System is highly scalable file storage service provided by Amazon Web Services. Unlike S3 storage service, EFS uses NFSv4 protocol to allow traditional hierarchical directory structure. Multiple EC2 instances from multiple Availability Zone can access the same EFS concurrently. So, if your application has a requirement of traditional directory structure, shared file access (like available in traditional NAS) and at the same time you need high scalability then you should go for EFS. According to Amazon, EFS is perfect fit for Big Data analytics, lift-and-shift enterprise applications, database backup etc. where with good throughput, consistent and low per-operation latency is required.
Now, as you have an idea about different AWS storage options, you can check AWS Pricing for different storage services.
