SoftNAS AWS backend storage selection guide
SoftNAS Cloud NAS supports the broadest range of backend storage options on Amazon Web Services (AWS), including Simple Storage Service (S3) and Elastic Block Store (EBS). It’s important to select the right backend storage to align with the performance requirements of your application.
Step 1:
Understanding AWS storage types
In general, AWS offers two types of native storage services that are used with SoftNAS products:
- Amazon S3
- Amazon EBS
To determine the most appropriate storage type, you will need to know the IOPS requirements of your application. Amazon EBS magnetic or SSD backed storage along with Amazon S3 object storage have benefits and limitations that work with or against certain workloads.
What is Amazon EBS?
Amazon EBS is a block level storage type that runs on Amazon Elastic Compute Cloud (EC2). There are three types of Amazon EBS volumes: General Purpose, Provisioned IOPS, and Magnetic Volumes. There are differences in total throughput capabilities between Provisioned IOPS (io1) and General Purpose SSD (gp2) volumes. io1 volumes are designed to provide up to 320 MB/second of throughput while gp2 volumes are designed to provide up to 160 MB/second.
Amazon EBS volume types
The following table from AWS shows use cases and performance characteristics of current generation Amazon EBS volumes.
What is Amazon S3?
Amazon S3 is a service that provides object storage, which is a type of storage where data is stored in a flat environment without a hierarchy. Amazon S3 is highly scalable with high durability and low cost, but Amazon S3 performance is generally slower than that of Amazon EBS. Using Amazon S3, SoftNAS AWS NAS Storage Solution can scale cloud storage to practically unlimited capacity. You can provision each cloud disk to hold up to four PB of data. If a larger data store is required, you can use RAID to aggregate multiple cloud disks.
Step 2:
Choose between Amazon EBS and Amazon S3 storage
Amazon S3 is usually chosen for applications that do not require heavy read/write operations. However, with SoftNAS® ObjectBacker™ Backend Storage Acceleration with patent-pending ObjFast™ adaptive flow technology, SoftNAS Cloud NAS can write data up to 400% faster than before, making cost-effective Amazon S3 storage a viable option for write intensive use cases that want to take advantage of lower cost Amazon S3 storage.
Amazon S3 storage with SoftNAS ObjectBacker is appropriate for write sensitive workloads, including:
Archival Backup
Lift and Shift Data Migrations
Large-Scale Content Publishing
When you use a cloud disk, we suggest using an ephemeral or block device local to the SoftNAS Cloud NAS virtual appliance as a read cache to reduce Amazon S3 I/O charges and improve IOPS and performance for read-intensive workloads. Amazon S3 cloud disks are suitable for archive or general access applications where low cost is important. They are not suitable for applications that need high IOPS performance and very high throughput on random read/write workloads where Amazon EBS cloud disks are more appropriate. The charts below show IOPS and throughput performance increases for NFS, CIFS/SMB, and iSCSI workloads using SoftNAS ObjectBacker.
Step 3:
Choose an Amazon EBS volume type
If Amazon EBS is appropriate for your performance requirements, SoftNAS recommends these general guidelines for selecting an Amazon EBS volume type.
General Purpose SSD (gp2) volumes
General Purpose Volumes are backed with Solid State Drive (SSD) and are designed for a broad range of purposes. The baseline performance of 3 IOPS/GB and a possibility to burst up to 10,000 IOPS makes them a good fit for applications that need a lot of read and write operations, like Microsoft SQL. gp2 and io1 SSD volumes can achieve faster IOPS performance and very high throughput on random read/write workloads when compared to magnetic disks, but at a higher price point. However, gp2 and io1 volume types are limited to a throughput of 320 MB/s (160 MB/s for gp2, 320 MB/s for io1). gp2 volumes provide a fixed 1:3 ratio between gigabytes and IOPS provisioned, so a 100 GB General Purpose volume provides a baseline of 300 IOPS. gp2 volumes less than 1TB in size can also burst for short periods, up to 3,000 IOPS. You can provision gp2 volumes up to 16 TB and 10,000 IOPS.
Provisioned IOPS (SSD) Volumes (io1)
Provisioned IOPS are backed with the same SSD as General Purpose Volumes, but Provisioned IOPS are designed for heavy workloads from 30 IOPS/GB up to 20,000 IOPS. For even more demanding workloads, multiple provisioned IOPS volumes can be striped, ensuring up to 48,000 IOPS or 800 MBps of throughput. Provisioned IOPS (io1) volumes are intended for workloads that demand consistent performance, such as relational (SQL) databases. You can create provisioned IOPS volumes up to 16 TB and 20,000 IOPS. Over a year, Amazon EBS Provisioned IOPS volumes are designed to deliver within 10% of the provisioned IOPS performance 99.9% of the time.
Magnetic Volumes (st1 and sc1)
Magnetic Volumes are a low cost volume type that can be used with testing and development environments on Amazon EC2. They can also be used with applications that don’t require a lot of read/write operations. Magnetic-backed volume types support higher block sizes up to 1,024 KB. Throughput Optimized HDD (st1) and Cold HDD (sc1) Amazon EBS volume types are based on magnetic storage technology.
Throughput Optimized HDD Volumes (st1)
The Throughput Optimized HDD (st1) volume type is designed for sequential read/write workloads (e.g., Big Data). It can achieve very high throughput (500 MB/s) for sequential read/write workloads (compared to 160 MB/s and 320 MB/s for SSD-backed gp2 and io1, respectively). Generally, Big Data workloads operate on very large sequential datasets and generate data for storage in a similar way. The st1 volume type has a baseline performance of 40 MB/s per TB of allocated storage and, like gp2, can burst beyond the baseline performance for a short period of time.
Cold HDD Volumes (sc1)
The Cold HDD (sc1) volume type is designed for high density and infrequent access workloads. This volume type is suitable for cold storage (infrequent access) applications where low cost is important. Unlike st1, the baseline performance of an sc1 volume is 12 MB/s per TB of allocated storage. It’s important to note that Amazon S3 achieves high availability (HA) by default within a single region, whereas sc1 volumes have to be mirrored across Availability Zones to achieve parity with Amazon S3 in durability and availability of the data. (This doubles and triples the cost of sc1 when compared to Amazon S3.) Nevertheless, depending on certain access patterns (e.g., cold versus warm) of the data, the cost of sc1 volumes can be cheaper for certain workloads.