Meeting Cloud File Storage Cost and Performance Goals – Harder than You Think

Meeting Cloud File Storage Cost and Performance Goals – Harder than You Think

Meeting Cloud File Storage Cost and Performance Goals

According to Gartner, by 2025 80% of enterprises will shut down their traditional data centers. Today, 10% already have. We know this is true because we have helped thousands of these businesses migrate workloads and business-critical data from on-premises datacenters into the cloud since 2013. Most of those workloads have been running 24 x 7 for 5+ years. Some of them have been digitally-transformed (code for “rewritten to run natively in the cloud”).

The biggest challenge in adopting cloud isn’t the technology shift – it’s finding the right balance of cost vs. performance and availability that justifies moving to the cloud. We all have a learning curve as we migrate major workloads into the cloud. That’s to be expected as there are many choices to make – some more critical than others.

Many of our largest customers operate mission-critical, revenue-generating applications in the cloud today. Business relies on these applications and their underlying data for revenue growth, customer satisfaction, and retention. These systems cannot tolerate unplanned downtime. They must perform at expected levels consistently… even under increasingly heavy loads, unpredictable interference from noisy cloud neighbors, occasional cloud hardware failures, sporadic cloud network glitches, and other anomalies that just come with the territory of large scale datacenter operations.

In order to meet customer and business SLAs, cloud-based workloads must be carefully designed. At the core of these designs is how data will be handled. Choosing the right file service component is one of the critical decisions a cloud architect must make.

For customers to remain happy, application performance must be maintained. Easier said than done when you no longer control the IT infrastructure in the cloud…

So how does one negotiate these competing objectives around cost, performance, and availability when you no longer control the hardware or virtualization layers in your own datacenter?  And how can these variables be controlled and adapted over time to keep things in balance? In a word – control. You must correctly choose where to give up control and where to maintain control over key aspects of the infrastructure stack supporting each workload.

One allure of the cloud is that it’s (supposedly) going to simplify everything into easily managed services, eliminating the worry about IT infrastructure forever. For non-critical use cases, managed services can, in fact, be a great solution. But what about when you need to control costs, performance, and availability? Unfortunately, managed services must be designed and delivered for the “masses”, which means tradeoffs and compromises must be made. And to make these managed services profitable, significant margins must be built into the pricing models to ensure the cloud provider can grow and maintain them.

In the case of public cloud shared file services like AWS® Elastic File System (EFS) and Azure NetApp® Files (ANF), performance throttling is required to prevent thousands of customer tenants from overrunning the limited resources that are actually available. To get more performance, you must purchase and maintain more storage capacity (whether you actually need that add-on storage or not). And as your storage capacity inevitably grows, so do the costs. And to make matters worse, much of that data is actually inactive most of the time, so you’re paying for data storage every month that you rarely if ever even access. And the cloud vendors have no incentive to help you reduce these excessive storage costs, which just keep going up as your data continues to grow each day.

After watching this movie play out with customers for many years and working closely with the largest to smallest businesses across 39 countries, at Buurst™ we decided to address these issues head-on. Instead of charging customers what is effectively a “storage tax” for their growing cloud storage capacity, we changed everything by offering Unlimited Capacity. That is, with Buurst SoftNAS® you can store an unlimited amount of file data in the cloud at no extra cost (aside from the underlying cloud block and object storage itself).

SoftNAS has always offered both data compression and deduplication, which when combined typically reduces cloud storage by 50% or more. Then we added automatic data tiering, which recognizes inactive and stale data, archiving it to less expensive storage transparently, saving up to an additional 67% on monthly cloud storage costs.

Just like when you managed your file storage in your own datacenter, SoftNAS keeps you in control of your data and application performance. Instead of turning control over to the cloud vendors, you maintain total control over the file storage infrastructure. This gives you the flexibility to keep costs and performance in balance over time.

To put this in perspective, without taking data compression and deduplication into account yet, look at how Buurst SoftNAS costs compare:

Buurst SoftNAS vs. NetApp ONTAP, Azure NetApp Files and AWS EFS

These monthly savings really add up. And if your data is compressible and/or contains duplicates, you will save up to 50% more on cloud storage because the data is compressed and deduplicated automatically for you.

Fortunately, customers have alternatives to choose from today:

  1. GIVE UP CONTROL – use cloud file services like EFS or ANF, pay for both performance and capacity growth, give up control over your data or ability to deliver on SLAs consistently
  2. KEEP CONTROL – of your data and business with Buurst SoftNAS, balance storage costs, and performance to meet your SLAs and grow more profitably.

Sometimes cloud migration projects are so complex and daunting that it’s advantageous to just take shortcuts to get everything up and running and operational as a first step. We commonly see customers choose cloud file services as an easy first stepping stone to a migration. Then these same customers proceed to the next step – optimizing costs and performance to operate the business profitably in the cloud and they contact Buurst to take back control, reduce costs, and meet SLAs.

As you contemplate how to reduce cloud operating costs while meeting the needs of the business, keep in mind that you face a pivotal decision ahead. Either keep control or give up control of your data, its costs, and performance. For some use cases, the simplicity of cloud file services is attractive and the data capacity is small enough and performance demands low enough that the convenience of files-as-a-service is the best choice. As you move business-critical workloads where costs, performance and control matter, or the datasets are large (tens to hundreds of terabytes or more), keep in mind that Buurst SoftNAS never charges you a storage tax on your data and keeps you in control of your business destiny in the cloud.

Next steps:

Learn more about how Buurst SoftNAS can help you maintain control and balance cloud storage costs and performance in the cloud.

Learn More:

Azure Managed Disks (Don’t get mixed up)
Crank Up the Performance for Your Cloud Applications
The Secret to Maximizing Returns on Auto-Optimized Tiered Storage

Best Practices for Amazon EBS with SoftNAS

Best Practices for Amazon EBS with SoftNAS

In this post AWS EBS Best Practices using SoftNAS Cloud NAS. AWS Elastic Block Storage (EBS) volumes are block-level, durable storage devices that attach to your EC2 Instances. EBS Volumes can use as your primary storage device for an EC2 instance or database, or throughput-intensive systems requiring constant disk scans.

AWS EBS volumes exist independently from your EC2 instances and can retain after the associated EC2 instance has been deleted. AWS provides various types of EBS volumes allowing you to tailor the right volume to meet your budget and application performance requirements.

SoftNAS AWS EBS best practices

Amazon Elastic Block Store (Amazon EBS) provides persistent block-level storage volumes for use with Amazon EC2 instances in the AWS Cloud.  EBS is not about any specific device type, it’s about providing EC2 instances with highly available and durable storage volumes. To achieve this, each Amazon EBS volume is automatically replicated within its Availability Zone. Amazon EBS volumes offer the consistent and low-latency performance needed to run your workloads.

According to Amazon documentation, these are the Amazon EBS limits within an AWS account:Amazon EBS limits

With large numbers of EBS volumes, the time to attach during boot can be high and potentially fail.   

When a volume does not attach, SoftNAS resolves through an AWS-supplied API to ensure each volume does get attached before completing the boot.  When SoftNAS is implemented along with Amazon Web Services, it has been proven that you can scale to very large numbers of EBS Volumes without boot-attach issues.

Best practices for using AWS EBS volumes with SoftNAS:

  • Each EBS volume attached to an instance will be constituted on independent storage hardware within AWS infrastructure.  Configure SoftNAS storage pools as RAID 0 to stripe across multiple EBS volumes to gain the highest possible bandwidth and performance
  • SoftNAS disk device protection (RAID levels 1, 10, 5, 6, 7) is unnecessary and should not be used in a storage pool with EBS volumes.  Using any RAID level beyond RAID 0 merely increases storage costs with little benefit in reducing failure rate or performance. EBS Volumes are IOP limited. EBS General Purpose SSD is limited to 10,000 IOPs per volume.  EBS Provisioned IOPs are limited to 20,000 IOPs per volume.  EBS Magnetic is more severely limited to 40-200 IOPs, which represents the capabilities of today’s spinning media.  In testing, we have seen the EBS SSDs provide more IOPs in shorter durations, but appear to have forced limiters with longer sustained IOP usage.  By striping across multiple EBS Volumes, of any type, the IOPs can be higher than a single EBS Volume can provide.  Of course, workload and queue depth dictates whether higher IOPs are achieved.
  • AWS EBS annual failure rate (AFR) is published to be between 0.1% and 0.2%.  Aggregating multiple EBS volumes within a SoftNAS storage pool will magnify the AFR.  The AFR is roughly the number of EBS volumes multiplied by the AFR rate.  Our recommendation is to understand the risk and size of storage pools appropriately.  Using 5 EBS  volumes within a storage pool (totaling up to 80 TB of capacity) will be an acceptable AFR for most use cases, and many use cases can tolerate an even higher AFR.
  • Use multiple SoftNAS storage pools for very high-capacity deployments.  EBS volumes separated by storage pools do not affect AFR.
  • Use SoftNAS backup to create EBS Snapshots of storage pools to further protect data. EBS snapshots are arguably the most useful and the most difficult to understand feature of EBS. You can backup the data on your EBS volumes to Amazon S3 by taking point-in-time snapshots. EBS Snapshots are incremental, which means that in order to create a subsequent snapshot, EBS saves only the disk blocks that have changed since the previous snapshots to S3.SoftNAS has integrated EBS snapshots with one-button backup and restore options for storage pools.
  • SoftNAS SNAP HA provides data protection and high availability.  Data is replicated across availability zones and failover is managed between a pair of SoftNAS instances in private or public VPCs.  SNAP HA is recommended for a complete data protection strategy, replicating the storage from one region/zone to another.
  • SoftNAS SnapReplicate can be used as part of a disaster recovery strategy by replicating data to a remote region or another data center.

SoftNAS AWS NAS Storage

SoftNAS is a software-defined AWS NAS delivered as a virtual appliance running within Amazon EC2. Provides NAS capabilities suitable for enterprise storage, including Multi-Availability Zone high availability with automatic failover in the Amazon Web Services (AWS) Cloud.

SoftNAS offers AWS customers an enterprise-ready NAS capable of managing your fast-growing data storage challenges including AWS Outpost availability. Dedicated features from SoftNAS deliver significant cost savings, high availability, lift and shift data migration, and a variety of security protection.

SoftNAS Named Best Cloud NAS Solution in Network World Review


networkworld

Network World recently reviewed software-based NAS solutions and concluded:

If you’re looking for a cloud-based solution, SoftNAS is your best bet.

SoftNAS 3.3.3, the latest version of our award-winning Cloud NAS solution, differs from other solutions reviewed by offering both software-based and on-premise versions.  It’s available using Amazon EC2, Microsoft Azure and most recently, CenturyLink Cloud.

Eric Geier, freelance tech writer with Network World, provides a thorough review, citing the following about how SoftNAS is typically deployed:

Commonly, SoftNAS is deployed in AWS VPCs serving files to EC2 based servers within the same VPCs. SoftNAS also supports a hybrid cloud model where one SoftNAS instance is deployed on-premise on a local PC in your office and a second instance in an AWS VPC. In this hybrid model, replication occurs from the local SoftNAS to cloud-based SoftNAS for cloud-based disaster recovery.

SoftNAS is quick and easy to configure and setup in minutes, something we strive to achieve with all our products. Eric alluded to this during the review:

Once we configured the EC2 instance of SoftNAS, we could access the web GUI, which they call the SoftNAS StorageCenter, via the Amazon IP or DNS address. After logging in, you’re prompted to register and accept the terms. Then you’re presented with a Getting Started Checklist, which is useful in ensuring you get everything setup.

Read the full review.

Better yet, give it a try yourself.

SoftNAS 30-day Free Trial
start free trial now

 

Learn more: https://www.softnas.com
Twitter: @SoftNAS
LinkedIn: https://www.linkedin.com/company/softnas

Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS: Which Performs Best?

Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS: Which Performs Best?


Comparing Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS. Great read from Nathan Wilkerson, Cloud Engineer with Metal Toad around NFS performance on AWS based on the upcoming Amazon EFS (Elastic File System). As Amazon EFS is not generally available, this is a good early look at a performance comparison among Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS.

According to Nathan:

SoftNAS had by far the best performance. Its cost is higher for low capacities, but will be important when latency is paramount.  At higher capacities the benefits of SoftNAS and lower overall cost are a clear winner, I would consider SoftNAS.

SoftNAS spends a lot of time testing its products, reviewing customer feedback and welcomes Nathan’s performance results.

But is SoftNAS actually more expensive? NO. Let me explain.

1) 100GB works well for performance tests but is not a realistic capacity point for most customers real-world use or a pricing analysis.  On average, SoftNAS customers tend to use  10TB – 20TB, and frequently surpass 100TB.

2) SoftNAS is less expensive than AWS EFS at 6TB and is dramatically more cost effective at higher capacities.

3) Not only is SoftNAS cost effective at real-world capacities (even with EBS pricing), you get all the benefits of an enterprise-class NAS that is available in all AWS regions and which includes snapshots, de-duplication, data protection, multi-tier caching and compression.

Pricing: 100GB

The configuration in Nathan’s tests used 100GB with a pair of instances for High Availability. SoftNAS offers 100GB storage at no charge for each SoftNAS instance when launched as Community AMIs. See below the pricing for each vendor at 100GB usable capacity for EBS General Purpose SSD, per month (as tested in the Metal Toad benchmarks).

Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS table

It’s also worth noting that for the pricing comparisons in Nathan’s blog he used 200GB for SoftNAS compared to 100GB for EFS and Gluster. SoftNAS offers stack prices for both small and medium EC2 instances.  The medium instance will provide a better performance, but speed is not always the objective.  Offering the flexibility to configure based on what you want to accomplish is a cornerstone of the SoftNAS approach.

Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS pricing

It’s also worth noting that for the pricing comparisons in Nathan’s blog, he used 200GB for SoftNAS compared to 100GB for EFS and Gluster. SoftNAS offers stack prices for both small and medium EC2 instances. The medium instance will provide a better performance, but speed is not always the objective. Offering the flexibility to configure based on what you want to accomplish is a cornerstone of the SoftNAS approach.

Pricing: scaling up to 20TB with SSD

Lets look at the comparisons as you scale out the capacities up to 20TB with both backed by SSD. The following table shows the cost comparison of 20TB for SoftNAS with High Availability and AWS EFS.

The following graph compares SoftNAS with HA and AWS EFS for capacities from 200GB through 20TB. At 20TB, SoftNAS price per GB is $.0.13 vs EFS at $0.30.  The conclusion is that EFS is less at low capacity due to lack of EC2 and software costs, whereas when capacity increases the costs of storage outweighs the overhead costs. The breakeven point is 6TB and SoftNAS is progressively cheaper as capacity grows. Additionally, you get all the benefits of an enterprise-class NAS that is available in all AWS regions and which includes snapshots, de-duplication,data protection, multi-tier caching and compression.

Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS performance

Also, it’s worth mentioning that SoftNAS offers a high amount of configurability. You can choose between performance vs. cost, options for L1 and L2 read cache, options for write cache, and a ton of other useful features, including CIFS with Active Directory integration.

Finally, it’s worth noting that the flexibility of SoftNAS means you can choose not only SSD-backed storage, but also mix in less expensive alternatives like EBS Magnetic ($0.05/GB) and highly-durable S3 backed object storage ($0.03/GB).

So who performs best when comparing Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS?

SoftNAS delivers the most storage flexibility and most powerful feature set for protecting business data in the AWS cloud, and provides great price/performance. Best of all, SoftNAS is available on the most popular cloud and premise-based platforms today (AWS, Azure, CenturyLink, and VMware vSphere), keeping you in control of your business-critical data and your business.

Why not try SoftNAS and see how it performs in your environment?

Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS trial

Why SoftNAS?

SoftNAS provides direct connectivity to cloud storage providing a private connection to resources owned by your organization. This eliminates the managed storage problems of inconsistent performance by noisy neighbors. In addition, SoftNAS adds value by placing data in the most appropriate location for increased application performance.

Cloud Storage: Delivering on a performance priority

One of the top priorities for Cloud architects is achieving high performance from applications. As we move more applications to the cloud, CPU, RAM, and fast networks are plentiful. However, storage rises to the top of the list as the cause for Cloud application bottlenecks.

The Good, the Bad and the Ugly of EBS Backup to S3

The Good, the Bad and the Ugly of EBS Backup to S3

The Good, the Bad, and the Ugly of EBS Backup to S3

Developing a comprehensive strategy for backing up and restoring data is not a simple task. In some industries, regulatory requirements for data security, privacy, and records retention can be important factors to consider when developing a backup strategy.

EBS Advantages

Amazon Elastic Block Storage (EBS) provides many features such as high durability and reliability, encryption, provisioned IOPS, and point-in-time snapshots—amongst others. The built-in volume snapshot feature is a good option for backing up data.

Like all things AWS, Amazon has many options for creating backups. If you setup your EC2 instance to use EBS, you can simply create a snapshot of your volume from the AWS Console. These EBS snapshots are incremental backups that persist on Amazon’s S3. Incremental means that only the blocks that have changed since your last snapshot are saved.

Problem

EBS snapshots are really slick, but manually creating snapshots from the AWS Console isn’t a good solution if your goal is to have daily, hourly, or snapshots on a customizable timeframe.

In order for any backup to be useful, the backed-up content has to be crash consistent.  This means that write IOs are quiesced and write caches flushed.  Enterprise backup software has solved that problem for years, but direct use of EBS Snapshots for backup does not.

Alternative

You can create your own services using scripts and command line tools. But to do this you need to have the knowledge and experience of writing shell scripts, scheduling and executing. The snippet of script below provides details about the server in the snapshot.

code snippet

With ease of use in mind, the next version of SoftNAS will include a number of features that will allow for a faster and more convienient backup and archival process.

Learn more about Buurst SoftNAS.