Meeting Cloud File Storage Cost and Performance Goals – Harder than You Think

Meeting Cloud File Storage Cost and Performance Goals – Harder than You Think

Meeting Cloud Storage Cost and Performance Goals – Harder than You Think

According to Gartner, by 2025 80% of enterprises will shut down their traditional data centers. Today, 10% already have. We know this is true because we have helped thousands of these businesses migrate workloads and business-critical data from on-premises data centers into the cloud since 2013. Most of those workloads have been running 24 x 7 for 5+ years. Some of them have been digitally transformed (code for “rewritten to run natively in the cloud”).

The biggest challenge in adopting the cloud isn’t the technology shift – it’s finding the right balance of storage cost vs performance and availability that justifies moving data to the cloud. We all have a learning curve as we migrate major workloads into the cloud. That’s to be expected as there are many choices to make – some more critical than others.

Applications in the cloud

Many of our largest customers operate mission-critical, revenue-generating applications in the cloud today. Business relies on these applications and their underlying data for revenue growth, customer satisfaction, and retention. These systems cannot tolerate unplanned downtime. They must perform at expected levels consistently… even under increasingly heavy loads, unpredictable interference from noisy cloud neighbors, occasional cloud hardware failures, sporadic cloud network glitches, and other anomalies that just come with the territory of large-scale data center operations.

In order to meet customer and business SLAs, cloud-based workloads must be carefully designed. At the core of these designs is how data will be handled. Choosing the right file service component is one of the critical decisions a cloud architect must make.

Application performance, costs, and availability

For customers to remain happy, application performance must be maintained. Easier said than done when you no longer control the IT infrastructure in the cloud.

So how does one negotiate these competing objectives around cost, performance, and availability when you no longer control the hardware or virtualization layers in your own data center?  And how can these variables be controlled and adapted over time to keep things in balance? In a word – control. You must correctly choose where to give up control and where to maintain control over key aspects of the infrastructure stack supporting each workload.

One allure of the cloud is that it’s (supposedly) going to simplify everything into easily managed services, eliminating the worry about IT infrastructure forever. For non-critical use cases, managed services can, in fact, be a great solution. But what about when you need to control costs, performance, and availability?

Unfortunately, managed services must be designed and delivered for the “masses”, which means tradeoffs and compromises must be made. And to make these managed services profitable, significant margins must be built into the pricing models to ensure the cloud provider can grow and maintain them.

In the case of public cloud-shared file services like AWS Elastic File System (EFS) and Azure NetApp Files (ANF), performance throttling is required to prevent thousands of customer tenants from overrunning the limited resources that are actually available. To get more performance, you must purchase and maintain more storage capacity (whether you actually need that add-on storage or not). And as your storage capacity inevitably grows, so do the costs. And to make matters worse, much of that data is actually inactive most of the time, so you’re paying for data storage every month that you rarely if ever even access. And the cloud vendors have no incentive to help you reduce these excessive storage costs, which just keep going up as your data continues to grow each day.

After watching this movie play out with customers for many years and working closely with the largest to smallest businesses across 39 countries, At Buurst™. we decided to address these issues head-on. Instead of charging customers what is effectively a “storage tax” for their growing cloud storage capacity, we changed everything by offering Unlimited Capacity. That is, with SoftNAS® you can store an unlimited amount of file data in the cloud at no extra cost (aside from the underlying cloud block and object storage itself).

SoftNAS has always offered both data compression and deduplication, which when combined typically reduces cloud storage by 50% or more. Then we added automatic data tiering, which recognizes inactive and stale data, archiving it to less expensive storage transparently, saving up to an additional 67% on monthly cloud storage costs.

Just like when you managed your file storage in your own data center, SoftNAS keeps you in control of your data and application performance. Instead of turning control over to the cloud vendors, you maintain total control over the file storage infrastructure. This gives you the flexibility to keep costs and performance in balance over time.

To put this in perspective, without taking data compression and deduplication into account yet, look at how Buurst SoftNAS costs compare:

SoftNAS vs NetApp ONTAP, Azure NetApp Files, and AWS EFS

These monthly savings really add up. And if your data is compressible and/or contains duplicates, you will save up to 50% more on cloud storage because the data is compressed and deduplicated automatically for you.

Fortunately, customers have alternatives to choose from today:

  1. GIVE UP CONTROL – use cloud file services like EFS or ANF, pay for both performance and capacity growth, and give up control over your data or ability to deliver on SLAs consistently
  2. KEEP CONTROL – of your data and business with Buurst SoftNAS, and balance storage costs, and performance to meet your SLAs and grow more profitably.

Sometimes cloud migration projects are so complex and daunting that it’s advantageous to just take shortcuts to get everything up and running and operational as a first step. We commonly see customers choose cloud file services as an easy first stepping stone to a migration. Then these same customers proceed to the next step – optimizing costs and performance to operate the business profitably in the cloud and they contact Buurst to take back control, reduce costs, and meet SLAs.

As you contemplate how to reduce cloud operating costs while meeting the needs of the business, keep in mind that you face a pivotal decision ahead. Either keep control or give up control of your data, its costs, and performance. For some use cases, the simplicity of cloud file services is attractive and the data capacity is small enough and performance demands low enough that the convenience of files-as-a-service is the best choice. As you move business-critical workloads where costs, performance and control matter, or the datasets are large (tens to hundreds of terabytes or more), keep in mind that Buurst never charges you a storage tax on your data and keeps you in control of your business destiny in the cloud.

Next steps:

Learn more about how SoftNAS can help you maintain control and balance cloud storage costs and performance in the cloud.

Tired of Slow EFS Performance? Need Predictable NFS and CIFS with Active Directory?

Tired of Slow EFS Performance? Need Predictable NFS and CIFS with Active Directory?

Tired of Slow AWS EFS Performance? Need Predictable NFS and CIFS with Active Directory?

I’m Rick Braddy, Founder and CTO of Buurst. Like you, I have been in a situation where I was dependent upon Storage-as-a-Service (SaaS) and forced to live with slow performance, with no control or say in how NFS storage was structured or delivered.

In my case, it was on-premise on VMware, using NFS-mounted multi-user, shared Isilon® storage arrays in a private cloud, where we could not install our own storage appliances. We were stuck and ended up having to switch data centers to resolve this storage performance issue that plagued our business. It’s very frustrating when the users complain about AWS slow application performance, then the bosses get riled up, and there’s little to nothing you can do about it. So I feel your pain.

Today customers have choices, along with the freedom to choose. So if you’re experiencing slow or inconsistent performance with your current storage provider ( AWS EFS or other) and need answers, you came to the right place. On the other hand, if you haven’t yet deployed and are evaluating your options, this is a great time to get the facts before you find yourself stuck and have to switch horses later, as we see all too often.

We commonly see Amazon customers who have tried AWS EFS (and other options) that have hit slow performance, high costs, and other barriers that do not meet their business requirements. But it’s not just throttling from burst limits that slow performance. It’s the very nature of shared, multi-tenant filesystems that makes them prone to bottlenecks. The very nature of shared, multi-tenant filesystems makes them prone to bottlenecks, as shown in the following animated image.

A shared, multi-tenant filesystem must deal with millions of concurrent filesystem requests from thousands of other companies, who compete for filesystem resources within a Shared Data Service Lane. It’s easy to see why performance becomes slow and unpredictable in a shared, multi-tenant filesystem that must service so many competing requests. In such an environment, bottlenecks and collisions at the network and storage levels are unavoidable at times. 

Contrast that with having a Dedicated Data Express Lane like Buurst SoftNAS, your own exclusive Cloud NAS with its caching and resources focused on your data alone, where your applications’ needs are met consistently and constantly at maximum performance. 

Below we see the published AWS EFS burst limits. As applications use more throughput, burst credits are used. Applications that use too much I/O are penalized, and Amazon EFS throttles performance, one of the potential causes for slow AWS EFS performance.

Amazon EFS is a shared, multi-tenant filesystem, subject to burst limits and multi-tenant bottlenecks. SoftNAS is a dedicated, high-performance filesystem with no burst limits and access to full AWS EBS performance, regardless of how much storage capacity you use. This is the first core difference between the two – shared vs. dedicated infrastructure. 

There’s a place in the market where each type of filesystem fits and serves its individual purpose. The question is, which approach best meets your application and business needs? 

Performance results can also vary based upon many other factors and each application’s unique behavior. Therefore, we recommend customers run benchmarks and test their applications at scale with simulated user loads before finalizing the components that will be placed into production as part of their cloud stack. This is often best accomplished during a proof-of-concept stage before applications go into production. Unfortunately, customers often discover too late in the process that they have performance issues and are in urgent need of a solution. 

Since 2013, SoftNAS has provided thousands of customers with high-performance, dedicated Cloud NAS solutions. We originally pioneered the “cloud NAS” category as the #1 Best-selling NAS in the Cloud. By coupling industry standards like ZFS on Linux with our patented cross-zone high-availability and high-performance AWS EBS and S3-backed storage technologies, SoftNAS has consistently provided customers with the most flexibility, choice, and value business-critical NFS and CIFS with Active Directory for AWS®. 

What do customers say about AWS EFS Slow Performance?

EFS has been in the marketplace for several years now, so there’s been plenty of time to hear from customers about their experiences. It’s helpful to consider what customers see and report.

Here are some examples of comments our Support Team sees below.

You can search for “AWS EFS slow performance” to do your own research.

“We want to migrate about 5 percent of our 150+ Centos based servers from on-prem to AWS and need NFS and CIFS but AWS does not offer EFS in Canada.”
“Hi we were suggested to contact you guys from our AWS rep in helping us deal with the issues EFS has due to the slowness of I/O … our AMI’s currently mount our EFS mount point and due to the nature of the software we are using it tends to use a lot of file_get_contents, is_dir functionality which is very slow with EFS compared to a standalone server with SSD. Is this something you guys can help us with and get it up and running in our VPC?”
“We host multiple Magento based websites for our clients in AWS and looking for a solution that we can use to replace AWS EFS as it is very slow with many small files and it is not yet available in many regions.”
“I am trying the trial of SoftNAS in our AWS cloud for potential use for SMB/CIFS & NFS to be used as a replacement for EFS. We have reached the limit of the capabilities of EFS and so looking into other products.”
“I need a demo by which I can decide the SoftNAS is suitable AMI for my production use case. This is very high priority as I need to implement this for persistence storage for Kubernetes as EFS is not available in Mumbai region.”

“Need ability to use S3 as the backend storage instead of EBS/EFS as they are much too expensive for us.”

The above is just a sample of what we typically hear from customers after evaluating or deploying EFS. EFS is a solid NFS-as-a-service that is meeting many customers’ cloud file storage needs. Some customers, however, need more than a multi-tenant, the shared filesystem can deliver. 

What’s needed is a full-featured, enterprise-grade Cloud NAS that delivers: 

  • A tunable level of dedicated, predictable performance 
  • Guaranteed cross-zone high-availability with an up-time SLA 
  • Storage efficiency features (e.g., compression, deduplication) that mitigate the costs of cloud storage 
  • Support for EBS (Provisioned IOPS, SSD, magnetic) and highly durable, low-cost S3 storage 
  • A full-featured POSIX-compliant filesystem with full NFS that supports all the expected features at scale 
  • CIFS with Active Directory integration and full ACL support for Windows workloads 
  • Ability to scale storage into the hundreds of terabytes or petabytes while maintaining performance consistency at a reasonable cost level that doesn’t break the bank or derail the project 
  • Storage auto-tiering that maximizes performance while minimizing storage costs 
  • Data checksumming and other data integrity features that verify retrieved data is accurate and correct 
  • Instant storage snapshots and writable clones that do not involve making expensive, time-consuming copies of data for rapid recovery and previous versions 
  • Integrated data migration tools that make lifting and shifting production workloads into the cloud faster and easier 

    How can SoftNAS address all these issues better than AWS EFS?

    Simple. That’s what it was designed to do from the beginning – provide cloud-native, enterprise-grade “cloud NAS” capabilities in a way that’s designed to squeeze the maximum performance from EC2, EBS, and S3 storage. And cloud storage and data management is all we have done, every day, since 2013, so we are among the world’s top cloud storage experts.

    Below we see SoftNAS running as a Linux virtual machine image on EC2. First, EBS and S3 cloud storage is assigned to the SoftNAS image that’s been launched from AWS Marketplace or the AWS Console. Next, that storage is aggregated into ZFS filesystems as “storage pools”. Then, these pools of storage can be thin-provisioned into “volumes” that are shared out as NFS, CIFS, AFP, or iSCSI for use by applications and users.

    SoftNAS leverages arrays of native EBS block devices and S3 buckets as underlying, scale-out cloud storage. Data is striped across arrays of EBS and S3 devices, thereby increasing the available IOPS (I/O per second).  To add more performance and storage capacity, add more devices at any time. 

    SoftNAS runs on a dedicated instance within your AWS account, so everything is under your control – how many and what types of storage devices are attached, how much RAM is available for caching, how much direct-attached SSD to use (based on instance choice), along with how much CPU to allocate to compression, deduplication, and performance. Incidentally, compression and deduplication can reduce your actual storage costs by up to 80%, depending upon the nature of your data (you don’t get any storage efficiency features with EFS). 

    And because ZFS is a copy-on-write filesystem and SoftNAS automatically creates storage snapshots for you based on policies, you always have previous versions of your data available at your fingertips, in case something ever happens. You can quickly go back and recover your data without rolling a backup restore (you don’t get any storage snapshots or the ability to recover with EFS). 

    ZFS provides complete POSIX-compliant semantics as the SoftNAS filesystem. CentOS Linux provides full NFS v4.1 and NFS v3 filesystem semantics, including native file_get_contents, is_dir functionality. And because SoftNAS is built upon Linux, you get native NFS support with no functional limitations. 

    SoftNAS supports Windows CIFS with SMB 3 protocol, with Active Directory integration for millions of AD objects. All of these details matter when you’re deploying applications that rely on proper locking, filesystem permissions, full NFS semantics, NTFS ACLs, etc.   

    SoftNAS delivers reliable performance with two levels of caching – RAM and direct-attached SSD for massive read-caching, which together provide both predictable performances that YOU control and performance increases. And later, if you need even more performance, you can upgrade your EC2 instances to add more IOPS and greater throughput with more network bandwidth. 

    SoftNAS keeps you in control instead of turning your data over to the multi-tenant EFS filesystem and living with its various limitations. 
    Cost Management
    • Highest Performance
    • Most Expensive Storage
    • Most Active Data
    Cost Management
    • Moderate Performance
    • Medium Expense Storage
    • Less Active Data
    Cost Management
    • Lowest Performance
    • Least Expensive Storage
    • Least Active Data
    • Highly-durable S3 object storage

    Using the combination of three data efficiency methods, SoftNAS can reduce cloud storage costs more than any other alternative available today. No other vendor offers these kinds of cloud storage cost savings while maximizing performance and providing the level of data protection and control that SoftNAS delivers. Additionally, by working with our Sales team, you can get volume discounts for larger capacity implementations to save you even more. 

    You may wonder “who else uses SoftNAS?”  Look at who’s chosen SoftNAS for their business-critical cloud application deployments below. This is just a sample of recognizable customers across 36 countries globally who trust their business data to SoftNAS on a 24 x 7 x 365 basis. 

    Logo

    You will find that customers say that Buurst provides the best technical support and cloud expertise available, along with its world-class SoftNAS Cloud NAS product. 

    By the way, over the years, we have developed an excellent relationship with AWS. AWS suggested that we create this and other comparison material to help customers understand the differences between EFS and SoftNAS. We are great partners with one common objective – to make our customers successful together. 

    How did this come about?

    I started Buurst in 2012 as a former traditional storage customer, so I know what you’re dealing with and what you need and want in a storage software product and partner. I believed in AWS and the future of the public cloud before it was apparent to everyone that it would be the next big thing. As a result, we were early to market in 2013 and have many years of head start vs. the alternatives. 

    Today, I ensure our team delivers excellence to our customers every day. Our management team ensures we hire, train, and provide the best technical support resources available globally. In addition, we bring years of cloud storage and multi-cloud performance experience to the table to ensure your success. 

    Customers always come first at Buurst. Our cloud experts are here to help you quickly address your most pressing cloud data management needs and ensure your success in the cloud – something we’ve done for thousands of customers since 2013. You can count on Buurst to be there with you. We help customers with all issues around networking, security, VPCs, and other areas all the time. 

    For example, our networking and storage experts have helped customers deploy thousands of VPCs in just about every imaginable security and network topology.

    Check out our prevalentBest Practices Learned from 1,000 VPC Configurations. This is just one of many examples of how Buurst helps its customers make the journey to the cloud faster, easier, and better than going it alone. 

    What regions is SoftNAS available in?

    Our products are available in all regions globally. We typically add support for new regions within 60 days or less (as soon as our QA team can test and certify the product in the new region). This means you can count on Buurst to be where you need it to be when you need it there. 

    What are my options to get started?

    There are many ways to get started with Buurst. If you need help fast, I recommend you reach out to our solutions team or call us at 1-832-495-4141  to see how Buurst can address your cloud file storage and data management needs. 

    Need help migrating your data from on-premise or EFS?

    SoftNAS now includes automated data migration capabilities that take the pain out of migration. We can also assist you in the planning and actual migration.  Learn more.

    Prefer some help getting started?

    If you’d prefer to see a demo first or get some assistance evaluating your options further, schedule a demo and free consultation with one of our experts now and get your questions answered one-on-one. 

    Need more information before proceeding?

    That’s understandable. Here’s some additional helpful links below to more information:

    Remember. We are here and ready to help. You don’t have to go it alone anymore– reach out, and let’s schedule a time together to explore how our cloud experts can be of service and quickly address your needs – at no cost and no obligation.Click here to schedule a free consultation now. 

    Thank you for visiting Buurst. Here’s to your cloud success! We look forward to being of service. 

    Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS: Which Performs Best?

    Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS: Which Performs Best?


    Comparing Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS. Great read from Nathan Wilkerson, Cloud Engineer with Metal Toad around NFS performance on AWS based on the upcoming Amazon EFS (Elastic File System). As Amazon EFS is not generally available, this is a good early look at a performance comparison among Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS.

    According to Nathan:

    SoftNAS had by far the best performance. Its cost is higher for low capacities, but will be important when latency is paramount.  At higher capacities the benefits of SoftNAS and lower overall cost are a clear winner, I would consider SoftNAS.

    SoftNAS spends a lot of time testing its products, reviewing customer feedback and welcomes Nathan’s performance results.

    But is SoftNAS actually more expensive? NO. Let me explain.

    1) 100GB works well for performance tests but is not a realistic capacity point for most customers real-world use or a pricing analysis.  On average, SoftNAS customers tend to use  10TB – 20TB, and frequently surpass 100TB.

    2) SoftNAS is less expensive than AWS EFS at 6TB and is dramatically more cost effective at higher capacities.

    3) Not only is SoftNAS cost effective at real-world capacities (even with EBS pricing), you get all the benefits of an enterprise-class NAS that is available in all AWS regions and which includes snapshots, de-duplication, data protection, multi-tier caching and compression.

    Pricing: 100GB

    The configuration in Nathan’s tests used 100GB with a pair of instances for High Availability. SoftNAS offers 100GB storage at no charge for each SoftNAS instance when launched as Community AMIs. See below the pricing for each vendor at 100GB usable capacity for EBS General Purpose SSD, per month (as tested in the Metal Toad benchmarks).

    Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS table

    It’s also worth noting that for the pricing comparisons in Nathan’s blog he used 200GB for SoftNAS compared to 100GB for EFS and Gluster. SoftNAS offers stack prices for both small and medium EC2 instances.  The medium instance will provide a better performance, but speed is not always the objective.  Offering the flexibility to configure based on what you want to accomplish is a cornerstone of the SoftNAS approach.

    Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS pricing

    It’s also worth noting that for the pricing comparisons in Nathan’s blog, he used 200GB for SoftNAS compared to 100GB for EFS and Gluster. SoftNAS offers stack prices for both small and medium EC2 instances. The medium instance will provide a better performance, but speed is not always the objective. Offering the flexibility to configure based on what you want to accomplish is a cornerstone of the SoftNAS approach.

    Pricing: scaling up to 20TB with SSD

    Lets look at the comparisons as you scale out the capacities up to 20TB with both backed by SSD. The following table shows the cost comparison of 20TB for SoftNAS with High Availability and AWS EFS.

    The following graph compares SoftNAS with HA and AWS EFS for capacities from 200GB through 20TB. At 20TB, SoftNAS price per GB is $.0.13 vs EFS at $0.30.  The conclusion is that EFS is less at low capacity due to lack of EC2 and software costs, whereas when capacity increases the costs of storage outweighs the overhead costs. The breakeven point is 6TB and SoftNAS is progressively cheaper as capacity grows. Additionally, you get all the benefits of an enterprise-class NAS that is available in all AWS regions and which includes snapshots, de-duplication,data protection, multi-tier caching and compression.

    Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS performance

    Also, it’s worth mentioning that SoftNAS offers a high amount of configurability. You can choose between performance vs. cost, options for L1 and L2 read cache, options for write cache, and a ton of other useful features, including CIFS with Active Directory integration.

    Finally, it’s worth noting that the flexibility of SoftNAS means you can choose not only SSD-backed storage, but also mix in less expensive alternatives like EBS Magnetic ($0.05/GB) and highly-durable S3 backed object storage ($0.03/GB).

    So who performs best when comparing Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS?

    SoftNAS delivers the most storage flexibility and most powerful feature set for protecting business data in the AWS cloud, and provides great price/performance. Best of all, SoftNAS is available on the most popular cloud and premise-based platforms today (AWS, Azure, CenturyLink, and VMware vSphere), keeping you in control of your business-critical data and your business.

    Why not try SoftNAS and see how it performs in your environment?

    Amazon EFS vs. GlusterFS vs. SoftNAS Cloud NAS trial

    Why SoftNAS?

    SoftNAS provides direct connectivity to cloud storage providing a private connection to resources owned by your organization. This eliminates the managed storage problems of inconsistent performance by noisy neighbors. In addition, SoftNAS adds value by placing data in the most appropriate location for increased application performance.