Do CIOs and CFOs Know Big Data’s Dirty Little Secret? By “Big Data,” I’m referring to the storage appliances businesses use to store and manage their data today (not business intelligence, databases, and applications that run atop these storage appliances).

With 60% or more of typical virtual computing, IT project being comprised of data management hardware and software, few stop to ask “what are we getting for our money?”

As the ones responsible for P&L, CFOs must have asked their CIO this question before. To justify the large expenditures involved, CIOs might say:

– We need to run application XYZ that we’re rolling out to support the business
– We need to protect our corporate data and ensure we never lose any critical data
– We need to ensure our business is always up and running
– It’s part of every IT project – a safe place to manage our data

I’m sure the actual explanations vary wildly… but one must wonder if most CFO’s (or CIO’s for that matter) know the other questions they should be asking, like:

“Okay, I understand WHY we need the Big Data storage appliances now… what are the alternatives to paying $10,000 per Terabyte (TB)? I’m obviously not a computer expert like you guys in IT, so help me understand the price difference between disk drives costing less than $500 per Terabyte on Amazon.com, and those costing $10,000 per TB?”

Good question! The answer to the CFO’s/CIO’s question and Big Data’s dirty little secret is:

“The price difference is the NAS software inside the Big Data appliances.”

The ‘storage operating system’ inside the Big Data appliances is very specialized and provides the performance, caching and data protection needed to run the business. At least that’s what the Big Data vendors would have you believe.

You’re paying for a combination of proprietary hardware (the appliances) and the software inside. The appliances and software are purpose-built for doing just one thing – managing corporate data. The $8,000 per TB premium is primarily for the software and proprietary hardware. You also get redundant data paths that probably increase the reliability of the system by a few percentage points (cables and controller cards don’t fail very often).

So where does that $8,000 per TB go? It goes to pay for the SG&A (sales, service, advertising, promotion, accounting, big buildings, big salaries, and bonuses, etc), plus some R&D, that is associated with operating billion-dollar Big Data franchises today.

When I was CTO of a start-up in the hosted virtual desktop marketplace, we discovered just how critical high-performing, reliable storage can be in a virtual computing environment (in our case, VMware). Turns out, it’s especially critical for VDI, where end-users see every little “burp” that storage and networks cause on their desktops.

When you have high-performance storage needs (and who doesn’t need reliability and storage performance these days?), it’s a lot easier to look past the high prices that Big Data is commanding and just pay it. The problem is, unless you’re Exxon Mobil or a huge corporation, you may not have the luxury of just overlooking these costs. In our case, the cost of goods associated with using Big Data storage appliances to power our IT needs to mean the difference between being profitable or losing money as a cloud service provider.

So the right answer as to whether a CFO and CIO should view Big Data as a “necessity” vs. a “luxury” item depends on whether you’re a large enterprise vs. a small to medium business (or a government agency), where every dollar counts and saving money can sometimes even mean the difference between profitability or survival in today’s tough economic times.

When you use off-the-shelf server hardware (the same servers you use for VMware, SQL Server, Oracle, Microsoft Exchange, etc. today), plus the Virtual NAS software that will run on that server, and a few hours of labor to create a “good enough” storage solution, you can do the same job for $2,000 per TB or less.

For example, using SoftNAS plus the popular HP DL 380 or Dell PowerEdge server equipment, the costs are typically in the $2,000 per TB range for a complete, high-performance storage solution…. Saving $160,000 on a typical 20 TB configuration.

Knowing this, perhaps CFO’s should re-phrase the question to:

“Why should we pay an $8,000 per TB premium for storage? Is it really necessary for this project?”