Optimization Defined Storage (Storage Trends Part 1)
Recently I found myself engaged in a discussion on Twitter with @DuncanYB , @vcdxnz001 and @bendiq regarding Software Defined Storage (SDS) when I realized that our definitions might be approaching the issue from slightly different perspectives. Therefore I decided it might be a prudent exercise to first explore these definitions so that we can have a common foundation to build from in this series.
When I looked at the storage market I found 4-5 different categories of storage solutions that were all converging one way or another towards a common set of qualities and features. I’ll be exploring these solutions in detail in part 2, but is this common set SDS or is it something else (and do we care)?
Whenever someone says they are going to define something it has an air of pretentiousness about it. Hopefully this post will not come across as such as my intent is to provide a workable definition that can be used for a future post (Part 2). As an engineer who continues to evolve, I reverse the right to modify my definition in the future But for now let’s proceed….
Note: In the spirit of full disclosure my current employer is a NetApp partner.
Do Definitions Matter? Why Define SDS?
This really is a great question and you might be surprised at my conclusion – I don’t think the definition of SDS is terribly relevant for most of us. There is an academic definition which we could debate endlessly along with “what is cloud” and “what is the meaning of life?” before it would change again in about two years — yet this definition might only offer limited value to those making technology decisions and investments. In short, SDS does speak to a set of characteristics but not necessarily to value and efficiency in any given environment.
A few years ago I wrote a post which defined cloud computing as Abstraction, Automation and Agility. By Abstracting from the hardware level, we were able to provide a new API and control plane which could serve as a platform for Automation. Then with the proper use of Automation in the organization one could being to achieve measures of Agility.
With Software Defined Storage or SDS I think the same paradigm can apply. First we must offer a level of abstraction which enables us to do more with the storage. On top of this abstraction we need to add services that provide value and efficiency such as caching algorithms, dedupe algorithms, data protection (RAID or erasure coding), storage tiering, instant clones (no copy on write), snapshots, replication services and more. Then we must have an addressable API to leverage as a control plane from which we can manage, control and ultimately automate.
Now many might just focus on a definition of SDS that simply revolves around abstraction and a control plane (RESTful API, etc). For example does SDS require deduplication? I don’t think so, however I do think that this one of several key features that provides value that the industry is trending towards. Perhaps we need an expanded definition of SDS that focuses on value and efficiency – leveraging the SDS foundation to provide efficiency, agility and value.
Optimization Defined Storage
Optimization Defined Storage (ODS) then could be a definition of SDS that focuses on efficiency and value. ODS would be built upon an SDS foundation, and then enabled with additional capabilities to add value, efficiency and optimization such as:
- Deduplication (increases storage efficiency as well as performance benefits).
- Caching and Storage Tiering (including flash)
- Instant Clones and Snapshots (no copy-on-write)
- Efficient Replication
- Thin Provisioning
In an ODS solution, the software can work across many levels to optimize how data is compressed, deduped, cached and tiered. One example is Nutanix’s Shadow Clone feature which combines cloning and caching functions by distributing “shadow clones” of volumes to be used as cache by additional nodes.
Another example is VMware’s VVOL initiative which intends to somewhat shift the control plane such that the VM and VMDK characteristics will define the LUN rather than vice versa – as well as allowing the SAN to perform snap and clone operations against VMDK’s versus LUNs.
There’s much more that could be done in this SDS/ODS space (whatever you want to call it) and we haven’t gotten to object based storage yet. The bottom line is that the ODS definition is focused on leveraging the SDS foundation to provide value, efficiency and optimization.
Relative Levels of Hardware Abstraction
Some questions / arguments I’ve heard tossed around include “is Nutanix really SDS because they sell hardware” and this argument also extends to NetApp.
Nutanix provides many features I would qualify as ODS – dedupe, instant clones and more. These features are provided by the software but Nutanix has chosen to make their platform dependent on their hardware (which uses commodity components). Because the product is sold as proprietary hardware does that mean it’s not SDS?
I will argue that Nutanix is still SDS despite this. They made a business and product decision to offer their product as a hardware device to improve support, procurement as well as to facilitate scale-out. The SDS/ODS features provided are ultimately in the software and not in the hardware (commodity components).
Also what about NetApp’s ONTAP platform – does this qualify as SDS? I think it does. NetApp has provided features like compression, deduplication, thin provisioning, efficient snaps, clones and replication for some time now. Yes this baked into NetApp FAS arrays but it’s really the ONTAP software platform that’s doing everything. To take this a step further lets add the VSA (Virtual Storage Appliance) which allows you to front-end non-NetApp storage with the ONTAP platform. Now let’s also add ONTAP Edge which allows you to add ONTAP capabilities to storage using a VMware virtual appliance which is obviously hardware agnostic. When you consider this full context, I think we can reasonably conclude that the “magic” happens within ONTAP (software) and that this is indeed Software Defined Storage.
Let’s Get Out of The Weeds
We could debate various definitions of SDS/ODS forever as an academic exercise but this isn’t what I want to focus on. My primary goal was to share my definition of SDS (and ODS) that I could leverage in my next post. Part 2 will take a broader look at the storage industry and how various solutions are trying to approach SDS/ODS from various vantage points so that we can effecitvly compare them and understand how each provides value. Some solutions will be more effective in one environment versus another. Some solutions focus more on CAPEX while others focus more on OPEX. We’ve only talked about Nutanix and NetApp so far but I’d also like to talk about VMware VSAN, PernixData FVP, EMC ViPR and more. Hopefully this post will make more sense when I get around to writing Storage Trends Part 2.