The Open vStorage future looks bright

Jason Ader, analyst at the William Blair & Company, released a new research document: The State of Storage – a 2014 Update. In this detailed report, which is a must-read for everyone in the storage industry, he discusses the biggest short and long term headwinds for traditional storage vendors. Some of these headwinds are caused by newcomers in the storage industry. He estimates that these newcomers will grow with a rate of around 40%, which will cause the combined impact to reach almost 10% of industry sales in 2014. These headwinds can not be ignored in terms of revenue or growth so it is worthwhile to discuss these headwinds in more detail and explain where Open vStorage fits in.

  • Object-Based Storage gets a second wind as migration of data to the cloud (private or public) is on the rise. Use cases and applications adjusted for Object Stores are still limited but on the rise. With Open vStorage you can turn your Object Store (Swift, Cloudian, …) into a block device for Virtual Machines and turn it into a high performance, distributed, VM-centric storage platform.
  • Emergence of Flash-Centric Architectures: Open vStorage leverages flash inside the ESXi or KVM hosts to accelerate reads and writes. It brings together the benefits of storage local to the server and external scalable storage by leveraging SSDs and PCIe flash technology inside the server for acceleration.
  • Software-defined Storage (SDS) will, according to report, “have the largest impact on the storage market in the long term as it will drive ASPs down and push toward the commoditization of specialized storage hardware platforms.” This trend is clearly to be seen in the rise of open-source file systems such as GlusterFS and Ceph and VMware’s vSAN. Open vStorage is a software-defined storage layer which brings storage intelligence into software that runs closer to compute with the ability to use multiple storage back ends in a uniform way.

Open vStorage 1.1

The Open vStorage team is on fire. We released a new version of the Open vStorage software (select Test as QualityLevel). The new big features for this release are:

  • Logging and log collecting: we have added logging to Open vStorage. The logs are also centrally gathered and stored in a distributed Search Engine (Elasticsearch). The logs can be viewed, browsed, searched and analyzed through Kibina, a very nice designed GUI.
  • HA Support: Open vStorage does with this release not only support vMotion but also the HA functionality of VMware. This means that in case an ESXi Host dies and vCenter starts the VMs on another Host, the volumes will automatically be migrated along.

Some small feature that got into this release:

  • Distributed filesystem can now be selected as Storage Backend in the GUI. Earlier you could select the file system but you could not extend it to more Grid Storage Routers (GSR). Now you can extend it across more GSRs and do vMotion on top of f.e. GlusterFS.
  • The status of VMs on KVM is now updated quicker in the GUI.
  • has now an option to specify the version you want to install (run -v ‘version number’)
  • Under administration there is now an About Open vStorage page displaying the version of all installed components.

In addition, the team also fixed 35 bugs.

A distribution center for Virtual Machine data

walmart_distribution_centerAt open vStorage we quite often get the question: “Ok, you are a Grid Storage Router but what is that exactly?”

To explain what a Grid Storage Router is and why it is essential in a virtual environment, I’d like to make the analogy with Walmart, the largest retailer in the world. You can compare Open vStorage to the grid of distribution centers of Walmart. Walmart has 42 regional U.S. distribution centers with over 1 million square feet. In total these distribution centers have more than 12 miles of conveyor belts to move 5.5 billion cases of merchandise.

Sam Walton, the founder of Walmart, realized very quickly that in order to sell a lot of goods the company had to develop a multilayered distribution system and identified logistics as its primary expertise. With multiple stores, they could have opted to arrange that goods go directly from manufacturer to the end-shops but the Walmart management quickly realized that having a central hub in between these 2 makes sense. Instead of having their more than 100,000 suppliers dropping of their goods at the door of the 5,000 stores, they ask the suppliers to drop their goods off at one of the regional distribution centers. Having only a limited amount of distribution centers allowed the trucks with supplies to be stocked to the maximum level and hence optimizing round-trips and making the best use of the available truckload capacity.

The distribution centers have grown from being just a temporary storage location, to a fully automated high-tech center where every move is orchestrated and tracked. On top of just temporarily storing goods, the distribution centers also offer additional services such as splitting up large volumes into smaller parcels, repackaging and keep track of the stock.
Typically one distribution center can cater the needs of multiple stores but there is a limit to this capacity. When needed a new distribution center, which typically follows a blueprint, will open to relieve the pressure and to prepare for yet more stores. This allows to scale-out the stores without bringing the chain of supply in danger.

Just as Walmart considers their distribution centers to be as important as their stores, you should attribute Open vStorage with the same importance. Open vStorage is the hub between one or more Storage Backends (Swiftstack, Ceph, …) and your Virtual Machines. Like distribution centers Open vStorage doesn’t only connect suppliers (Storage Backend) and stores (Virtual Machines) with each other in an optimized fashion to build scale-out Virtual Machine storage but it also brings additional value to the table. With its VM-centric architecture it allows efficient unlimited snapshotting, VM replication*, compression and encryption at the VM level*.

Having this grid of Storage Routers, where one can take over from another, it allows to improve the reliability and as all metric of these Storage Routers and the Virtual Machines are tracked, the troubleshooting in case of an issue becomes much easier.

Where distribution centers work with different types of stores (Walmart Supercenters or Walmart Express), Open vStorage is hypervisor agnostic and can handle both VMware ESXi and KVM workloads. For both hypervisors every write gets accelerated on SSD or PCI flash cards and reads are optimized by deduplicating identical content and hence making the best use of the available fast storage.

Just as the distribution centers are key in the success of Walmart, Open vStorage is key in building out scalable, high performance, VM-centric storage.

* Planned for Q3 2014

White Paper: Turning Object Storage into Virtual Machine Storage

In this White Paper “Turning Object Storage into Virtual Machine Storage” we discuss Object Storage as storage solution for Virtual Machines. Due to technical hurdles it is impossible to run Virtual Machines directly from an Object Store. The main issues are eventual consistency, performance and latency and different management paradigms. A distributed file system is also no fit for the job.

To turn Object Storage into primary storage for hypervisors, the solution must be especially designed for Virtual Machines and take a radical different approach compared to existing storage technologies. Open vStorage takes this different approach and is designed from the ground up with Virtual Machines and their performance requirements in mind. Open vStorage is the layer between the hypervisor and Object Store and turns the Object Store into a high performance, distributed, VM-centric storage platform.

You can download the full white paper here.

Open vStorage 1.0.3 Beta

We couldn’t wait until the end of April before pushing out our next release. That is why the Open vStorage Team decided to release today (April 17 2014) a small version with following set of features:

  • Remove a vPool through the GUI: in version 1.0.2 it was already possible to create a vPool and extend the vPool across Hosts through the GUI. To make the whole vPool management process complete and more user friendly, we have added the option to remove a vPool from a Host. This is of course only possible in case there are no vDisks being served from the vPool to a vMachine on that Host.
  • New version of Arakoon: Arakoon 1.7, the distributed key-value store that guarantees consistency above anything else, integrated in Open vStorage had some major rework done. This release also fixes bugs we encountered.
  • Some small improvements include adding an NTP server by default to the Host and displaying the different used ports on the VSA detail page.

The version also includes bug fixes:

  • Issue with connecting to SAIO via S3.
  • Footer displaying incorrect values.
  • Update sshd configuration to avoid client to send its own specific LOCALE settings.
  • Shutting down first node in 4 node setup caused ovs_consumer_volumerouter to halt on node 2.
  • KVM clones are now no longer automatically started.
  • Open vStorage GUI sometimes didn’t list a second VM.
  • Clone from vTemplate on KVM does not work in the absence of “default” network on Archipel.
  • Restrict vTemplate deletion from the Open vStorage GUI when a child vMachine is present.
  • Add vpool without copying ceph.* files now gives error.

To give this version a try, download the software and install with Quality Level Test.

Why both converged storage and external storage make sense

Today the peace in storageland has been disturbed by a post of Storage Swiss, The Problems With Server-Side Storage, Like VSAN. In this post they highlight the problems with VSAN. At the same time Coho Data, developer of flash-tuned scale-out storage appliances, released a post about why converged storage, such as VSAN, only seems to be usefull in a niche area. You can read the full post here. VMware had to respond these “attacks” and released a post countering the problems raised by Storage Swiss. As with every black-and-white story both sides have valid points, so we took the liberty to summarize these valid points. On top, we believe both sides can play a role in your storage need and that is why we believe Open vStorage, as only solution in storage land, is the solution by allowing to mix and match converged storage and external storage.

Valid points for the converged stack:

  • VSAN and other converged software stacks typically can run on almost any hardware as they are software based.
  • Pricing for a software based solution to give the same reliability as a hardware based solution can be a fraction of the cost and will become commodity over time. Take as example traditional storage redundancy and fault tolerance which is typically implemented via dual controller hardware systems. Software based techniques provide the same level of reliability at much lower costs and much better scaling.
  • Converged stacks are VM-centric, treating a single volume as a LUN. This allows for flexibility and cost reduction. Test tiers can only have a limited or no protection and important volumes might be replicated multiple times across the storage pool and run with best performance.
  • The network in a converged stack is also less complex as only the hosts need to be connected. With external storage appliances you also need to take the dedicated storage network into account.
  • Easy to setup and manage, even for less experienced IT support. This can well be the case in a branch office.
  • Running storage as close as possible to the compute power makes sense. If not, we all would be using Amazon S3 to power our VM storage needs.

Valid points for the external storage stack:

  • External storage arrays take resource intensive tasks upon themself. It allows for example processor and network resources to manage replication and recovery.
  • External storage arrays allow to be linked to different hypervisors while converged infrastructure solutions are tightly linked to the hypervisor. For example VSAN can only be used with the VMware ESXi hypervisor.
  • The storage network can be completely separated from the computer and outbound traffic.
  • Better usage of flash drives as they are shared between multiple hypervisors and writes don’t have to be mirrored to a second appliance.

The Open vStorage view:
Open vStorage believes there isn’t a ‘One Size Fits All’ solution to storage demands. Today, specific applications and functionalities demand specific storage. That is why Open vStorage is configured to work as a truly open solution and works with any storage backend. Open vStorage provides the flexibility of using an existing distributed file system, external storage appliances and even object stores. This allows for more flexibility but at the same time keeps the ability to protect existing investments.

We are firm believers in the approach taken by VMware vSAN where software running on a cluster of machines provides you a reliable storage pool and where the storage intelligence is moved closer to the compute layer. However, we believe this is too important a piece of the virtualization stack for a proprietary solution that is either hypervisor specific, hardware specific, management stack specific and storage back-end specific. Our goal behind Open vStorage is not only build something open and non-proprietary but something modular enough which allows developers and hardware manufacturers to innovate on top of Open vStorage. We believe with contributors from the open source community Open vStorage can become a far superior compared to proprietary offerings whether these be hardware or software based.

To conclude Open vStorage:

  • Leverages server based flash and storage back-ends to create shared virtual machine storage
  • Redundant, shared storage without the costs and scaling problems of dual controller systems
  • Works with any storage back-end, including filesystems, NAS, S3 compatible object storage, …
  • Works with KVM and VMware
  • Delivered under the Apache License, Version 2.0

VM-Centric Storage

We’ve designed Open vStorage to be VM-centric and a lot of the features we create over the next few months will leverage the VM-centric architecture. The idea of VM-centric storage has been implemented by new age storage companies such as Tintri, VMware (with vSAN) and us when designing our products.This approach is a huge departure from the “traditional” LUN based approach taken by SAN vendors. This brings the question – “What exactly do we mean by VM-centric architecture?”.

Managing VMs and not storage
The key feature with VM-centric storage is that IT administrators manage virtual machines and not storage. All storage actions are taken on a per virtual machine basis rather than having to understand LUNs, RAID groups, storage interfaces, etc. With Open vStorage any IT administrator having knowledge on managing virtual machines should be able to manage it rather than having to get into the complexities of storage and storage networking specifics. A simple example could be cloning. With Open vStorage an administrator can use a VM template to create clone(s) while a traditional storage administrator would need to clone a LUN and then export a particular VM to achieve the same result.

Storage Policies by Virtual Machines
VM-centric design allows for storage policies to be defined on a per virtual machine basis. A few examples of this is as follows:

  1. Replication from one storage cluster to another, can be done on a per virtual machine basis. Moreover, different replication schedules can be used for different virtual machines – as an administrator you may choose an important VM to be replicated every 30 minutes while a not so important VM to be replicated once a day. Moreover, different virtual machines could be replicated to different clusters.
  2. Data retention policies can be defined on a per virtual machine basis. Open vStorage takes snapshots of virtual machines every hour and soon we will be adding the capability whereby an administrator can set the retention policies for these snapshots differently for different virtual machines. For example for one virtual machine an administrator may set retention for 7 days while for another virtual machine it could be 3 years.
  3. With Open vStorage one can use multiple back-ends such as a NAS, file system, object store. We call these back-end storage systems as vPools. With Open vStorage one can have multiple such vPools per host. The administrator can then select vPool on which the data for a particular virtual machine is stored on again on a per virtual machine basis. Hence, an example would be a VM that needs fast I/O could use a vPool that provides fast throughput while a VM requiring not so fast I/O could be stored on a vPool that has a slower throughput.
    We feel that VM-centric storage is here to stay, and that over a period of time the idea of requiring separate storage management processes, software and people would become obsolete. Our engineers are not stopping at the above mentioned VM-centric features, and our key R&D staff are working on capabilities such as encryption (on a per VM basis), dynamic QoS (tuned per VM) and many more exciting innovations.

Upcoming release (End April)

The Open vStorage tries to release a new version once month. Since version 1.0.2 was released end of March, the next (big) release release can be expected by the end of April. This release, Open vStorage 1.1 is planned to have following content:

  • HA Support: in version 1.0.2 you need to set a host offline to be able to steal volumes away from the VSA on that host. In the next release we would allow to steal volumes automatically when a host is down. This means that Virtual Machines which are migrated by the hypervisor to another node will automatically be stolen by the new hosts. Keep in mind that in version 1.0.2, live migration is f.e. already supported. As long as both hosts are online, volumes can move withouth downtime between VSAs.
  • Logging: in version 1.0.2 the logging was very minimal. In the 1.1 version we will make sure all components log into a central location for the whole cluster. This should make troubleshooting much more user-friendly.

That is the plan for the next big release. In case you want to see things added to the roapmap, let us know!

Open vStorage 1.0.2 Beta

Open vStorage supports KVM as Hypervisor.

Swift Support
Open vStorage supports Swift (SAIO – Swift All In One, Swiftstack, …) as Storage Backend.

Configuration of a vPool through the GUI
You can now add a new vPool through the GUI to a VSA. In case the vPool is a not a local filesystem, you can also stretch it across multiple VSAs.

Protection against moving a vDisk to a VSA where it can’t be served
Each VSA can only serve only a limited amount of vDisks. In case a vDisk is moved between VSAs, f.e. due to a vMotion, the process will first check if there are enough resources available to serve the vDisk. In case there are not enough free resources on the destination VSA, the handover will be aborted.