A distribution center for Virtual Machine data

walmart_distribution_centerAt open vStorage we quite often get the question: “Ok, you are a Grid Storage Router but what is that exactly?”

To explain what a Grid Storage Router is and why it is essential in a virtual environment, I’d like to make the analogy with Walmart, the largest retailer in the world. You can compare Open vStorage to the grid of distribution centers of Walmart. Walmart has 42 regional U.S. distribution centers with over 1 million square feet. In total these distribution centers have more than 12 miles of conveyor belts to move 5.5 billion cases of merchandise.

Sam Walton, the founder of Walmart, realized very quickly that in order to sell a lot of goods the company had to develop a multilayered distribution system and identified logistics as its primary expertise. With multiple stores, they could have opted to arrange that goods go directly from manufacturer to the end-shops but the Walmart management quickly realized that having a central hub in between these 2 makes sense. Instead of having their more than 100,000 suppliers dropping of their goods at the door of the 5,000 stores, they ask the suppliers to drop their goods off at one of the regional distribution centers. Having only a limited amount of distribution centers allowed the trucks with supplies to be stocked to the maximum level and hence optimizing round-trips and making the best use of the available truckload capacity.

The distribution centers have grown from being just a temporary storage location, to a fully automated high-tech center where every move is orchestrated and tracked. On top of just temporarily storing goods, the distribution centers also offer additional services such as splitting up large volumes into smaller parcels, repackaging and keep track of the stock.
Typically one distribution center can cater the needs of multiple stores but there is a limit to this capacity. When needed a new distribution center, which typically follows a blueprint, will open to relieve the pressure and to prepare for yet more stores. This allows to scale-out the stores without bringing the chain of supply in danger.

Just as Walmart considers their distribution centers to be as important as their stores, you should attribute Open vStorage with the same importance. Open vStorage is the hub between one or more Storage Backends (Swiftstack, Ceph, …) and your Virtual Machines. Like distribution centers Open vStorage doesn’t only connect suppliers (Storage Backend) and stores (Virtual Machines) with each other in an optimized fashion to build scale-out Virtual Machine storage but it also brings additional value to the table. With its VM-centric architecture it allows efficient unlimited snapshotting, VM replication*, compression and encryption at the VM level*.

Having this grid of Storage Routers, where one can take over from another, it allows to improve the reliability and as all metric of these Storage Routers and the Virtual Machines are tracked, the troubleshooting in case of an issue becomes much easier.

Where distribution centers work with different types of stores (Walmart Supercenters or Walmart Express), Open vStorage is hypervisor agnostic and can handle both VMware ESXi and KVM workloads. For both hypervisors every write gets accelerated on SSD or PCI flash cards and reads are optimized by deduplicating identical content and hence making the best use of the available fast storage.

Just as the distribution centers are key in the success of Walmart, Open vStorage is key in building out scalable, high performance, VM-centric storage.

* Planned for Q3 2014

White Paper: Turning Object Storage into Virtual Machine Storage

In this White Paper “Turning Object Storage into Virtual Machine Storage” we discuss Object Storage as storage solution for Virtual Machines. Due to technical hurdles it is impossible to run Virtual Machines directly from an Object Store. The main issues are eventual consistency, performance and latency and different management paradigms. A distributed file system is also no fit for the job.

To turn Object Storage into primary storage for hypervisors, the solution must be especially designed for Virtual Machines and take a radical different approach compared to existing storage technologies. Open vStorage takes this different approach and is designed from the ground up with Virtual Machines and their performance requirements in mind. Open vStorage is the layer between the hypervisor and Object Store and turns the Object Store into a high performance, distributed, VM-centric storage platform.

You can download the full white paper here.

Open vStorage 1.0.3 Beta

We couldn’t wait until the end of April before pushing out our next release. That is why the Open vStorage Team decided to release today (April 17 2014) a small version with following set of features:

  • Remove a vPool through the GUI: in version 1.0.2 it was already possible to create a vPool and extend the vPool across Hosts through the GUI. To make the whole vPool management process complete and more user friendly, we have added the option to remove a vPool from a Host. This is of course only possible in case there are no vDisks being served from the vPool to a vMachine on that Host.
  • New version of Arakoon: Arakoon 1.7, the distributed key-value store that guarantees consistency above anything else, integrated in Open vStorage had some major rework done. This release also fixes bugs we encountered.
  • Some small improvements include adding an NTP server by default to the Host and displaying the different used ports on the VSA detail page.

The version also includes bug fixes:

  • Issue with connecting to SAIO via S3.
  • Footer displaying incorrect values.
  • Update sshd configuration to avoid client to send its own specific LOCALE settings.
  • Shutting down first node in 4 node setup caused ovs_consumer_volumerouter to halt on node 2.
  • KVM clones are now no longer automatically started.
  • Open vStorage GUI sometimes didn’t list a second VM.
  • Clone from vTemplate on KVM does not work in the absence of “default” network on Archipel.
  • Restrict vTemplate deletion from the Open vStorage GUI when a child vMachine is present.
  • Add vpool without copying ceph.* files now gives error.

To give this version a try, download the software and install with Quality Level Test.

Why both converged storage and external storage make sense

Today the peace in storageland has been disturbed by a post of Storage Swiss, The Problems With Server-Side Storage, Like VSAN. In this post they highlight the problems with VSAN. At the same time Coho Data, developer of flash-tuned scale-out storage appliances, released a post about why converged storage, such as VSAN, only seems to be usefull in a niche area. You can read the full post here. VMware had to respond these “attacks” and released a post countering the problems raised by Storage Swiss. As with every black-and-white story both sides have valid points, so we took the liberty to summarize these valid points. On top, we believe both sides can play a role in your storage need and that is why we believe Open vStorage, as only solution in storage land, is the solution by allowing to mix and match converged storage and external storage.

Valid points for the converged stack:

  • VSAN and other converged software stacks typically can run on almost any hardware as they are software based.
  • Pricing for a software based solution to give the same reliability as a hardware based solution can be a fraction of the cost and will become commodity over time. Take as example traditional storage redundancy and fault tolerance which is typically implemented via dual controller hardware systems. Software based techniques provide the same level of reliability at much lower costs and much better scaling.
  • Converged stacks are VM-centric, treating a single volume as a LUN. This allows for flexibility and cost reduction. Test tiers can only have a limited or no protection and important volumes might be replicated multiple times across the storage pool and run with best performance.
  • The network in a converged stack is also less complex as only the hosts need to be connected. With external storage appliances you also need to take the dedicated storage network into account.
  • Easy to setup and manage, even for less experienced IT support. This can well be the case in a branch office.
  • Running storage as close as possible to the compute power makes sense. If not, we all would be using Amazon S3 to power our VM storage needs.

Valid points for the external storage stack:

  • External storage arrays take resource intensive tasks upon themself. It allows for example processor and network resources to manage replication and recovery.
  • External storage arrays allow to be linked to different hypervisors while converged infrastructure solutions are tightly linked to the hypervisor. For example VSAN can only be used with the VMware ESXi hypervisor.
  • The storage network can be completely separated from the computer and outbound traffic.
  • Better usage of flash drives as they are shared between multiple hypervisors and writes don’t have to be mirrored to a second appliance.

The Open vStorage view:
Open vStorage believes there isn’t a ‘One Size Fits All’ solution to storage demands. Today, specific applications and functionalities demand specific storage. That is why Open vStorage is configured to work as a truly open solution and works with any storage backend. Open vStorage provides the flexibility of using an existing distributed file system, external storage appliances and even object stores. This allows for more flexibility but at the same time keeps the ability to protect existing investments.

We are firm believers in the approach taken by VMware vSAN where software running on a cluster of machines provides you a reliable storage pool and where the storage intelligence is moved closer to the compute layer. However, we believe this is too important a piece of the virtualization stack for a proprietary solution that is either hypervisor specific, hardware specific, management stack specific and storage back-end specific. Our goal behind Open vStorage is not only build something open and non-proprietary but something modular enough which allows developers and hardware manufacturers to innovate on top of Open vStorage. We believe with contributors from the open source community Open vStorage can become a far superior compared to proprietary offerings whether these be hardware or software based.

To conclude Open vStorage:

  • Leverages server based flash and storage back-ends to create shared virtual machine storage
  • Redundant, shared storage without the costs and scaling problems of dual controller systems
  • Works with any storage back-end, including filesystems, NAS, S3 compatible object storage, …
  • Works with KVM and VMware
  • Delivered under the Apache License, Version 2.0

VM-Centric Storage

We’ve designed Open vStorage to be VM-centric and a lot of the features we create over the next few months will leverage the VM-centric architecture. The idea of VM-centric storage has been implemented by new age storage companies such as Tintri, VMware (with vSAN) and us when designing our products.This approach is a huge departure from the “traditional” LUN based approach taken by SAN vendors. This brings the question – “What exactly do we mean by VM-centric architecture?”.

Managing VMs and not storage
The key feature with VM-centric storage is that IT administrators manage virtual machines and not storage. All storage actions are taken on a per virtual machine basis rather than having to understand LUNs, RAID groups, storage interfaces, etc. With Open vStorage any IT administrator having knowledge on managing virtual machines should be able to manage it rather than having to get into the complexities of storage and storage networking specifics. A simple example could be cloning. With Open vStorage an administrator can use a VM template to create clone(s) while a traditional storage administrator would need to clone a LUN and then export a particular VM to achieve the same result.

Storage Policies by Virtual Machines
VM-centric design allows for storage policies to be defined on a per virtual machine basis. A few examples of this is as follows:

  1. Replication from one storage cluster to another, can be done on a per virtual machine basis. Moreover, different replication schedules can be used for different virtual machines – as an administrator you may choose an important VM to be replicated every 30 minutes while a not so important VM to be replicated once a day. Moreover, different virtual machines could be replicated to different clusters.
  2. Data retention policies can be defined on a per virtual machine basis. Open vStorage takes snapshots of virtual machines every hour and soon we will be adding the capability whereby an administrator can set the retention policies for these snapshots differently for different virtual machines. For example for one virtual machine an administrator may set retention for 7 days while for another virtual machine it could be 3 years.
  3. With Open vStorage one can use multiple back-ends such as a NAS, file system, object store. We call these back-end storage systems as vPools. With Open vStorage one can have multiple such vPools per host. The administrator can then select vPool on which the data for a particular virtual machine is stored on again on a per virtual machine basis. Hence, an example would be a VM that needs fast I/O could use a vPool that provides fast throughput while a VM requiring not so fast I/O could be stored on a vPool that has a slower throughput.
    We feel that VM-centric storage is here to stay, and that over a period of time the idea of requiring separate storage management processes, software and people would become obsolete. Our engineers are not stopping at the above mentioned VM-centric features, and our key R&D staff are working on capabilities such as encryption (on a per VM basis), dynamic QoS (tuned per VM) and many more exciting innovations.

Upcoming release (End April)

The Open vStorage tries to release a new version once month. Since version 1.0.2 was released end of March, the next (big) release release can be expected by the end of April. This release, Open vStorage 1.1 is planned to have following content:

  • HA Support: in version 1.0.2 you need to set a host offline to be able to steal volumes away from the VSA on that host. In the next release we would allow to steal volumes automatically when a host is down. This means that Virtual Machines which are migrated by the hypervisor to another node will automatically be stolen by the new hosts. Keep in mind that in version 1.0.2, live migration is f.e. already supported. As long as both hosts are online, volumes can move withouth downtime between VSAs.
  • Logging: in version 1.0.2 the logging was very minimal. In the 1.1 version we will make sure all components log into a central location for the whole cluster. This should make troubleshooting much more user-friendly.

That is the plan for the next big release. In case you want to see things added to the roapmap, let us know!

Open vStorage 1.0.2 Beta

KVM
Open vStorage supports KVM as Hypervisor.

Swift Support
Open vStorage supports Swift (SAIO – Swift All In One, Swiftstack, …) as Storage Backend.

Configuration of a vPool through the GUI
You can now add a new vPool through the GUI to a VSA. In case the vPool is a not a local filesystem, you can also stretch it across multiple VSAs.

Protection against moving a vDisk to a VSA where it can’t be served
Each VSA can only serve only a limited amount of vDisks. In case a vDisk is moved between VSAs, f.e. due to a vMotion, the process will first check if there are enough resources available to serve the vDisk. In case there are not enough free resources on the destination VSA, the handover will be aborted.

Software for the hybrid cloud future

cloudfutureThe demand for data storage shows no sign of slowing down. According to IDC, in 2012, external disk storage capacity rose 27 percent over the previous year to reach over 20 exabytes worth a staggering $24 billion in annual spending. Media and entertainment as well as web based services are consuming vast quantities of data storage and the arrival of big data analytics is accelerating the growth further. But quantity of storage is simply not enough, especially as organisations increasingly need to share data or make it available for inspection by third parties such as regulators or partners.

Organisations with the largest data repositories are looking at spiralling costs. The requirements to have data available secure and in a form that can undergo analysis through technologies such as Hadoop are becoming daunting.

The largest cloud providers such as Amazon, Google and Microsoft are using their economies of scale to offer ‘pay as you go’ type business models. But even their relatively low per GB costs quickly add up as data volumes continue to grow without expiry dates. It is also clear that certain use cases do not work with basic cloud services.

For example delivering VDI directly from Amazon S3 is feasible but probably not economically viable. While storing patient data in a public cloud would cause all kinds of compliance concerns for healthcare providers. Even traditional relational database driven applications are better suited to big iron on premise than clouds. In fact, the examples where moving data into the cloud offers fewer benefits than on premise are numerous.

Instead, the hybrid model is starting to become the preferred route for organisations that need some data to be close and performing and other sets to reside in cheaper and slower clouds. The building of these hybrid clouds is an early example of the power of software defined storage.

Let’s provide a use case that many will recognise. A large enterprise has a production environment with several critical applications including a transactional database, an ERP system and several web based applications. The enterprise could be within financial services, life sciences, M&E – it doesn’t really matter which industry as the most critical element is the petabyte of data spread between real-time and archive stores that the business needs to run its day-to-day operations and longer term commitments.

Like many large enterprises, it maintains its own data centre and several regional ICT hubs as well as a disaster recovery data centre. This disaster recovery position is kept perfectly synced with the main centre with the ability to failover rapidly to ensure that the business can continue even with a physical site issue such as fire, flood or similar disruption.

Sound familiar – it should – as the setup is used by enterprises across the globe. Enterprises spend trillions of dollars on redundant IT infrastructure which is often never used but still critical to guard against failure. In an ideal world, organisations could instead replicate data into a cloud and spin up a replacement environment only when needed. A complete production environment that would pop up only on demand based on an accurate snapshot of the current data position of the live environment.

Software Defined Storage and Open vStorage in particular has the capability to make this hybrid DR use case finally viable. As production environments move from physical to entirely virtualised workloads, SDS technologies that use time based storage allow for zero copy snapshot, cloning, and thin provisioning that are ideal for spinning up VMs from master templates in a fast and very efficient way. Better still, testing an entire disaster recovery position in a cloud would be relatively straight forward and repeatable.

But let’s not get ahead of ourselves. The technology is here and small scale examples for these hybrid clouds used for disaster recovery are starting to appear. However, the larger enterprises have entrenched positions and huge investments in the status quo so have less desire to move. However, the up and coming enterprises that have already embraced cloud and virtualisation will be able to benefit further from software defined storage in ways that will reduce their costs giving them a competitive advantage that rivals will be forced to counter. DR is just one use case where software defined will shake up the industry and from small scale examples that we are working on with our partners, the theories will be tested and the cost savings proven in the real world.

In our view, the future is bright, and the cloud future is certainly hybrid!

Standing at the software defined crossroads…

Cross roads horizonIt seems that the term “software defined…” is rapidly becoming the hottest topic across the IT landscape. Across networking, storage, security and compute – the idea of using commodity hardware mixed with open standards to break the monopoly of hardware suppliers will radically change the face of information technology over the next decade. Software defined fundamentally changes the economics of computing and is potentially both a blessing and a curse to the vested interests within the IT industry.

If we start with three clear evolutional trends that have progressed for the last three decades. The performance of Intel based x86 architectures have grown almost exponentially every 18 months. Alongside compute; storage capacity over density has also grown at another almost exponential rate based on a 24 month cycle. Both these critical IT elements have also become more energy efficient and physically smaller. In the background, the performance of internal data buses and external network connections has improved with speeds heading towards 100GBp/s over simple Ethernet.

In previous times, to gain a competive edge, vendors would invest in custom ASICs to make up for deficiencies in performance and features offered by Intel’s modest CPUs or limitation in storage performance – in the days before low cost SSD and flash. The rise of the custom ASIC provided the leaders across areas such as storage and networking with a clear and identifiable benefit over rivals but at the cost of expensive R&D and manufacturing process. For this edge, customers paid a price premium and were often locked into a technology path dependent on the whims of the vendor.

The seeds of change were sown with the arrival of the hypervisor. The proof point that a software layer could radically improve the utilisation and effectiveness of the humble server prompted a change of mind-set. Why not the storage layer or the network? Why are we forced to pay a premium for proprietary technologies and custom hardware? Why are we locked into these closed stacks even as Intel, AMD, ARM, Western Digital, Matsushita and others spending tens of billions to develop faster “standards based” CPU, larger hard-disks and connectivity? Why indeed!

In a hardware centric world, brand values like the old “never got fired for buying IBM” adage could make sense but well written software that can undergo rigorous testing breaks that dependency. It is still fair to say that the software defined revolution is at an early stage but if you look at the acquisition frenzy of start-ups acquired by heavyweights across networking and storage, it’s clear that the firms with the biggest vested interests are acutely aware that they need to either get on board or get out of the way.

There is one significant danger. Our software defined future needs adherence to open standards or at least an emergent standard that becomes the defacto baseline for interoperability. We are at the crossroads of a major shift and without global support for initiatives such as OpenStack there is a danger that software defined may fragment back into the vendor dominated silos of the legacy IT era. Ourselves and other pioneers in the software defined movement need to resist the temptation to close gates to open API’s and backward compatibility – we must treat the rise of software defined as a real chance to change the status quo for the benefit of both customers and an industry that thrives on true innovation.