A distribution center for Virtual Machine data

walmart_distribution_centerAt open vStorage we quite often get the question: “Ok, you are a Grid Storage Router but what is that exactly?”

To explain what a Grid Storage Router is and why it is essential in a virtual environment, I’d like to make the analogy with Walmart, the largest retailer in the world. You can compare Open vStorage to the grid of distribution centers of Walmart. Walmart has 42 regional U.S. distribution centers with over 1 million square feet. In total these distribution centers have more than 12 miles of conveyor belts to move 5.5 billion cases of merchandise.

Sam Walton, the founder of Walmart, realized very quickly that in order to sell a lot of goods the company had to develop a multilayered distribution system and identified logistics as its primary expertise. With multiple stores, they could have opted to arrange that goods go directly from manufacturer to the end-shops but the Walmart management quickly realized that having a central hub in between these 2 makes sense. Instead of having their more than 100,000 suppliers dropping of their goods at the door of the 5,000 stores, they ask the suppliers to drop their goods off at one of the regional distribution centers. Having only a limited amount of distribution centers allowed the trucks with supplies to be stocked to the maximum level and hence optimizing round-trips and making the best use of the available truckload capacity.

The distribution centers have grown from being just a temporary storage location, to a fully automated high-tech center where every move is orchestrated and tracked. On top of just temporarily storing goods, the distribution centers also offer additional services such as splitting up large volumes into smaller parcels, repackaging and keep track of the stock.
Typically one distribution center can cater the needs of multiple stores but there is a limit to this capacity. When needed a new distribution center, which typically follows a blueprint, will open to relieve the pressure and to prepare for yet more stores. This allows to scale-out the stores without bringing the chain of supply in danger.

Just as Walmart considers their distribution centers to be as important as their stores, you should attribute Open vStorage with the same importance. Open vStorage is the hub between one or more Storage Backends (Swiftstack, Ceph, …) and your Virtual Machines. Like distribution centers Open vStorage doesn’t only connect suppliers (Storage Backend) and stores (Virtual Machines) with each other in an optimized fashion to build scale-out Virtual Machine storage but it also brings additional value to the table. With its VM-centric architecture it allows efficient unlimited snapshotting, VM replication*, compression and encryption at the VM level*.

Having this grid of Storage Routers, where one can take over from another, it allows to improve the reliability and as all metric of these Storage Routers and the Virtual Machines are tracked, the troubleshooting in case of an issue becomes much easier.

Where distribution centers work with different types of stores (Walmart Supercenters or Walmart Express), Open vStorage is hypervisor agnostic and can handle both VMware ESXi and KVM workloads. For both hypervisors every write gets accelerated on SSD or PCI flash cards and reads are optimized by deduplicating identical content and hence making the best use of the available fast storage.

Just as the distribution centers are key in the success of Walmart, Open vStorage is key in building out scalable, high performance, VM-centric storage.

* Planned for Q3 2014