Wim Provoost, Product Manager of Open vStorage, gave a vBrownBag TechTalk at the OpenStack Summit in Austin about OpenStack storage across datacenters. In this talk he discusses the problems encountered by the Open vStorage team and solutions they applied to build a scalable, block storage cluster across 5 datacenters in the US.
The vibe at the Open vStorage office is these days best explained by a song of the early nineties:
I like to move it, move it ~ Reel 2 Reel
While the summer time is in most companies a more quiet time, the Open vStorage office is buzzing like a beehive. Allow me to give you a short overview of what is happening:
- We are moving into our new, larger and stylish offices. The address remains the same but we are moving into a completely remodeled floor of the Idola business center.
- Next to physically moving desks at the Open vStorage HQ, we are also moving our code from BitBucket to GitHub. We have centralized all our code under https://github.com/openvstorage. To list a few of the projects: Arakoon (our consistent distributed key-value store), ALBA (the Open vStorage default ALternate BAckend) and of course Open vStorage itself. Go check it out!
- Finishing up our Open vStorage 2.2 GA release.
- Adding support for RedHat and Cent OS by merging in the Cent-OS branch. There is still some work to do around packaging, testing and upgrades so feel free to give a hand. As this was really a community effort, we owe everyone a big thank you.
- Working on some very cool features (RDMA anyone?) but let’s keep those for a separate post.
- Preparation for VMworld (San Francisco) and the OpenStack summit in Tokyo.
As you can see, many things going on at once so prepare for a hot Open vStorage fall!
As last year, the Open vStorage team will attend the OpenStack Summit in Vancouver. Who would want to miss the success stories of OpenStack in the field, the new developments and many interesting sessions for OPS and developers? Not this team! In case you want to meet us, locate booth T21 on the show floor or scan the summit for someone with a blue shirt with the Open vStorage logo on the back. We will be there with a whole bunch of people so whether you want to discuss business, operations, deep technical issues/wishes or just socialize, we have someone in Vancouver that can address your questions and needs.
For us the kilo release is also time for a little party: Open vStorage will be an official, certified Cinder plugin. How cool is that!
During the OpenStack Summit we will present 2 new developments:
- The vRUN appliances: a stackable, hyperconverd private cloud appliance based on OpenStack and Open vStorage. The appliance comes by default with Open vStorage support but if needed we can add OpenStack support and monitoring to the mix.
- The free Open vStorage community edition: the new Open vStorage community edition allows to easily setup a hyperconverged private cloud by using local SATA drives as Tier 2 storage. It allows small users to build and run a complete OpenStack based cloud without any software cost. It also allows users which are looking at Open vStorage as storage layer for a large scale-out cloud to test the hyperconverged functionality without any cost.
After the successful first Open vStorage Roadshow, we decided to do a second US Roadshow. You can meet us during one of the following Meetups in the US:
- Thursday Apr. 16 (Santa Clara, CA): Register here
- Monday Apr. 20 (Menlo Park, CA): Register here
- Wedndesday Apr. 22 (Morrisville, NC): Triangle OpenStack Meetup – Register here
During these meetups we will discuss what Open vStorage exactly does and the latest developments around the project such as how to set up Open vStorage in a HyperConverged fashion on local disks and the new metadata server architecture. We will of course provide pizzas and drinks during these meetups!
Next to these community events, we are also organizing 2 business events in the Bay Area (Santa Clara 04/16, Menlo Park 04/20). During these business events we will discuss how to setup a profitable IAAS (Infrastructure as a Service) or private cloud business and unveil our HyperConverged solution build on top of OpenStack and our own Open vStorage. This solution will include 24/7 support for Open vStorage as well as OpenStack! You can register for one of these free, business events here.
The end of 2014 is near so it is time to look forward and see what 2015 will have to offer. Some people say there’s no point in making predictions and that it’s not worth speculating because nothing is set in stone and things change all the time in storage land. Allow us to prove these people wrong by sharing our 2015 storage predictions*:
Acceleration of (hyper-)converged platforms
Converged platforms are here to stay. Converged solutions are even the fastest growing segment for large storage vendors. But it will be players like Open vStorage who will really break through in 2015. Hyperconverged solutions showed that there is an alternative to expensive SAN or all-flash solutions by adding a software based caching layer to the host. Alas, these overhyped hyperconverged solutions are even more expensive per TB storage than an already expensive all-flash array. In 2015 we will see more solutions which unite the good of the hyperconverged appliance and the converged platform but at a significantly lower cost. Storage solutions that will be extremely hot and prove to be future proof will have to have following characteristics:
- Caching inside the (hypervisor) host: caching on SSD or PCIe flash should be done inside the host and not a couple of hops down the network.
- Scalability: all-flash will continue to be a waste of money due to the huge cost of flash. It is better to go for a Tiered solution: Tier 1 on flash, Tier 2 on scalable, cheap (object) storage. In case the Tier 1 and Tier 2 storage are inside the same appliance (hyperconverged), your scalability and flexibility will be limited. A much better solution is to keep the 2 Tiers completely separate in a different set of appliances but managed by the same software layer.
- Open and programmatically: storage silo’s should in 2015 be something of the past. Integration and openness should be key. Automation will be one of the hottest and most important features of a storage solution.
It should not come as a surprise that Open vStorage checks all of the above requirements.
OpenStack will dominate the cloud in 2015
This is probably the most evident prediction. During the OpenStack conference in Paris it was very clear that OpenStack will dominate the cloud the next few years. In 2014 some new kids showed support for OpenStack such as VMware (they understand that hypervisor software is now a commodity and that the data center control plane has become the high-margin battle arena). With VMware releasing their own OpenStack distribution the OpenStack distribution battlefield will be crowded in 2015. We have RedHat, Ubuntu, Mirantis, HP, VMware and many more so it is safe to say that some consolidation will happen in this area.
A new OpenStack battlefield that will emerge in 2015 will be around OpenStack storage. Currently this area is being dominated by the traditional arrays but as the software-defined storage solutions gain traction, solutions such as Open vStorage will grab a huge market share from these traditional vendors. They can compete with these SANs and all-flash arrays as they offer the same features and have the benefit of a much lower cost and TCO. While they maybe not top the total revenue achieved by the big vendors, they will definitely seize a large share of the OpenStack storage market.
If we add the fact that the Chinese government is promoting OpenStack and open-source in general, you can bet your Christmas bonus on the fact that open-source (OpenStack) storage projects (Swift, Open vStorage, …) will be booming next year. These projects will get a lot of support from Chinese companies both in adoption and development. It will be essential for traditional high-tech companies and countries not to miss the boat as once it has left the harbor it will be very hard to catch up.
New markets for object storage vendors
2015 will be the year where object storage will break out of its niche market of large video or object repositories. This has been said for many years now but 2015 will be THE year as many companies have started to realize which benefits they achieved by implementing their first object storage projects. The next target for these enterprises is to make better use of their current object storage solution. Changing all of their legacy code will not happen in 2015 as this might impact their business. Solutions where they don’t have to change their existing code base and still benefit from the cost saving of object storage will be selected. Open vStorage is one of those solutions but we are pretty sure other solutions like for example storage gateways to object storage will flourish in 2015.
Another reason why object storage vendors will enter new markets is because currently too many players are after the same customer base. This means that if they want to keep growing and provide ROI for the venture capital invested, new markets will definitely need to be addressed. The 15-25 billion dollar SAN market is a logical market to address. But entering this market will not be a bed of roses as object storage vendors have no experience in this highly-competitive market or sometimes not even the right sales competencies and knowledge. They will have to look for partnerships with companies such as CloudFounders who are seasoned in this area.
Seagate Kinetic drives
The Kinetic drives are the most exciting, fundamental change in the storage industry in several years. These drives went GA at the end of 2014 but in 2015 we will gradually see new applications and solutions who make use of this technology. With these IP drives, you will, for the first time, be able to manage storage as a scalable pool of disks. Open vStorage will support the Kinetic drives as Tier 2 backend. This means Virtual Machine will have their hot data inside the host on flash and their cold data on a pool of Kinetic drives.
* We will look back on this post at the end of 2015 to see how good we scored.
OpenStack started to gain momentum in 2014 but will really kick off in 2015. Gartner has put software-defined storage in its top 10 technology trends for 2015. The amounts of data will continue to grow as the Internet of Things is unfolding before our eyes. As Open vStorage is in the middle of OpenStack and storage, 2015 will be the year of Open vStorage. To kick off the year in style we are doing a US Roadshow in cooperation with local OpenStack User Groups. We had to make some heartrending decisions and disappoint some groups as we couldn’t do a Meetup in their community but below are the lucky ones for the first Open vStorage Roadshow:
- Monday Jan. 12 – Toronto: register here
- Wednesday Jan. 14 – Boston: register here
- Thursday Jan. 15 – NYC: register here
- Monday Jan. 19 – San Francisco: register here
- Wednesday Jan. 21 – Washington DC: register here
During this session we will discuss the current status of OpenStack storage projects and give an overview of the new features in Open vStorage 2.0 (to be released in Q1 2015).
Note that registration is required and there is an attendee limit.
PS. In case you organize an OpenStack User Group and would like to host an Open vStorage session, contact us by email at firstname.lastname@example.org.
Open vStorage is hot! You can find the most important video’s, podcasts and articles of the last few weeks here:
- OpenStack Online Meetup: How does OpenStack Swift play with Open vStorage? Podcast with Swift PTL John Dickinson and our own Wim Provoost as they discuss OpenStack storage and how Swift and Open vStorage, 2 open-source projects, work together to shake the OpenStack eco-system.
- The Register: Belgian upstart to ‘bridge gap’ between object, block storage. Or that’s the plan at least . Learn more about the latest developments in the Open vStorage project: support for Seagate Kinetic drives. Open vStorage turns a pools of these disks into a storage solutions which has the features of a high-end SAN (performance, zero-copy snapshots, thin cloning) but is also scale-out and low-cost, like an object storage solution.
- Open vStorage and OpenStack – the Storage Switzerland take. Podcast with George Crump, founder and lead analyst at Storage Switzerland and Wim Provoost, Product Manager of Open vStorage. They discuss the different storage projects in OpenStack and how the Open vStorage Cinder plugin bridges the gap between Swift and Cinder.
- OpenStack® Summit: CloudFounders Open vStorage: CLoudFounders’ CTO Stefaan Aelbrecht explains to HP what Open vStorage is about and where it comes from.
On Tuesday November 25th (09:00 AM PST), you can join the OpenStack Online Meetup community for a conversation between Swift’s Program Technical Lead John Dickinson and Wim Provoost of Open vStorage as they discuss OpenStack storage and how their respective projects work together. They both had presented before and those were some of the most popular meetups. You can see Wim’s episode here. This time they will be talking with both together about how Open vStorage can provide a block interface on top of OpenStack Swift.
You can register here to be part of the session.
Open vStorage is HyperFast storage for OpenStack Virtual Machines. SSDs and PCIe Flash cards inside the hosts are used as Tier 1 storage while OpenStack Swift, Ceph, or any S3 compatible object storage solution is used as Tier 2 storage. George Crump and Charlie Hodges from Storage Switzerland and our own Wim Provoost discussed the OpenStack Open vStorage relation during a 12 minute podcast.
Some of the questions that are raised and answered in this podcast:
- What is Open vStorage?
- Which problems is Open vStorage trying to solve?
- Why does Open vStorage focusses on the OpenStack market?
- Is Open vStorage trying to replace Ceph as OpenStack storage?
Open vStorage is an open-source project which is supported by a spirited community. Without this community Open vStorage would not be the success it is today. To boost that community, CloudFounders (the sponsor behind Open vStorage) will be hosting an Open vStorage booth at following events:
- Powering the Cloud, 28-29 October, Frankfurt, Germany
- OpenStack Paris Summit 2014, 3-5 November, Paris, France
We would like to welcome on our booth contributors, users and supporters. In case you would like to discuss things with the team or have burning questions, we are there for you. So, look for the CloudFounders/Open vStorage booth and stop by to say “HI” or “Bonjour”.