The demand for data storage shows no sign of slowing down. According to IDC, in 2012, external disk storage capacity rose 27 percent over the previous year to reach over 20 exabytes worth a staggering $24 billion in annual spending. Media and entertainment as well as web based services are consuming vast quantities of data storage and the arrival of big data analytics is accelerating the growth further. But quantity of storage is simply not enough, especially as organisations increasingly need to share data or make it available for inspection by third parties such as regulators or partners.
Organisations with the largest data repositories are looking at spiralling costs. The requirements to have data available secure and in a form that can undergo analysis through technologies such as Hadoop are becoming daunting.
The largest cloud providers such as Amazon, Google and Microsoft are using their economies of scale to offer ‘pay as you go’ type business models. But even their relatively low per GB costs quickly add up as data volumes continue to grow without expiry dates. It is also clear that certain use cases do not work with basic cloud services.
For example delivering VDI directly from Amazon S3 is feasible but probably not economically viable. While storing patient data in a public cloud would cause all kinds of compliance concerns for healthcare providers. Even traditional relational database driven applications are better suited to big iron on premise than clouds. In fact, the examples where moving data into the cloud offers fewer benefits than on premise are numerous.
Instead, the hybrid model is starting to become the preferred route for organisations that need some data to be close and performing and other sets to reside in cheaper and slower clouds. The building of these hybrid clouds is an early example of the power of software defined storage.
Let’s provide a use case that many will recognise. A large enterprise has a production environment with several critical applications including a transactional database, an ERP system and several web based applications. The enterprise could be within financial services, life sciences, M&E – it doesn’t really matter which industry as the most critical element is the petabyte of data spread between real-time and archive stores that the business needs to run its day-to-day operations and longer term commitments.
Like many large enterprises, it maintains its own data centre and several regional ICT hubs as well as a disaster recovery data centre. This disaster recovery position is kept perfectly synced with the main centre with the ability to failover rapidly to ensure that the business can continue even with a physical site issue such as fire, flood or similar disruption.
Sound familiar – it should – as the setup is used by enterprises across the globe. Enterprises spend trillions of dollars on redundant IT infrastructure which is often never used but still critical to guard against failure. In an ideal world, organisations could instead replicate data into a cloud and spin up a replacement environment only when needed. A complete production environment that would pop up only on demand based on an accurate snapshot of the current data position of the live environment.
Software Defined Storage and Open vStorage in particular has the capability to make this hybrid DR use case finally viable. As production environments move from physical to entirely virtualised workloads, SDS technologies that use time based storage allow for zero copy snapshot, cloning, and thin provisioning that are ideal for spinning up VMs from master templates in a fast and very efficient way. Better still, testing an entire disaster recovery position in a cloud would be relatively straight forward and repeatable.
But let’s not get ahead of ourselves. The technology is here and small scale examples for these hybrid clouds used for disaster recovery are starting to appear. However, the larger enterprises have entrenched positions and huge investments in the status quo so have less desire to move. However, the up and coming enterprises that have already embraced cloud and virtualisation will be able to benefit further from software defined storage in ways that will reduce their costs giving them a competitive advantage that rivals will be forced to counter. DR is just one use case where software defined will shake up the industry and from small scale examples that we are working on with our partners, the theories will be tested and the cost savings proven in the real world.
In our view, the future is bright, and the cloud future is certainly hybrid!