Menu

iQIYI & Open vStorage

Summary

  • With Open vStorage’s solution, iQIYI has made its virtual machines (VMs) management system much more flexible and achieve better IO performance.

  • The Cluster (more than ~50 nodes) run on SATA SSDs as Read and Write cache, and SWIFT object storage as its backend.

  • Hundreds of optimized VMs of iQIYI provide scalable computing capability, such as NGINX webservers, Docker Applications and video transcoding, etc.
    • logo-iqiyi

    Open vStorage has allowed iQIYI to turn our OpenStack Swift object store into a scalable, high-performance SAN for production Virtual Machines. ~ Ke Yu, Technical Director, iQIYI Cloud Infrastructure Team

    In detail

    The way how and when people watch video has changed drastically around the world in recent years. More and more people are moving away from the traditional linear TV-model to online video streaming platforms. This is also a trend happening in China. As the largest online video platform in China, iQIYI has more than 600 million viewers and it broadcast a total of 39.2 billion hours of TV shows and movies in 2015.It should not come as a surprise that the IT platform that stores, transforms and serves these videos is massive. To power their cloud, iQIYI has hundreds of nodes running on OpenStack. When storing and serving large video files, the underlying storage technology becomes an important component to ensure availability and boost customer satisfaction. This leads to a challenge for storage capacity: the storage should run on commercial X86 servers and support OpenStack / KVM. Migration between physical machines should be possible and performance is a must. Additional requirements are unlimited snapshots, fast cloning, seamless storage expansion and simple operation and maintenance.
    While VMs in iQIYI run on top of local disks for high-performance workloads and GlusterFS for workloads with large storage requirements, operation efforts are becoming increasingly important due to a growing cluster. In order to optimize operation costs and offer customers with consistent user experience, a new solution is required.
    In their search for the best storage technology available, the iQIYI team came across Open vStorage, a scale-out, reliable, high-performance and software-based storage platform. As iQIYI is already managing a multi-petabyte Swift cluster, Open vStorage seized their attention as it allows to turn Swift, the default Object Storage Solution in OpenStack, into fast block storage for Virtual Machines. For iQIYI, the benefits of using Open vStorage are clear:

  • Storage scales seamlessly.
  • Only a single distributed storage backend (Swift) needs to be maintained for both Virtual Machines and Objects.
  • Stunning performance as SSDs are used to address the needs of IO intensive applications.
  • A single unified namespace which allows to move VMs between hosts.
  • VM-centric storage functionality such as rapid cloning and unlimited snapshots.


    • Based on its strong capability of software development and customization, iQIYI has set up a large cluster of compute servers with multiple SSDs as cache disks and one petabytes Swift storage as backend. VMs on cluster are used to run Web server, docker applications. Besides running offline workload, VMs are also providing scalable online service, such as video transcoding, web service and massive data processing. In summary, the Open vStorage solution has helped iQiyi achieve its goals for high-quality user experience and high operation efficiency.

    Collaboration

    Having worked closely with Open vStorage, iQiyi is a top 5 contributor for this project, with four of its engineers getting involved. iQIYI played a leading role in porting Open vStorage from Debian/Ubuntu to CentOS7. It also contributed to the non-deduplication feature, which gains more than 80% increase in write IO performance.

    About iQIYI

    iQIYI, Founded in April 2010, is one of the largest Internet and mobile video service providers in China. An independently operated subsidiary of Baidu, the world’s largest Chinese search engine, iQIYI is focused on providing Chinese users with the best possible online video experience.

    With over 600 million users and 10 million paid subscribers, iQIYI has built up one of the largest movie libraries in China’s online video industry, which features over 6,000 titles.