Product Tech

Is Your Storage System Preventing You From Working with 4K?

By  Björn Kolbeck  on  

    Vote on Hacker News  

Hollywood’s drive towards higher resolution video formats shows no signs of easing. Even while home users barely scratch the surface of 4K viewing, digital content creators already explore 5K and 8K variants, searching for ever more photo-realistic renditions.

Along with these higher resolutions, wider color gamuts and 10 or 12-bit color depths are further driving up file sizes and the commensurate bandwidth necessary to carry all that information. While storage capacity for these behemoth files is readily available, storage throughput that can support streaming these formats without jitter is indeed a challenge.

4K Requires Sustained High Bandwidth

Two reasons why 4K workloads can be very demanding for most underlying storage system:

  1. The application requires a sustained high bandwidth of over 1GB/s!
  2. For file types like DPX and EXR, each frame is a separate file which the application needs to open and read; that’s difficult when done with networked storage. Supporting multiple concurrent streams, 8K, and higher frame rates will make the problem even worse.

Let’s look at two examples. A “normal” 4K stream (4096×2160, 8bit, 60fps, 4:2:0) requires a throughput of 8.91Gbit/s. By contrast, an 8K stream (7680×4320, 8bit, 60fps, 4:4:4) pushes that to 71.28Gbit/s! Two things become immediately apparent: One, if you’re running over Ethernet you’ll need at least a 10GigE network for 4K and 100GigE to handle 8K streams. And two, if you’re running over Fibre Channel or Infiniband, the hardware requirements can get expensive! (Quobyte uses Ethernet and simply requires inexpensive standard hardware from the servers to the switches.)

The ever-growing demand for throughput, low latency, concurrency and the necessity to store 10s of petabytes of raw footage, intermediate versions, and final material creates post-production video workflows that most studios and production houses are ill-equipped to handle.

The aggravating effects of outdated workflows and legacy storage on 4K workflows

For the most part, visual effects and rendering applications were often written with the assumption of local storage in mind. That meant they had high bandwidth and low latency available. But the video content data needs to be shared between workstations as part of a normal workflow. This results in a few problems:

  1. Several copies of virtually the same file exist, unnecessarily consuming storage capacity.
  2. Copies are transferred back and forth over the network, eating into available bandwidth.
  3. Equipping each workstation with sufficient local storage is expensive and wastes resources.

One Storage System for Your Entire Post-Production Workflow

Quobyte storage software provides features specifically designed to address the needs of this market segment. They include:

  1. User configurable file prefetching to minimize latencies – which enables jitter-free playback with apps written for local storage.
  2. Writeback caching for the fast saving of files on the network storage.
  3. High-performance erasure coding that enables 4K streaming without expensive SSDs.
  4. No more “cold tier”, large-scale storage is possible with all data being immediately available for streaming in raw 4K.

See Us at NAB

The 2018 NAB Show will be a great time to come by our booth and find out more about how Quobyte can improve operational efficiency in your digital production workflow. Benefit from increased performance, bandwidth, and scalability. Any questions? Get in touch and schedule a meeting.

Photo of Björn Kolbeck

Written by

Björn is Quobyte’s co-founder and CEO.