Provides 30 percent faster throughput performance improvement for ML training workflows
April 16, 2019
Santa Clara, CA · April 16, 2019 — Quobyte® Inc., a leading developer of modern storage system software, today announced that the Quobyte Data Center File System is the first distributed file system to offer a TensorFlow plug-in, providing increased throughput performance and linear scalability for ML-powered applications to enable faster training across larger data sets while achieving higher-accuracy results.
TensorFlow is an open source library for numerical computation and large-scale machine learning used across industries such as autonomous vehicles, robotics, financial services, healthcare, government, aerospace, defense, and many others. Using Quobyte storage with TensorFlow helps to simplify and streamline the operation of machine learning.
Quobyte’s TensorFlow File System Plug-in allows TensorFlow applications to talk directly to Quobyte, bypassing the operating system kernel to significantly reduce kernel mode context switches and lower CPU usage. While Quobyte storage can be used with all stages of ML, the resulting increased GPU utilization from the TensorFlow plug-in speeds up model training of ML workflows.
Quobyte provides users the flexibility to train anywhere and seamlessly move models into production to better support ML workloads from the data center to the cloud to the edge. The TensorFlow plug-in can be used to train models locally on sample data sets and use the Google Cloud Platform for training at scale because Quobyte runs on-prem and in the cloud. Additionally, because it bypasses the kernel entirely, Quobyte’s TensorFlow plug-in works with both current and older versions of Linux, providing a full range of flexible deployment options for use in ML. Using the Quobyte TensorFlow-Plugin is seamless since there are no application modifications required.
“As more and more businesses look to leverage ML to increase innovation, achieve a faster time to market and provide a more positive customer experience, there is an increasing need for storage infrastructures that offer higher performance and increased flexibility that these workloads need,” said Frederic Van Haren, Lead Analyst HPC and AI Systems of analyst firm Evaluator Group. “Vendors, like Quobyte, that offer high performance, broad platform support and flexibility of deployment options are well positioned to help companies handle bigger data sets, achieve more accurate results and run ML workloads in any environment.”
With Quobyte, there is no need for specialized storage systems to get the most out of ML. Quobyte is a single storage system that addresses many different performance profiles, including the high-throughput, low-latency requirement of ML’s model training stage, as well as large block sequential, small block random or mixed general workloads. Quobyte supports the broadest set of access protocols and clients, such as S3, Linux, Hadoop, Windows and NFS for greater platform flexibility and more complete data ingest and preparation. Data is readily available at any stage all within a single global namespace and all managed through Quobyte’s intuitive management console.
Additional benefits of Quobyte’s TensorFlow File System Plug-in include:
“By providing the first distributed file system with a TensorFlow plug-in, we are ensuring as much as a 30 percent faster throughput performance improvement for ML training workflows, helping companies better meet their business objectives through improved operational efficiency,” said Bjorn Kolbeck, Quobyte CEO. “With the higher accuracy of results, scalability to handle bigger data sets and flexibility to run on-prem to the cloud, and edge, we believe we are providing an optimal experience that allows customers to fully leverage the value of their Machine Learning infrastructure investments.”
Building on a decade of research and experience with the open-source distributed file system XtreemFS and from working on Google’s infrastructure, Quobyte delivers on the promise of software defined storage for the world’s most demanding application environments including High Performance Computing (HPC), Media & Entertainment (M&E), Life Sciences, Financial Services, and Electronic Design Automation (EDA). Quobyte uniquely leverages hyperscaler parallel distributed file system technology to unify file, block, and object storage. This allows customers to easily replace storage silos with a single, scalable storage system — significantly saving manpower, money, and time spent on storage management. Quobyte allows companies to scale storage capacity and performance linearly on commodity hardware while eliminating the need to expand administrative staff through the software’s ability to self-monitor, self-maintain, and self-heal.
Please visit www.quobyte.com for more information.