Loading…
Attending this event?
Seattle, WA
December 10–13, 2018
Click Here for More Information & Registration
View Venue Map
View analytic
Wednesday, December 12 • 11:40am - 12:15pm
Nezha: A Kubernetes Native Big Data Accelerator For Machine Learning - Huamin Chen, Red Hat & Yuan Zhou, Intel

Sign up or log in to save this to your schedule and see who's attending!

Feedback form is now closed.
Large training datasets used by machine learning frameworks, such as Kubeflow, are usually stored in low cost and high capacity S3 or Google Cloud Storage. However, S3’s rating limiting and slow data downloading significantly challenges training performance and limits compute scalability.

We introduce NeZha and explain how it improves Kubeflow’s training. Nezha is an open source, community driven, and highly collaborative project, contributed by storage and big data engineers. Nezha is based on Kubernetes Initializer: it rewrites Pod spec, adds a sidecar S3 cache, and redirects Pod to use local cache to accelerate.

Nezha is self contained and easy to use. It does not require modification to existing applications or user visible Pod changes.

Nezha improves big data application performance. Our initial Kubeflow benchmark results using MNIST dataset shows NeZha achieves ~2x speedup.

Speakers
avatar for Huamin Chen

Huamin Chen

Principal Software Engineer, Red Hat
Dr. Huamin Chen works at Red Hat's CTO office and is passionate inventor and developer about storage and cloud technologies. He is one of the founding members of Kubernetes SIG-Storage, and created the storage volume plugins for Kubernetes and OpenShift. He is also a member of Rook... Read More →
avatar for Yuan Zhou

Yuan Zhou

Senior Software Development Engineer, Intel
Yuan Zhou is a Senior Software Development Engineer in the Software and Service Group for Intel Corporation, working in the OpenSource Technology Center primarily focused on BigData Analytics Storage. He has been working on Databases, Virtualization and Cloud computing for most of... Read More →



Wednesday December 12, 2018 11:40am - 12:15pm
Ballroom 6A