Seattle, WA
December 10–13, 2018
Click Here for More Information & Registration
View Venue Map
Back To Schedule
Wednesday, December 12 • 11:40am - 12:15pm
Nezha: A Kubernetes Native Big Data Accelerator For Machine Learning - Huamin Chen, Red Hat & Yuan Zhou, Intel

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Large training datasets used by machine learning frameworks, such as Kubeflow, are usually stored in low cost and high capacity S3 or Google Cloud Storage. However, S3’s rating limiting and slow data downloading significantly challenges training performance and limits compute scalability.

We introduce NeZha and explain how it improves Kubeflow’s training. Nezha is an open source, community driven, and highly collaborative project, contributed by storage and big data engineers. Nezha is based on Kubernetes Initializer: it rewrites Pod spec, adds a sidecar S3 cache, and redirects Pod to use local cache to accelerate.

Nezha is self contained and easy to use. It does not require modification to existing applications or user visible Pod changes.

Nezha improves big data application performance. Our initial Kubeflow benchmark results using MNIST dataset shows NeZha achieves ~2x speedup.

avatar for Huamin Chen

Huamin Chen

Senior Principal Software Engineer, Red Hat
Dr. Huamin Chen is a passionate developer at Red Hat' CTO office. He is one of the founding member of Kubernetes SIG Storage, member of Ceph, Knative, and Rook. He previously spoke at KubeCon, OpenStack Summits, ContainerCon and other technical conferences.
avatar for Yuan Zhou

Yuan Zhou

Senior Software Development Engineer, Intel
Yuan Zhou is a Senior Software Development Engineer in the Software and Service Group for Intel Corporation, working in the OpenSource Technology Center primarily focused on BigData Analytics Storage. He has been working on Databases, Virtualization and Cloud computing for most of... Read More →

Wednesday December 12, 2018 11:40am - 12:15pm PST
Ballroom 6A