Loading…
Attending this event?
Seattle, WA
December 10–13, 2018
Click Here for More Information & Registration
View Venue Map
View analytic
Thursday, December 13 • 11:40am - 12:15pm
Running Serverless HPC Workloads on Top of Kubernetes and Jupyter Notebooks - Christopher Woods, University of Bristol

Sign up or log in to save this to your schedule and see who's attending!

Log in to leave feedback.
Jupyter notebooks, deployed on top of k8s, are finding rapid adoption in universities and industry. While k8s can spawn new pods for each notebook session, launching high performance computing (HPC) jobs during dynamic workflows is complicated. Fn is a serverless solution that sits on top of k8s. Individual simulations are deployed as Fn functions that are called dynamically from workflows run within Jupyter notebooks. A notebook running on a lightweight k8s cluster can burst HPC workloads via Fn serverless calls to another cluster running on a bare metal HPC/GPU cloud. Using Fn, we have constructed a distributed access, authentication and accounting layer around a shared object store. This adds security and accountability, thereby making it easy for end users to manage complex multi-cloud workflows involving lots of different types of hardware from within a easy-to-use Jupyter notebook.

Speakers
avatar for Christopher Woods

Christopher Woods

Research Software Engineer, University of Bristol
Christopher Woods manages the Research Software Engineering (RSE) Group at the University of Bristol. He is an EPSRC RSE Fellow and Joint-Chair of the UK RSE Association. He obtained his undergraduate and postgraduate degrees in Chemistry from the University of Southampton, working... Read More →



Thursday December 13, 2018 11:40am - 12:15pm
Tahoma 1/2 @ TCC The Conference Center (TCC) - Separate from WSCC building
  • Skill Level Any