Seattle, WA
December 10–13, 2018
Click Here for More Information & Registration
View Venue Map
Back To Schedule
Thursday, December 13 • 11:40am - 12:15pm
Running Serverless HPC Workloads on Top of Kubernetes and Jupyter Notebooks - Christopher Woods, University of Bristol

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Jupyter notebooks, deployed on top of k8s, are finding rapid adoption in universities and industry. While k8s can spawn new pods for each notebook session, launching high performance computing (HPC) jobs during dynamic workflows is complicated. Fn is a serverless solution that sits on top of k8s. Individual simulations are deployed as Fn functions that are called dynamically from workflows run within Jupyter notebooks. A notebook running on a lightweight k8s cluster can burst HPC workloads via Fn serverless calls to another cluster running on a bare metal HPC/GPU cloud. Using Fn, we have constructed a distributed access, authentication and accounting layer around a shared object store. This adds security and accountability, thereby making it easy for end users to manage complex multi-cloud workflows involving lots of different types of hardware from within a easy-to-use Jupyter notebook.

avatar for Christopher Woods

Christopher Woods

Research Software Engineer, University of Bristol
I'm the Head of the Research Software Engineering Group (HoRSE) at the University of Bristol and am one of the first cohort of EPSRC RSE Fellows. I've been involved with the RSE community for many years now, and am Talks Chair for RSE19. Obviously I'm biased, but I think this will... Read More →

Thursday December 13, 2018 11:40am - 12:15pm PST
Tahoma 1/2 @ TCC The Conference Center (TCC) - Separate from WSCC building