Seattle, WA
December 10–13, 2018
Click Here for More Information & Registration
View Venue Map
Wednesday, December 12 • 4:30pm - 5:05pm
Towards Trading on Kubernetes: Operating Multi-Tenant and Secure Clusters - Andrew Kochut & Javier Diaz-Montes, Two Sigma

Sign up or log in to save this to your schedule and see who's attending!

Feedback form is now closed.
Two Sigma, a financial company, performs large-scale data processing for modeling and trading while facing risks, such as data exfiltration. We present how we addressed this by building multi-tenant Kubernetes clusters to run over 500 services on 30K cores and 200TB of RAM. These include parts of our trading system and our document translation system, build and test farms, and artifact caches.

Kubernetes doesn't provide full tenant isolation so users often create per-tenant clusters. Two Sigma has many teams with unique data and service access needs, so such a model would have large overheads. We built multi-tenant clusters by coupling namespace, RBAC and PSPs with Two Sigma’s entitlement system. We also integrated Kerberos via annotations to inject tickets, keytabs, and SSL certs into Pods. We discuss lessons operating this both on-prem and public cloud, including pros and cons of GKE.

avatar for Javier Diaz-Montes

Javier Diaz-Montes

Software Engineer, Two Sigma
Javier Diaz-Montes is currently Software Engineer at Two Sigma, where is a member of the Platform Orchestration team working on enabling Kubernetes firm-wide. Before joining Two Sigma, he was Assistant Research Professor at Rutgers University (2012 to 2016), where he focused on researching... Read More →

Andrzej (Andrew) Kochut

Vice President, Two Sigma
Andrzej (Andrew) Kochut serves as a Vice President and engineering manager at Two Sigma. His responsibilities include container based compute, software deployments, and load balancing. His team focuses on transforming Two Sigma to embrace cloud-native design paradigm via adoption... Read More →

Wednesday December 12, 2018 4:30pm - 5:05pm