Loading…
December 12-13, 2023
San Jose, CA
View More Details & Registration

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for AI.dev & Cassandra Summit to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in Pacific Standard Time (UTC -8). To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date."
Wednesday, December 13 • 11:10am - 11:40am
Democratization of ML Pipelines and Bringing ML Workflows to Heterogeneous Cloud Native ML Platforms - Yihong Wang & Tommy Li, IBM

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.


In Kubeflow Pipelines(KFP) v1, the pipeline spec describing ML flow is platform-dependent, which makes it impossible to bring your ML flows to other pipeline frameworks. To lower the barrier, Intermediate Representation(IR) is available in KFP v2. It's a generic component/step-oriented specification that fits container orchestration frameworks. More importantly, a new pipeline orchestration engine is created on the backend to support automatic lineage tracking and components that consume or produce metadata. In light of these new features, it gets a foot into ML pipeline portability. Porting fundamental components allows you to run your pipelines on the desired platforms. Besides, IR is used in the Kubeflow pipeline registry protocol as the format of pipeline templates. Currently, IR specification is in a stable version and used by Kubeflow Pipelines, Google Vertex AI, and Kubeflow Pipelines with Tekton. In this talk, we walk through the new IR spec, a list of components that constitute the new pipeline orchestration engine, how we adapt the new features in another pipeline framework, and use the new registry protocol to democratize the ML pipeline development.

Speakers
avatar for Tommy Li

Tommy Li

Senior Software Developer, IBM
Tommy Li is a senior software developer in IBM focusing on Cloud, Kubernetes, and Machine Learning. He is one of the Kubeflow committers and worked on various open-source projects related to Kubernetes, Microservice, and deep learning applications to provide advanced use cases on... Read More →
avatar for Yihong Wang

Yihong Wang

Sr. Software Developer, IBM
Yihong Wang is a software developer with the Watson AI and Data Open Technologies group at the IBM Silicon Valley Lab, where he actively works on the Kubeflow, Presto, and Node.js communities. He has been successfully delivering several Kubeflow releases to IBM Kubernetes Service... Read More →



Wednesday December 13, 2023 11:10am - 11:40am PST
LL20D (Lower Lobby Level)