All videos

Orchestration of ML workloads via Cloud Composer & GKE Kueue

October 24, 2024

Our presentation demonstrates efficient ML workload orchestration using Cloud Composer (managed Airflow) and GKE Kueue. After brief introductions to GKE Kueue and Cloud Composer, it dives into a practical example: LoRA FineTuning of Gemma models. This ML workflow, implemented as an Airflow DAG, includes benchmarking and uploading results to VertexAI. A step-by-step guide and recorded demo complete the presentation, offering a comprehensive understanding of the process.

About the speakers

Piotr Leśniak

Piotr Leśniak

Sr Staff Software Engineer at Google
Staff Software Engineer, ex-Akamai, ex-Qualtrics currently working in Google, experienced in distributed and microservice architecture. Focusing on scalability, operational excellence and high quality of delivered…
Read more
Michał Wysokiński

Michał Wysokiński

Software Engineering Manager at Google
Michal Wysokinski is a Software Engineer and Engineering Leader currently leading the development of Kueue at Google, where his mission is to make Kubernetes the…
Read more

Other videos that you might like

fallback image
Scala
Towards cloud-native Scala
Łukasz Biały
fallback image
Frontend/JS
IA behind the scenes – when UX meets Dev to create a reliable Information Architecture
Monika Soja, Tomasz Piechaczek
fallback image
Frontend/JS
The search for App-iness: Building the future with Progressive Web Apps
Jamie Maria Schouren
fallback image
Kotlin ecosystem
Inkuire – Using types as search keys in Kotlin libraries
Andrzej Ratajczak, Kacper Korban
All videos