Theme: Machine Learning Engineering + Halloween Party

Our presentation demonstrates efficient ML workload orchestration using Cloud Composer (managed Airflow) and GKE Kueue. After brief introductions to GKE Kueue and Cloud Composer, it dives into a practical example: LoRA FineTuning of Gemma models. This ML workflow, implemented as an Airflow DAG, includes benchmarking and uploading results to VertexAI. A step-by-step guide and recorded demo complete the presentation, offering a comprehensive understanding of the process.

Retrieval-Augmented Generation (RAG) has emerged as a transformative application for large language models, enhancing their ability to generate accurate, context-aware responses by integrating external knowledge.

RAG is particularly effective in incorporating information that is either missing from the model’s training data or consists of private, proprietary company data, making it invaluable for custom applications. Additionally, RAG helps reduce the risk of hallucinations by grounding responses in specific, retrieved documents.

At the core of RAG is the process of generating document embeddings—dense numerical representations that enable efficient retrieval and comparison across vast datasets. While many introductory resources touch on basic embedding techniques (e.g., model.encode(text)), they often overlook the significant challenges that arise when scaling this process to handle millions of documents.

Embedding large corpora demands careful consideration of computational efficiency, memory optimisation, data quality and chunking strategies, model selection and fine-tuning, and index design choices to ensure robust and efficient text encoding, reliable vector upserting and fast retrieval.

In this talk, I will explore the complexities of embedding text at scale, highlighting the practical challenges and advanced solutions that enable efficient and effective large-scale implementations.

This presentation delves into the world of Large Language Models (LLMs), focusing on the  efficiency of LLM inference. We will discuss the tradeoff of latency and bandwidth, followed by a deep dive into techniques for accelerating inference, such as KV caching, quantization, speculative decoding, and various forms of parallelism. We will compare popular inference frameworks and address the challenge of navigating the multitude of design choices. Finally, we’ll introduce Nvidia Inference Microservices as a convenient one-stop solution for achieving efficient inference on many of the popular models.

This tech-talk explores the intersection of Generative AI (LLMs) and hybrid cloud architectures to ensure safe, secure, and compliant development and deployment. We’ll explore LLM selection and configuration strategies, design scalable and secure architectures for generative AI applications, and discuss tactics to prevent bias, manipulation, or other security threats.

Get ready for networking with fellow community members in a relaxed atmosphere with snacks and drinks. It’s the perfect way to unwind and connect after a day of innovation and inspiring talks. 🥂