[Aug] ML Community — Highlights and Achievements
Let’s explore highlights and accomplishments of the vast Google Machine Learning communities over the month. We appreciate all the activities and commitment by the community members. Without further ado, here are the key highlights!
Featured stories
China Gemma Sprint
The China Gemma Sprint led China GDEs to generate various localized, Gemma-related content, such as informative blog posts, practical code samples, illustrative slide decks and engaging videos.
Key projects by AI/ML GDEs
- Gemma NameCraft: Name Idea Generation for Your Baby by AI/ML GDE Wei Zheng (China)
- A tutorial of using Gemma 2: from deploy to fine-tuning by AI/ML GDE Yu Chen (China)
- Multilingual translation by gemma-2–2B by AI/ML GDE Jingtian Peng (China)
Imagen 3
Do It Yourself Imagen 3 — Practical Demo with Vertex AI by AI/ML GDE Sungmin Han (Korea) is a shared Colab Notebook to help you explore the capabilities of Imagen 3. He’s put together a demo focusing on 3 key aspects: 1) high quality image generation, 2) realism and creativity, 3) multilingual model.
Picasso: A Multimodal Chat Demo that integrates Vertex AI, Gemini and Imagen 3 by AI/ML GDE Henry Alonso Ruiz Guzman (US) is a multimodal chat demo that leverages the combined power of Imagen 3 and Gemini to provide an enriched interactive experience.
Imagen 3 on Vertex AI introduction by AI/ML GDE Sam Witteveen (Singapore) on the Google Cloud Tech’s X channel. He briefly shares how to generate images with natural lighting and photorealism within seconds.
Open-source project using Gemma & Colab
Building a User Insights-Gathering Tool for Product Managers from Scratch by AI/ML GDE Hugo Zanini Gomes (Brazil) shares how he built a note-taking and analysis tool to organize a large number of user/customer interviews. The tool helped him, as a technical product manager, fully engage with the interviewees. The whole process starts with breaking down interview recordings or videos into text and organizing them into a document format with tagging, highlighting, and commenting. This article presents the detailed structure of the tool, tutorials, and how he used tools such as Gemma and Colab. This has received 298+ Claps on Medium.
Highlights by products
Gemini
Gemini’s Power for Tourism Apps: A Function Calling Tutorial by AI/ML GDE Nathaly Alarcon Torrico (Bolivia) is a tutorial to create a Tourism web app using Gemini function calling. The app allows users to query the weather for future days in a specific location.
Build AI Agents using Gemini and OpenAGI (slides) by AI/ML GDE Tarun R Jain (India) was a talk demonstrating a code demo on how to build agentic workflow using Gemini 1.5 Flash and OpenAGI.
Boost Development Productivity using Gemini AI-Assisted Coding by AI/ML GDE Lai Fong Leong (Malaysia) shared how to use Gemini Code Assist in your IDE to increase productivity, accuracy, and creativity through code completion, code chat, auto-documentation, and more.
Build agent with Gemini/Gemma and AutoGen by AI/ML GDE Jimmy Liao (Taiwan) was a hands-on workshop covering Gemini Pro, Pro Vision API, and various techniques such as RAG, building agents, and using Autogen.
Building Multimodal Search with Gemini Vision model and RAG by AI/ML GDE Rishiraj Acharya (India) shared how to train multimodal models using contrastive learning, implement any-to-any multimodal search for retrieving relevant context across different data types, and understand how LLMs can be trained to comprehend multimodal data through visual instruction tuning.
Definitive guide: Professional use of the Google Gemini API (Spanish) (Notebook) by AI/ML GDE Carlos Alarcon (Colombia) shows you how to use AI Studio API to interact with Gemini 1.5 Pro. This tutorial guides you through the process of obtaining your API key, configuration of a development environment, and inferences with different models of language. He also explores how to control the tokens used for each operation, function calling, JSON mode, caching and how to optimize your AI projects.
Your Playground for Gemini and Multimodal AI with Gemini are Github repositories by AI/ML GDE Juan Guillermo Gomez Torres (Colombia) for hands-on workshops.
Emotion detection from text, images, audio, Text to Speech and chatbot using Gemini API and Mesop by Usha Rengaraju (India) aims to help autistic individuals identify emotions.
Gemma
PaliGemma on Android using Hugging Face API (repository) by AI/ML GDE Nitin Tiwari (India), Sagar Malhotra, and Savio Rodrigues is an implementation of inferring the PaliGemma on Android using Hugging Face-Gradio Client API for tasks such as zero-shot object detection, image captioning and visual question-answering. Afterwards, he expanded the project to cover tasks like image captioning, visual question-answering, and zero-shot object detection using the Florence-2 vision language model on Android (demo).
Creating an English Tutor with Gemma and Open Source AI Models by AI/ML GDE Carlos Alarcon (Colombia) was a talk about how to create an AI teacher of English that can listen, speak and create a class for teaching the language using Gemma.
COSCUP 2024: RAG and fine tune (FT) Gemma and Advance RAG with PaliGemma, keynote at I/O Extended Tainan by AI/ML GDE Jimmy Liao (Taiwan) introduced Gemma 2, PaliGemma and how to fine-tune the models.
History of Gemma by AI/ML GDE Jerry Wu (Taiwan) at Google for Taiwan shared how to use Gemma and Gemini to reinvent different industries.
Teach you AI new tricks: Using agents to build new concepts in Gemma by AI/ML GDE Rabimba Karanjai (US) was to share how to train Gemma without fine-tuning. His another talk, Rapid Prototyping using Gemma and Production deploy in Vertex AI guided on from finding a model project & pulling up code snippets to creating a simple prompt in just a few clicks. He also showed how to run Gemma locally on your laptop using quantized models.
dstack: Your LLM Launchpad — From Fine-Tuning to Serving, Simplified by AI/ML GDE Chansung Park (Korea) shows how to fine-tune Gemma models on GCP with Hugging Face open-source ecosystem by leveraging the dstack. dstack is an open-source framework, in which Chansung participated, that allows developers to manage virtual machine instances, making multi-node fine-tuning of LLM more accessible.
AI/ML GDE Konstantinos Kechagias (Greece) and UniWattAI have been utilizing Gemma 2 to tackle energy efficiency issues. The team trained Gemma 2 and enhanced it with several techniques such as LoRA and RAG to make it communicate in natural language about the energy management system. By leveraging Gemma 2, they are trying to reduce energy waste and solve some social challenges related to energy innovation in Europe.
Colab
Building Efficient RAG pipeline using Open Source LLMs (Colab Notebook) by AI/ML GDE Tarun R Jain (India) at PyCON Malaysia was a talk on how to choose the parameters for open source LLMs, how to use prompt templates, how to evaluate the pipeline, and challenges mitigation.
Two Colab Notebooks for hands-on workshops, Fine tune Gemma2–2B with LoRA and Fine tune PaliGemma-3B were shared by AI/ML GDE Jimmy Liao (Taiwan).
AI Community Day by Machine Learning Uyo covered key topics such as Gen AI, the latest developments in deep learning frameworks like Keras, and emerging AI tools. And they conducted a workshop running Gemma 2 with Colab Notebook. Participants gained hands-on experience with AI concepts and tools participating in exercises ranging from building simple ML models to deploying more complex AI-n applications.
Keras
AI/ML GDE Anshuman Mishra (India) and Abheesht Sharma (India) have contributed to Keras by co-authoring the paper, KerasCV and KerasNLP: Vision and Language Power-Ups with the Keras team. This paper was recently accepted into JMLR, a prestigious journal.
Usha Rengaraju (India) has contributed to Keras by adding Mobilenet to Keras Hub (PR on Github).
Explore LLMs with Keras by AI/ML GDE Marvin Naftali Ngesa (Kenya) was a hands-on workshop on basic and advanced LLM workflows, including chat generation, LoRA fine-tuning, and model parallelism to train on large-scale infrastructure. He also introduced Gemma and other open source models.
Getting Started Gemma using KerasNLP by AI/ML GDE Lai Fong Leong (Malaysia) was a talk on getting started with Gemma using KerasNLP. Participants learned how to generate text responses to various prompts using Gemma.
JAX
Training LLMs at Scale by AI/ML GDE Phillip Lippe (Netherlands) gave an introduction to training LLMs at scale. He focused on practical and technical aspects, such as memory and compute management, compilation, and parallelization strategies. He also discussed various distributed training strategies like fully-sharded data parallelism, pipeline parallelism, and tensor parallelism, alongside single-GPU optimizations including mixed precision training and gradient checkpointing.
JAX and NNX training on TPUs by AI/ML GDE David Cardozo (Canada) was an in-depth exploration of Vertex AI Pipelines and to discover how to harness the power of TPUs to train SOTA image classification models. Especially he walked through practical examples of building TPU training pipelines using Kubeflow and Vertex AI.
ML Ascent 7: Building your first Multimodal Deep Learning model with Flax/JAX/TensorFlow Part 1 (slides) by AI/ML GDE Taha Bouhsine (US) broke down fundamentals of multimodal deep learning for beginners. He mainly covered basics of multimodality, understanding different modalities and their preprocessing needs, strategies for fusing multiple modalities, when and how to choose the right fusion approach, and etc.
ODML
Unleash the Power of Gemini Models on Android: Secure Integration with Vertex AI and Firebase by AI/ML GDE George Soloupis (Greece) guides a streamlined and secure approach to integrating Gemini models with Android apps using Vertex AI and the Firebase SDK.
Unlocking Generative AI on Android: Creating Interactive Experiences with Vertex AI for Firebase by AI/ML GDE Juan Guillermo Gomez Torres (Mexico) explored Gen AI for Android app development. He discussed the power of Vertex AI for Firebase to leverage Gen AI models and how to use them to create innovative app engaging user experiences.
BigQuery
Accelerate Generative AI Journey in BigQuery with Gemini by AI/ML GDE Lai Fong Leong (Malaysia) explained how Gemini in BigQuery transforms the data and AI journey through assistance and automation. Attendees learned how to get assisted SQL and Python data analysis, to generate or suggest code in SQL or Python. Her another talk, Building and deploying an AI agent with LangChain on Vertex, shared how LangChain on Vertex AI helps developers simplify the complexities of deploying and managing your AI agents.
Vertex AI
Analyze any GitHub Repo in 10 Minutes with Gemini Pro (Colab Notebook) by AI/ML GDE Juantomas Garcia (Spain) is a step-by-step guide for analyzing a GitHub repository using Gemini Pro.
Building and Deploying an Agent with Reasoning Engine in Vertex AI by AI/ML GDE Rabimba Karanjai (US) introduced Reasoning Engine (LangChain on Vertex AI) which helps build and deploy an agent reasoning framework. He explained how it gives you the flexibility to choose how much reasoning you want to delegate to the LLM and how much you want to handle with custom code. He also shared how it integrates with the Python SDK for the Gemini model in Vertex AI, and can manage prompts, agents, and examples in a modular way.
Firebase
Zero to Serverless AI Agents with Firebase Genkit and IDX at DMCON by DevOps Malayalam by AI/ML GDE Anubhav Singh (India) explored how to build and deploy serverless AI agents using Firebase Genkit via Project IDX.
Revolutionize Your App with Genkit — Gen AI Magic Unleashed by AI/ML GDE Pankaj Rai (India) was a session introducing Firebase Genkit is and how to get started with it. He also covered how Genkit can solve the problems that occur by directly using APIs on the client side.
Building SLM powered apps with Genkit and Ollama by AI/ML GDE Anubhav Singh (India) introduced small language models and looked into how to use them with Ollama and Firebase Genkit.
