[MLDP Newsletter] Nov 2023 — Machine Learning Communities: highlights and achievements
Let’s explore highlights and accomplishments of the vast Google Machine Learning communities over the month. We appreciate all the activities and commitment by the community members. Without further ado, here are the key highlights!
Generative AI/LLM
MakerSuite / PaLM
How to leverage Vertex AI Palm API to take Customer Service to the next level by ML GDE Gabriel Moreira (Brazil) uploaded on the Google Cloud Brazil blog. He provides a Colab notebook demonstrating how to tackle different use cases of customer service analytics and automation with Vertex AI PaLM API. His other project, Photo StoryTelling — Leveraging Generative AI and Google APIs to compose posts from your photo albums was also shared. It demonstrates how you can use Gen AI to write a blog post capturing the moments registered in a photo album. It is accompanied by a Jupyter/Colab notebook containing the whole solution, which includes EXIF photo metadata extraction, uses Google Maps API for extracting place information where photo was taken, and Gen AI APIs like Vertex Imagen (for image) and PaLM API (for text generation).
No-code Prototyping with Flowise (slides) by ML GDE Martin Andrews (Singapore) at Devfest Singapore 2023 included live demonstrations using the Flowise UI to which he had contributed PaLM API ‘nodes’, as part of the MakerSuite Sprint. It included converting a formatted question into a REST API without programming; and building a RAG system to answer questions about Singapore.
MakerSuite Quick Guide by ML GDE Sam Witteveen (Singapore) provides an overview of the new MakerSuite features. He also shared Fine Tune Palm 2 + How to make a dataset which explains how to fine tune a PaLM model. It reached 10K+ views.
Support in LangChain JS to use MakerSuite with Google Drive as a source of template by ML GDE Allen Firstenberg (United States) known as “MakerSuite Hub” allows people to create prompts and model configurations in MakerSuite that are stored in Google Drive. With 3 lines of code in LangChain JS, a developer can load the prompt and turn it into a template, a model object, or a chain of the two. His other contribution, VertexAI PaLM Streaming Support for LangChain JS also helps reduce response time.
Building an AI Chatbot using Flutter with Makersuite and Palm API: A Step-by-Step Guide & Creating a Q&A Bot Quickly in Google Colab with Makersuite and PaLM API by ML GDE Esther Irawati Setiawan (Indonesia) guide you creating user-friendly AI chatbots using PaLM API and MakerSuite.
Build your own Gen AI Apps with Maker Suite and PaLM API & Exploring Creativity with Generative AI: PaLM API and MakerSuite by ML GDE Nathaly Alarcon (Bolivia) shared introductions to Gen AI at two Devfest events.
Using LLMs to bridge the Fuzzy Human / Digital Computer Boundary — tools for EVERY developer by ML GDE Allen Firstenberg (United States) at Devfest Montreal introduced the PaLM model, insights on LLMs’ usage in a wide range of applications, and examples of using LangChainJS to access Google’s PaLM model through the MakerSuite PaLM API and/or the Google Cloud Vertex AI API.
Getting Hands-on with MakerSuite and the PaLM API by ML GDE Ruqiya Bin Safi (Saudi Arabia) delved into the future of AI development by exploring MakerSuite and the PaLM API to create LLM applications.
Building Generative AI Applications Using Vertex AI by ML GDE Yüksel Tolun (Turkey) covered the basics of Gen AI, PaLM APIs and Gen AI support on Vertex AI.
More activities
A series of Youtube videos: RAG: Advanced RAG 01–05 (playlist for RAG) by ML GDE Sam Witteveen (Singapore) covers examples of techniques to do Retrieval Augmented Generation with LangChain and other tools.
Beyond the Basics: Mastering Retrieval-Augmented Generation in 90 Minutes by ML GDE Martin Andrews (Singapore) and ML GDE Sam Witteveen (Singapore) was a workshop at Devfest Singapore. They explored how RAG combines knowledge retrieval with generative models, strengthening AI’s ability to address complex challenges.
Retrieval Augmented Generation — Giving LLMs access to source documents hosted by Machine Learning Singapore MeetUp covered a hot topic in the LLM space. ML GDE Sam Witteveen (Singapore) talked through the basic ideas of RAG, and how it can be extended via more advanced engineering techniques. Peng Zhao discussed building a RAG-enabled chat-bot MVP. ML GDE Martin Andrews (Singapore) gave a forward-looking talk (slides) including ‘RAG-fusion’, MEM-GPT (and LLMs as operating systems) and etc.
Two Voice Devs Episode 168 — Defining Retrieval Augmented Generation by ML GDE Allen Firstenberg (United States) covered an exploration of what RAG means in the Gen AI and conversational AI world.
Code Generation using Retrieval Augmented Generation + LangChain by ML GDE Rubens Zimbres (Brazil) presents RAG + LangChain for code generation, using additional context as embeddings to generate a customized version of Duet AI.
Image Generation using Vertex AI Imagen by ML GDE Sascha Heyer (Germany) walks you through the new foundation model, Imagen. Protect your LLM against Prompt Injection in Production & How to Fine Tune LLMs also outlines useful techniques in the LLM field.
Efficient Fine Tuning Methods: Exploring LoRA by ML GDE Saurav Maheshkar (UK) gave a brief overview of recent studies on parameter efficient fine-tuning methods. He explored how LoRA and subsequent work can significantly reduce the computational demands of fine-tuning large models while maintaining metrics.
Generative AI in 2023 and Beyond by ML GDE Kshitiz Rimal (Philippines) covered LLMs generating epic tales to SOTA models creating the modern Mona Lisa. It included explorations of tools such as MakerSuite, PALM API, Vertex AI, TensorFlow, Keras, Hugging Face, and Langchain.
Large Language Models and Fine Tuning Recipes by ML GDE Tarun R Jain (India) at DevFest Bhilai shared why fine-tuning is needed, how to downsize your model, PEFT, LoRA, QLoRA, quantization, and etc.
LLaMA 2 and LLaMa Code by ML GDE Juantomas Garcia (Spain) is about the new versions of LLaMA 2 and LLaMa Code and that they are integrated in the Model Garden of Vertex AI.
Generative AI on Google Cloud by ML GDE Ralph Regalado (Philippines) was a talk at Devfest Bacolod 2023. It was about the Gen AI products under Google Cloud such as Foundational Models, Vertex AI Search, and Vertex AI Conversation.
A deeper look into LLM (video) by ML GDE Mohamed Buallay (Bahrain) at Devfest Almaty; Fine-tuning LLMs in Vertex AI by ML GDE Olayinka Peter (Kenya) at Devfest Nairobi; and Building LLM Applications With Vertex AI by Derrick Mwiti (Kenya) at Devfest Pwani were sessions exploring Vertex AI and its potential in the context of LLMs.
AI for everyone: communities and devs by ML GDE Lesly Zerna (Bolivia) introduced AI, mostly to Gen AI and how to get started an LLM as best companions for software developers and also for communities’ members for their events.
Devfest Indore 2023 by TFUG Indore & GDG Indore covered a wide AI field including sessions like Duet AI : Your assistant in Google Cloud, Overview of Gen AI on Vertex AI, Firebase for Startups: Scaling Your App from Zero to Hero, and Building advanced Gen-AI solutions using Google Cloud.
Keras
RAGging Up with Keras and Vertex AI by ML GDE Wei Zheng (China) showed how to build a RAG pipeline using Keras NLP and Vertex AI. He discussed the basics of RAG, and a step-by-step guide on how to build a RAG pipeline.
Keras Reloaded by ML GDE Song Lin (China) at Devfest Beijing shared the new features of Keras and what to look out for using it.
How to train a GPT model using KerasNLP? by ML GDE Jerry Wu (Taiwan) delivered what Gen AI is and how to train a GPT model using KerasNLP. He also gave a talk on Using PaLM API to analysis the legal documentation and shared how to analyze judgment papers using PaLM to extract criminal content from legal documents. Using the PaLM API, a prompt can be formulated to conduct analysis extracting key information such as criminal, nature of the crime, tools used in the crime, and locations.
Deep dive with Keras: A hands-on workshop by ML GDE Charmi Chokshi (United States) at Devfest ATL & Ottawa were sessions with hands-on practice. It explored Keras and how it helps with computer vision (KerasCV) and language processing (KerasNLP). The goal was to give people useful skills for today’s job market and to help them feel confident using Keras to solve problems.
ML for Everyone: No Coding with Google Sheets, Low Coding with MediaPipe, and Full Stack with Keras by ML GDE Guan Wang (Singapore) at Devfest Singapore was an introduction and demo of Simple ML for Google Sheets, MediaPipe API, MediaPipe studio, the newest Keras Core, KerasNLP, KerasCV and some of their gen AI models.
Powering your Model for Creating a Great Application using Keras Core by ML GDE Joan Santoso (Indonesia) at Devfest Bogor was about new technology of Keras Core. He discussed the advantages of Keras Core to build a model; how it can accommodate multiple frameworks; and how to blend the backend operation.
NLP Supercharged: Building Scalable and Powerful NLP Applications with Keras NLP,LLMs, and Makersuite by ML GDE Esther Irawati Setiawan (Indonesia) at Devfest Balil discussed how to use Keras NLP, LLMs, and Makersuite to build powerful and scalable NLP applications.
Getting Started with KerasNLP by TFUG Taipei introduced open-source models from Keras and actual code for specific conditions.
Advanced Workshop on Contrastive Learning with TF Keras by MLNomads focused on developing a robust codebase for data preparation, specifically tailored to facilitate contrastive learning methods for face recognition tasks. This session introduced an in-depth review of the ‘FaceNet’ research paper, which provided a comprehensive understanding of the face recognition techniques.
Kaggle
[KerasCV] train and infer on thumbnails by ML GDE Aritra Roy Gosthipaty is a new starter notebook using Keras for UBC Ovarian Cancer Subtype Classification and Outlier Detection (UBC-OCEAN) competition. It guides you through training a CNN model with KerasCV.
Machine Learning with Kaggle by Usha Rengaraju (India) was a talk delivered in India Edu program. Plus, she shared several Keras notebooks for NeurIPS Machine Unlearning Competition, Zzz — Detect Sleep state competition, Optiver Trading competition, Stanford Ribonanza RNA Folding, and Open Problems -SIngle cell Perturbations.
On-device ML
diffground — Simplifying generative AI with mobile-friendly interfaces (slides) by ML GDE Radostin Cholakov (Bulgaria) was a talk showcasing a mobile app built with Flutter with Firebase Cloud Functions as a backend with Firebase Auth, Hosting, and Remote Config. It uses on-device ML and remote APIs to “remix” photos with models from recent advances in CV such as Stable Diffusion, ControlNet, etc.
Easy on-device Machine Learning with MediaPipe by ML GDE Xiaoxing Wang (China) introduced the device-side ML framework, MediaPipe. He explained the concept and characteristics of device-side ML and demonstrated how MediaPipe simplifies the complexities of implementing ML on-devices. He also led a workshop, Supercharge your web app with Machine Learning and MediaPipe at Devfest Xiamen.
Use TensorFlow Lite Model Maker with a custom dataset by ML GDE George Soloupis (Greece) explores the practical application of TF Lite Model Maker using a custom audio dataset.
Implementation of MediaPipe on Raspberry Pi by ML GDE Yucheng Wang (China) introduced MidaPipe and showed how to write codes and run the demo on Raspberry Pi.
The power of Embeddings and MediaPipe to find similarities in texts by ML GDE Juan Guillermo (Mexico) talked about the use of embeddings, how to obtain them, and what functions to operate on them, and all this in a very simple way with MediaPipe. His article, How to add text similarity to your applications easily using MediaPipe and Python also explains who to use MediaPipe for mobile applications.
Machine Learning — SimpleML & MediaPipe by ML GDE Rena Baba (UAE) introduced ML project workflow and Google products for beginners & intermediate developers at Devfest Accra.
Introduction to On-device ML Solutions with MediaPipe by TFUG Hajipur was to empower participants with a deep understanding of on-device ML solutions using MediaPipe, fostering developers and enthusiasts ready to leverage the transformative capabilities of real-time processing.
Applytics — Navigating Business Growth via Al-powered Apps with TensorFlow and Flutter by TFUG Hyderabad aimed to provide insights into leveraging cutting-edge technologies to build intelligent applications that can drive business success in the digital age, and explore the smart intersection of AI driven apps using TensorFlow &Flutter.
ML Research
ML GDE Grigory Sapunov (UK) shared an overview of Geoffrey Hinton’s paper, The Forward-Forward Algorithm. He also shared a blog post about Mortal Computers. Plus, he shared a review of ConvNets Match Vision Transformers at Scale, a new paper from DeepMind. For more reviews, you can subscribe to his channel on Facebook or Telegram.
Machine Learning Journey with JAX by Burak PASTIRMACI (Turkiye) covered the basics of ML. For beginners, he shared how to use JAX and TensorFlow for ML research. He also gave a talk, “Computer Vision with Jax/Flax” at Devfest Duzce.
Activities by ML Frameworks & Others
Duet AI Review: My Perception and Use Cases — Part 1 and Part 2 by ML GDE Rubens Zimbres (Brazil) presented his perceptions about the usage of the tool. He approached some use cases: the use of Duet AI to parse logs in Google Cloud Logging (Logs Explorer), how to easily create SQL queries in BigQuery with Duet AI, how Duet AI can help understanding Google Cloud documentation and extracting exactly what you need, and etc.
#TensorFlow.js Realtime clothes segmentation with Tensorflow.js by ML GDE Hugo Zanini (Brazil) is a Segformer model on the ATR dataset enabling real-time usage right in the browser.
#TensorFlow #TF Lite Post-training quantization approaches with TensorFlow (slides) by ML GDE Radostin Cholakov (Bulgaria) discussed various techniques to reduce model size, hardware latency, and processing time with little degradation of model accuracy in modern ML systems. He also touched on some SOTA techniques from recent quantization papers and practical applications with TF and TF Lite.
#VertexAI Vertex AI Model Garden by ML GDE Sascha Heyer (Germany) introduced Model Garden as a playground for AI models, with an impressive collection of pre-built foundation models, task-specific models, and Google ML APIs.
#DuetAI Is Google’s Duet AI the competition for Github Copilot? by ML GDE Juan Guillermo (Mexico)
discussed and tested different uses of Duet AI, such as how to use it to create and improve code writing.
In Women in AI, ML GDE Gema Parreno (Spain) was introduced as one of the AI engineers, and she spoke about the field of AI.
Devfest Tashkent by GDG Tashkent was a mega event with ML, Android, Web and Cloud tracks including sessions, workshops, and showcases. Googlers Martin Gorner and Mark McDonald gave talks on Keras Core and MakerSuite/PaLM API respectively. ML GDE Lesly Zerna (Bolivia), Juan Guillermo (Mexico), Sachin Kumar (Qatar), MLAct organizer Imen Masmoudi (Tunis), and Usha Rengaraju (India) led keynotes and ML sessions (session details).
