[MLDP Newsletter] Dev 2023 — Machine Learning Communities: highlights and achievements
Let’s explore highlights and accomplishments of the vast Google Machine Learning communities over the month. We appreciate all the activities and commitment by the community members. Without further ado, here are the key highlights!
Generative AI
Gemini
Getting Started with Gemini Pro on Google AI Studio & Gemini Pro + LangChain — Chains, Mini RAG, PAL + Multimodal by ML GDE Sam Witteveen (Singapore) are tutorial videos for Gemini Pro.
Ask Questions Directly to Invoices using Google’s Gemini Pro Vision by ML GDE Bhavesh Bhatt (India) is a tutorial showing how you can use a model to ask questions directly on your invoices using Gemini Pro Vision in Google AI Studio. Also, I’ve Built a Diabetes Management App With Google’s New AI Gemini, demonstrates the development process with Gemini Pro Vision.
Gemini AI and Python: My first app by ML GDE Linda Lawton (Denmark) is a tutorial explaining how to use Gemini with Python to read a webcam and send multimodal requests to Gemini Pro Vision model. She also shared a written tutorial.
PharmaScan by ML GDE Nitin Tiwari (India) and Aashi Dutt (India) is an ML app powered by Gemini Pro Vision API that empowers you to scan your medicines and analyze them for instant prescription information.
Two Voice Devs Episode 175 — Gemini: A First Look by ML GDE Allen Firstenberg (US) and ML GDE Linda Lawton (Denmark) discussed Gemini’s potential for multi-modal support, unique reasoning capabilities, and the challenges.
LLM: the power of PALM (& Gemini) in your hands by ML GDE Lesly Zerna (Bolivia) talked about what Gen AI is, how to start with Bard, how to use MakerSuite, and a short info of Gemini.
Responsible AI: Interpretability, Privacy, and Governance by ML GDE Xiaohu Zhu (China) covered interpretability, privacy, and governance, and introduced Gemini’s best practice on responsible AI.
Performance of multi-modal large language models on different tasks GPT-4V/Gemini/Sphinx comparison test by ML GDE Qinghua Duan (China) introduces the effects of three multi-modal LLMs on different tasks and comparisons of their performance based the comparison paper of the three models.
Gemini + AlphaCode2 Technical Paper & Gemini hands-on by TFUG Taipei was to introduce Gemini to the community and get the people started using Gemini for their ML apps. The group also hosted Parameter-efficient fine-tuning of GPT-2 with LoRA and introduced a training flow with Keras.
Technical Report Summary of Gemini by Thomas Chong summarized the technical report into a 9-mins blog post on Medium for ML enthusiasts to know more about the capabilities of Gemini, and this post hit 315+ claps.
AI Studio & PaLM
Using LLMs to Bridge the Fuzzy Human / Digital Computer Boundary — Tools for EVERY Developer (slides) by ML GDE Allen Firstenberg (US) introduced how to tap into the power of LLMs using REST APIs skills. He shared how to use the PaLM model through Google AI Studio and Vertex AI. Two Voice Devs Episode 170 — At the Hub of MakerSuite and LangChain also shares the practical uses and advantages offered by Google AI Studio.
Discovering PaLM by ML GDE Aqsa Kausar (Pakistan) covered Gen AI studio in Google Cloud. In particular, she focused on the language component to share how PaLM can be used to generate text.
PaLM API & MakerSuite by ML GDE Jun Jiang (China) introduced how anyone can easily create AI apps with LLMs and Google AI Studio.
Create Gen AI apps with Google models by ML GDE Nathaly Alarcon (Bolivia) delivered a demo and guide of Model Garden and Google AI Studio with Chirp, Codey, and PaLM 2.
AI In the “PaLM” of Your Hands by ML GDE Stephen Wylie (US) covered a variety of LLMs topics so people can make their own models on top of PaLM using AI Studio. He showed how much can be done without writing code. He also covered advanced retrieval techniques such as LangChain agents, RAG, and etc.
It looks like it’s alive! How Large Language Models work by ML GDE Bianca Ximenes (France) talked about language models and LLMs, PaLM 2 API, Markov Chains, and Probability.
PaLM 2 — From dates to palm trees of language by ML GDE Moises Martinez (Spain) showed how PaLM 2 works and how it generates results and its possible applications.
LLMs
Can we be satisfied with LLM as Service? (slides) by ML GDE Chansung Park (Korea) highlights the power of Gen AI, the importance of prompt engineering, the almighty LLM to control other text-to-anything Gen models, and how we could prepare for the outage of LLM as service. He also delivered a tutorial, LLM: From fine-tuning to its applications, designed to give in-depth explanations about LLMs. He focused on the strength and weakness of LLM, why we need to build alternative solutions with open source LLM, how to develop RAG-like applications, and the importance of embracing LLM into a system.
Mixture of Experts for Dummies by ML GDE Ritwik Raha (India) is a tutorial getting started with the basic concept of a Mixture-of-Expert and navigating it. He covers what MoE is, how it works, and various types of it.
SQL queries + pgvector: Retrieval Augmented Generation in PostgreSQL by ML GDE Rubens Zimbres (Brazil) shows how to use pgvector in a PostgreSQL database to retrieve Top-K results with a SQL query and how to build a product search, whose results will be analyzed and summarized by LLM given an additional context.
LoRA and friends by ML GDE Saurav Maheshkar (UK) at AICamp London was a brief overview of recent studies on parameter efficient fine-tuning methods. He showed how LoRA and follow up works can reduce the computational demands of fine-tuning large models while maintaining metrics.
Less Coding, More Prompt Engineering! by ML GDE Stephen Wylie (US) explains how a software engineer can leverage LLMs to write code. It shows how prompt engineering can make a difference in the quality of output.
Understanding RAG and Fine Tuning for Large Language Models (slides) by ML GDE Tarun R Jain (India) explained what RAG and fine-tuning are and parameter efficient fine-tuning techniques.
Multi-Agent LLM Applications | A Review of Current Research, Tools, and Challenges by ML GDE Victor Dibia (US) evaluates the current state of affairs with multi-agent LLM apps, covering why we need multiple agents; what the landscape of multi-agent frameworks and emerging startups/services is; and etc. Plus he gave a talk, Reliable pipeline for the automatic generation of data viz with Gen AI (LLMs) explaining how to harness the power of LLMs to generate visualizations and infographics for your data. He introduced LIDA, an open-source library for LLM-powered visualization generation as well.
Fine-tuning de Mistral 7B by TFUG Rennes — IA par le Code was an interactive and in-depth session on the refinement of a specific model: Mistral 7B. It aimed to share knowledge and to stimulate a discussion on best practices, challenges and advances.
Trends in the evolution of LLMs and issues and alternatives for corporate use of LLMs by ML GDE Jeongkyu Shin (Korea) is a brief technical introduction to the LLM as well as issues and alternatives for enterprise use of LLM.
Generative AI in 2023 and Beyond by ML GDE Kshitiz Rimal (Malaysia) delved into 2023’s exciting Gen AI scene — from LLMs’ generating epic tales to SOTA models creating modern Mona Lisa with explorations of tools such as AI Studio, PALM API, Vertex AI, TensorFlow, Keras, Hugging Face, and Langchain.
Unleashing the Power of LLMs: A Guide to Responsible AI for the Next Generation (slides) by Usha Rengaraju (India) discussed the security and privacy risks of LLMs.
Deploying LangChain on Cloud Functions by ML GDE Daniel Gwerzman (UK) walks you through the process of deploying LangChain to Cloud Functions. He explores the benefits of using Cloud Functions, demonstrates the deployment process, and discusses how LangChain’s framework can rapidly enhance your AI applications. He also shared a text tutorial.
Vertex AI Grounding Large Language Models by ML GDE Sascha Heyer (Germany) explains how to use Google Grounding instead of implementing a custom solution. This article hit more than 135+ claps on Medium.
Using ML to “Understand” Images — Tools for Everyone to Use by ML GDE Allen Firstenberg (US) explained how to make your own ML model work like Google Lens or Google Photo using VertexAI and multimodal embedding models.
How to build a chatbot using GenAI with Vertex AI conversation and Dialogflow CX by ML GDE Yannick Serge Obam (Cameroon) introduced the latest Gen AI features in Vertex AI Conversation & Dialogflow CX and how to combine traditional agent design techniques and best practices of Google’s new LLM to create complex conversational applications.
Gen AI vs. Intent-based: The evolution of Dialogflow by ML GDE Mercedes Rodríguez (Spain) introduced how Gen AI is changing the conversational agent development landscape. She presented the new Gen AI features integrated into Dialogflow CX, and how it’s related to Vertex AI Search and Conversation.
Keras
LLaMA backbone to KerasNLP by Anshuman Mishra (India) was a contribution to KerasNLP library adding Llama backbone. Which can now be used with KerasNLP’s modular pipelines.
Using KerasNLP, Large Language Models, and Makersuite to Build Powerful and Scalable NLP Applications by ML GDE Esther Setiawan (Malaysia) presented a guide building powerful and scalable NLP applications showcasing the synergistic potential of KerasNLP and AI Studio.
Introduction to Keras Core: unlocking the power of JAX with Keras Core by ML GDE Kuan Hoong (Malaysia) talked about the power of multi backend with Keras Core.
Making Deep Learning Easier with Keras 3 & Hyperparameter Optimization using Keras Tuner by ML GDE Marvin Ngesa (Kenya) guided practical steps of using Keras 3 & Keras Tuner.
KerasCV for the young and restless by ML GDE Suvaditya Mukherjee (India) introduced basic operations of computer vision and an introduction to Keras and KerasCV as a multi-backend library.
How to survive to Naked and Afraid using an Autoencoder by ML GDE Arnaldo Gualberto (Brazil) presented different types of Autoencoders with Keras and TensorFlow for various applications such as dimensionality reduction, embeddings search, image generation, segmentation, etc.
Convolutional Neural networks(CNN) with Keras and TPUs by Armel Yara (Ivory Coast) was to help beginners learn CNN, Keras, and TPUs and understand the fundamentals.
TFUG Chennai December 2023 Meetup by TFUG Chennai covered 101 topics in KerasCV and KerasNLP. Aashi Dutt (India) covered Getting started with KerasCV and Usha Rengaraju (India) gave Introduction to KerasNLP with a demo.
Kaggle
Community matters: 8 reasons why you should be involved with Kaggle by the first Kaggle GDE Andrada Vulpe (Romania) introduced Kaggle as one of the biggest online communities of data scientists and ML engineers.
Fine-tuning a large language model on Kaggle Notebooks for solving real-world tasks — part 3 by ML GDE Luca Massaron (Italy) explains how to fine-tune for financial sentiment analysis with Mistral 7B Instruct v0.2 and Phi-2.
Use Gemini to Create Student Essays by ML GDE Ertuğrul Demir (Turkey) is a notebook for LLM — Detect AI Generated Text competition. It demonstrates the use of Python SDK for Gemini API, showcasing how to effectively leverage the API for the competition. Another notebook, Train your own Tokenizer, employs an approach to train and utilize tokenizers for detecting LLM-written text. It has been forked 300+ times.
On-device ML
Offline Speech to Text with TensorFlow and the Whisper model by ML GDE George Soloupis (Greece) explains how to convert Whisper, the open-source speech-to-text model into a TF-compatible format.
Give your web apps superpower with Generative AI and Mediapipe by ML GDE Rabimba Karanjai (US) empowered web developers to build custom and cross-platform ML solutions on the web.
Easy on-device Machine Learning with MediaPipe by ML GDE Xiaoxing Wang (China) introduced the concept and characteristics of device-side ML and demonstrated how MediaPipe simplifies the complexities of implementing ML on-device.
MidiaPipe workshop — Custom hand gesture recognization by ML GDE Eliyar Eziz (China) introduced MediaPipe and conducted a MediaPipe workshop training custom gesture recognition models.
ML Research
Summary of some RLHF replacements by ML GDE Rumei LI (China) takes you through the recent RLHF replacement work and explores how to achieve more stable results.
Activities by ML Frameworks
AI + Cloud — Healthcare Access to millions by ML GDE Rafael Figueroa (Brazil) introduced production examples of healthcare AI, such as how my company applied TensorFlow, Vertex AI, Web RTC and GCP in general, to provide healthcare access to Brazilians. He also led a 3-day Healthcare AI workshop at MIT for high profile leaders from organizations & governments and investors & entrepreneurs from several countries. He focused on demonstrating the results of multilateral collaboration to implement healthcare AI for underserved areas in Brazil.
Google Solution Challenge — AI/ML workshop (Building Models using Tensorflow) by TFUG Jalandhar was a workshop focusing on contributing to the United Nations’ 17 sustainability goals with Google’s AI/ML products & technology.
Building an ML driven image tagger on Google Cloud Platform by ML GDE Thushan Ganegedara (Australia) shows how to build a simple image tagger that can tag an user uploaded image with keywords based on the objects present in the image.
Vertex AI:Export and deploy a BQML Model for Prediction (slides) by Armel Yara (Ivory Coast) a talk about MLOps with Vertex AI. They focused on how to build, export, and deploy BigQuery ML model on Vertex AI.
Others
In Global developers use Google tools to build solutions in recruiting, mentorship and more, ML GDE Rubens Zimbres (Brazil) introduced his favorite tools and projects he did with Google products.
AI/ML & Data Talks Podcast Episode 20: Jason Mayes by ML GDE Kuan Hoong (Malaysia) and Googler Jason Mayer discussed Web AI/ML related technologies and introduced TensorFlow.js, MediaPipe, and best examples projects.
Machine Learning Zoomcamp Study Group by Machine Learning Study Group (Myanmar) and ML GDE Aye Hninn Khine (Thailand) is a study group for Burmese developers. Based on the curriculum, 70+ group members help each other learn ML over the online and two participants voluntarily lead for each discussion by delivering technical presentations.
Open Source Deep Neural Speech Model — ConVoiFilter by ML GDE Binh Nguyen (Germany) is a specifically designed model to filter target speaker’s voices. It is based on this paper. This model downloaded more than 160k in a month.
