GPT-4 Turbo vs Gemini vs Open Source Community
It seems like this quarter’s AI sprint is getting close to the end! A few days ago, we heard from Microsoft about GPT-4 Turbo, DALL-E 3, multi-modality with Search Grounding and more. Yesterday Google announced Gemini, and it is using the current GPT-4 specs for its multi-modal performance benchmarking, which seems to outperform on all benchmarking datasets (except for HellaSwag for general reasoning). Which is indeed exciting!
Now, I wonder how much better GPT-4 Turbo is going to be in comparison to the current GPT-4 that Gemini seems to surpass. It would not only depend on how well these models perform on these benchmarking datasets, but things like costs, context window, speed, availability, accessibility, and so on. Either way, it seems like it would not take that long for us to get a Jarvis of our own on our phone, or a pin, or maybe glasses, or even inside of your brain as an implant!
The tech report [3] for Gemini looks also quite exciting. It would be a great read on my way to NeurIPS 2023 in New Orleans this Saturday (if anyone else is going!).
In particular, this figure 2 sticks out to me. I would love to see how they ETL the data from different sources into a single funnel. It seems like they somehow become concatenated into a long sequences of tokens. And, I would love to learn more about that process!
This is a brief video [4] from Google on introducing Gemini and some of its functionalities
Recently, I had some free time deploy OpenChat model locally on my laptop and ran it on CPU. It took about 3–4 mins for each 400–500 response tokens, but if I have the time to wait for it to spit out all the responses in sequence, the quality is worth the wait especially for a 7B model. If you are curious, the reason why I am running the model on CPU is that the model is 24GB, and my GTX3070 has only 8GB. If you have a several GPUs, you can actually parallelize the process by adding this flag --tensor-parallel-size N to your command line.
My thoughts on GPT-4 Turbo, Gemini and all other upcoming SOTA models is that Open Source community will soon catch on and make something equivalent if not better. For that, I think this open research movement has a non-trivial value not just in pushing boundary of AI research and democratizing AI capabilities but also to enforce ethics and guidance around proper AI practice.
With all these advancements in AI, I am hoping to get myself a couple brand new GPUs and start playing around different models myself. I could certainly just buy a subscription and use the API, but my ego as a ML engineer/scientist don’t make that a simple decision (plus the charge by token and cloud compute cost can add up).
Anyways, I hope everyone is as hype up as I am about this unprecedented advance in machine intelligence. Not too long ago, people were skeptical about very idea of the neural network, and consequently we all went through the AI winter(s). Now it’s a Spring again, and not just any Spring, but Miami Spring where all four seasons are warm and hot! I am carefully optimistic about where we are heading, and hopeful that we will all live better life at the end.
Please feel free to leave any comments to share your thoughts! Thank you for reading, and as always, enjoy your AI journey!
Reference
[1] https://deepmind.google/technologies/gemini/#capabilities
[2] https://blog.google/technology/ai/google-gemini-ai/?utm_source=gdm&utm_medium=referral#capabilities
[3] Gemini: A Family of Highly Capable Multimodal Models
[4] https://www.youtube.com/watch?v=jV1vkHv4zq8
[5] https://www.actuaries.digital/2018/09/05/history-of-ai-winters/
If you want to read more about AI & ML subjects and more, please come check out my Medium articles!
or visit me in different communities!
def visit_me_elsewhere():
Website = 'https://modrev.org/jshinm' #personal website
Linkedin = 'https://www.linkedin.com/in/jshinm/' #Linkedin
Twitter = 'https://twitter.com/JongMShin' #aka X
