LLM News: ESM3, CriticGPT, Gemma 2, LLM Compiler, LongRAG, GraphReader

Lots of exciting developments in AI and LLMs in the past couple of days.

1) Heavy Use of Synthetic Data

For instance, I am noticing a lot of applications of synthetic data for all sorts of tasks ranging from unlocking reasoning capabilities in LLMs to training large-scale biological models.

2) Leveraging Structured Inputs and Outputs One trend in LLM research that has been fascinating is the leveraging of external structures like trees and graphs to enhance LLMs. What we are seeing is that it’s not only useful to structure outputs to mitigate erroneous outputs but also to structure inputs via complex structures to enable better task understanding.

3) Hybrid LLMs It’s also interesting to see ideas that propose hybrid systems to make the most out of LLMs. An interesting new paper proposed LongRAG where they combined long-context LLMs with RAG to improve the overall system by significantly lowering the burden of retrievers.

The best way to help you keep track of all these developments is via my LLM News series. Here is my new episode: https://youtu.be/AoeK1b5-NQ0