LLM apps 2023: A report by Streamlit, and the Future of LLM applications

This article highlights the Streamlit report: State of LLM Apps 2023, provides insights into the statistics of all LLM applications deployed on the Streamlit Community Cloud in 2023.

We will discuss:

• What is Streamlit?
• Key definitions on LLM landscape
• Key takeaways from the report
• Future implications

Streamlit

Streamlit is an open-source framework designed for the convenient deployment of data science and machine learning applications. Recent developments in LLMs has made Streamlit an attractive deployment tool/UI for developers.

The report contains data from about 30,000 LLM apps built by around 18,000 developers, which were hosted on the streamlit community cloud.

Key definitions on LLM landscape

• Large Language Model (LLM): a model that can understand and generate natural language text. Common LLMs include GPT by OpenAI, Gemini by Google, Llama by Meta, Claude, Cohere, etc. Read this article for more information on LLMs: Large Language Models: Open Source vs Proprietary
• LLM orchestrators: A coherent framework for augmenting various components with LLM applications to improve reasoning, reduce bias, and integrate various other data and tools with the LLM. Common LLM orchestrators are Langchain and LLamaIndex.
• Vector Retrieval: a method used to categorise data (usually unstructured) as a vector of weights to retrieve similar chunks of data. Useful for building LLM applications with an external data source. Vector search methods include Faiss and ElasticSearch. Vector databases include Pinecone, ChromaDB, Weaviate, etc.

Key Takeaways

From the report State of LLM Apps 2023 by Streamlit, we can takeaway the following insights:

1. OpenAI’s GPT models have a monopoly on the LLMs used with 73%.

2. 55% of LLM applications are multi-component. They use LLM orchestrators.

3. Only 18% use Vector Retrieval in their applications. Pinecone and Faiss lead the count charts.

4. 69% of applications are single text input/output. 31% are conversational chatbots. This trend is increasing more towards the use of chatbots over the past year.

5. There are various challenges/concerns in building effective LLM applications for users

6. The outlook of the percentage use of the different LLM tools are presented as:

with OpenAI and Langchain leading the charts.

Future implications

AI has incredible potential to disrupt or foster growth in human society. The increasing adoption of multi-component features with LLMs in chatbots, retrieval-augmented generation and other natural language processing tasks signifies a shift towards more diverse range of applicantions acroos various industries. However, several challenges and concerns need to be addressed for the widespread adoption of LLM applications.

1. Addressing User Concerns: Building trust with users regarding accuracy, privacy, bias, and data security is crucial. Users need to feel confident that the data being generated by the LLM application for it to be used responsibly and ethically. As LLM applications become more prevalent, addressing these concerns will be paramount.
2. Technological Advancements: Continued advancements in devices, compute, LLMs, LLM orchestrators, and vector retrieval methods will drive the evolution of LLM applications. Improvements in these technologies and frameworks will lead to more accurate, efficient, and powerful applications.
3. Skills Development: As the demand for LLM applications grows, so too will the need for skilled developers who can build and maintain these applications. Investing in education and training programs to develop these skills will be essential for the future of LLM applications.
4. Cost Considerations: The cost of developing and maintaining LLM applications can be significant. Finding ways to reduce costs while maintaining quality and effectiveness will be a key challenge for developers and organizations.
5. Open-Source Contributions: The open-source community plays a crucial role in the development of LLM applications. Continued collaboration and contributions from the community will drive innovation and accelerate the development of new LLM architectures and applications.
6. Understanding Diverse Contexts and Industries: LLM applications are being deployed across a wide range of industries and contexts, each with its own unique challenges and requirements. Understanding these diverse contexts and industries will be crucial for developing tailored, technically proficient, effective, and impactful LLM applications.

AI has incredible potential to disrupt or to conjure growth in human society. I am very exited to see how LLMs and its potential applications unfolds over the next few years. Before such a reality is possible, we need to address the user concerns regarding trust, privacy, cost, and skills on building LLM applications. Addressing the challenges and concerns outlined in the report will be essential for realizing this potential and ensuring that LLM applications are beneficial for society as a whole.

Visualise the streamlit report here.