NeuML — 2023 Year in Review
Recapping 2023 and looking ahead to 2024
NeuML is the company behind txtai, an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows. We are building a suite of applications to bridge the gap between research and production.
NeuML continued to build on it’s strong open-source foundation in 2023. The majority of our focus throughout the year was on txtai and our consulting efforts. This article will recap the progress made in 2023 and look ahead to 2024.
txtai
txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows. This is the foundational piece of software that all of our work stands on.
Highlights for txtai in 2023:
- ⭐2,660 stars on GitHub to bring the total to ⭐5,790
- 300 total commits on GitHub
- 211 total issues resolved on GitHub
- 7 releases. Entered the year at v5.2.0 and finished at v6.2.0
- 24 articles and example notebooks added
Let’s recap the major functionality released.
txtai 6.0
txtai 6.0 was released in August 2023. This release added sparse/keyword indexes, hybrid indexes, subindexes and improved LLM support. The following sections cover these new features.
Sparse Indexes
Sparse indexes add another dimension to txtai. In addition to vector search, keyword importance can now be integrated. Sparse indexes were a large engineering effort in that it required a full BM25 implementation given there weren’t any performant Python-based options.
txtai 6.0 added a performant sparse index component with speed and accuracy on par with Apache Lucene. More can be read on this in the article below.
Hybrid Search
Building on sparse indexes is hybrid search. Hybrid search combines the results from sparse and dense vector indexes for the best of both worlds. Sparse and dense query results are combined to improve search relevancy. More on hybrid search can be read in the article below.
Subindexes
Previously, each txtai index only had a single associated vector index. Starting with txtai 6.0, indexes can now have multiple associated vector indexes. Subindexes add a number of new ways to build txtai embeddings instances. More on subindexes can be read in the 6.0 release article.
LLMs and RAG
2023 was the year of the large language model (LLM). ChatGPT took the world by storm in late 2022 and it’s impact was felt throughout the year. It was well-known entering the year that LLMs would be a major focus but it was even more of a focus than anticipated.
A significant portion of 2023 was spent on LLM integration. New LLMs are being released at a blistering pace and a number of challenges come with each of them. Earlier in the year, most open models struggled with their licensing strategy. This made open-source integration a challenge. Towards the end of 2023, Mistral emerged as one of the best available models licensed under easy-to-understand terms (Apache 2.0).
In 2023, txtai became a major player in the retrieval augmented generation (RAG) space. txtai combines a strong vector search component with a robust LLM framework, hence the all-in-one embeddings database tagline.
See the article below for a comprehensive example of RAG with txtai.
Attention to detail
Crafting a project one believes in has a number of sneaky time sinks. In 2023, significant time was spent on reformatting documentation, rephrasing messaging and improving txtai’s graphics.
For example, txtai’s logo was updated:
From
To
Writing documentation, updating websites, publishing content and otherwise finding the best way to share content about txtai. This is a crucial part of the process and one that took a fair amount of time in 2023.
Other Projects
In addition to txtai, a number of subprojects have been created over the years. The strategy with each of these projects is to build an initial implementation and support future work based on interest.
The biggest current downstream project is paperai. paperai is a semantic search and workflow application for medical/scientific papers. It helps automate tedious literature reviews allowing researchers to focus on their core work. paperetl is a companion project for parsing medical literature. The paperai stack has been integrated in a number of NeuML’s consulting efforts (see consulting section below).
NeuML also has the following open source libraries.
- txtchat — 💭 Conversational search and workflows
- txtinstruct — 📚 Datasets and models for instruction-tuning
- codequestion — 🔎 Semantic search for developers
- tldrstory — 📊 Semantic search for headlines and story text
NeuML previously had a hosted version of neuspo available. Unfortunately, this was discontinued in 2023 due to X (formerly known as Twitter) deprecating their free streaming API.
These “other projects” will continue to be supported on an as-needed basis.
Community
Prior year versions of this article would list out all the mentions across the community. This would be a much more time-consuming effort in 2023 given the increased interest in txtai. Thank you to the amazing community growing around this platform!
One item that will be called to attention is published research work using txtai. There are a sizable number of papers citing txtai and NeuML. The highlight of the year was txtai being featured in DebateKG at EMNLP 2023. Thank you to Allen Roush for graciously including txtai as part of his work.
Consulting Services
NeuML provides consulting services around our open-source stack:
- Generative AI Build retrieval-augmented generation (RAG), large language model (LLM) orchestration and chat with your data systems
- AI-driven Literature Analysis Automate analysis of unstructured medical, scientific and technical literature
- Model Development Create AI, Machine Learning and/or NLP models that excel in industry-specific domains
- Advisory and Strategy Support Leverage our expertise to plan your data, engineering and AI strategy
- Speaking Engagements Discuss txtai, industry trends, insights and developments in the space with your team
- Group Training Custom training sessions to learn how to integrate the txtai stack in your environment
Our efforts in 2023 were once again centered around txtai. Consulting work is symbiotic with our open-source projects, each helping to push the other ahead. This is the main source of revenue for NeuML and required for the viability of the company, as it is structured today.
NeuML delivered a number of RAG projects in 2023 using txtai and paperai. Through these consulting efforts, we have developed strong expertise in the medical literature domain. This is important and extremely fulfilling work. The lessons learned also have broad applications across other domains.
This symbiotic relationship only got stronger in 2023. The development of txtai’s LLM component was a major factor in all our projects throughout the year. One of the most production-ready applications of Generative AI is RAG. Best practices and improvements were integrated back into txtai. We expect to build on this even more in 2024, more on that later.
Rating our progress in 2023
We’ve covered quite a lot of information already recapping 2023. Next, let’s discuss how we stacked up against what we set out to do back in January. This section will cover both txtai and our consulting services. Each goal will be rated from 1–5 with 5 being the highest and 1 the lowest.
txtai
These were the goals set at the beginning of the year. Each goal is an abbreviated version from NeuML’s 2022 Year in Review article.
Going small
txtai wants to make it easy to build small to medium systems with easy-to-enable options to scale up as needed. Add complexity when necessary not by default.
⭐⭐⭐⭐⭐ (5 of 5)
txtai is a strong player in the small to medium project space. It can also be scaled up to large systems. Much work was spent in 2023 on messaging, sharing notebooks and applications. The majority of development was already there, it was just a matter of making it a point, which is was.
Micromodels
Can we create vector and similarity search models under 1MB for limited-resourced devices? Stay tuned!
⭐⭐⭐ (3 of 5)
There was a sustained push on this in the first quarter of 2023. This article covered the progress made at the time. Unfortunately, other efforts superseded this initiative. This is an area we’ll set out to pursue again in 2024.
Generative semantic search
Generating knowledge-grounded content. This will be a focus in the coming year.
⭐⭐⭐⭐⭐ (5 of 5)
This is also known as retrieval augmented generation (RAG). A significant investment of time and energy was placed here in 2023. NeuML is now reaping the benefits of this work though a number of exciting consulting opportunities. Couldn’t ask for a better outcome with this goal.
Multimodal indexing
Work will be done this year to add flexibility with how txtai indexes content. This will lead to easier-to-build multimodal indexes.
⭐⭐⭐⭐⭐ (5 of 5)
Sparse indexes, hybrid indexes and subindexes were added with txtai 6.0. Significant progress was made, enabling new ways to search data with txtai.
Cloud offering
Adding a cloud offering enables rapid development, especially for those with small and/or overloaded technical teams.
⭐⭐ (2 of 5)
Like micromodels this was superseded by other efforts. NeuML and txtai weren’t ready to have a cloud offering in 2023. Fortunately, there were significant advancements in 2023 that make this a plausible goal in 2024.
Consulting
Looking ahead to 2023, we will continue to build on the success of our current projects. We will also ramp up our outreach efforts to demonstrate how txtai can be applied to other domains.
⭐⭐⭐⭐ (4 of 5)
The number of projects NeuML undertook expanded in 2023. These projects were primarily developed using retrieval augmented generation (RAG).
RAG is a hot topic and txtai is a major player in that space. It’s a testament to the symbiotic relationship between open-source txtai and paid consulting efforts.
Overall
In 2023, the self-proclaimed score for NeuML is 🥁 🎶
⭐⭐⭐⭐ (24 of 30)
This averages out to a 4 out of 5, a fair assessment. Goals are that, goals. Sometimes you hit them and other times you don’t and you learn how to do better. 2023 was a great year for NeuML but that doesn’t mean there isn’t room for improvement in 2024.
Playbook for 2024
Looking ahead to 2024, we’ll focus on the following areas.
Generative knowledge graphs
Retrieval augmented generation (RAG) powered by knowledge graphs. All the pieces are in place with txtai’s semantic graph component to use knowledge graphs as context for LLM generation. By the end of 2024, switching between vector search and knowledge graph search will be seamless.
Micromodels
Models that can run on limited-resourced systems such as microcontrollers, phones and embedded devices. We expect significant progress from the overall AI community on this front.
At the end of 2023, a couple of exciting and relevant papers came out.
- EELBERT: Tiny Models through Dynamic Embeddings
- LLM in a flash: Efficient Large Language Model Inference with Limited Memory
We’ll build on our work started in 2023 and integrate with these new community efforts.
Cloud offering
Adding a cloud offering enables rapid development, especially for those with small and/or overloaded technical teams. It also adds a hosting platform for NeuML’s consulting projects.
Join the txtai.cloud preview to receive updates as this progresses in 2024.
Consulting 2x
This article has mentioned the symbiotic relationship between NeuML’s consulting services and open-source projects multiple times. It was abundantly clear in 2023 that having these projects is a great way to strengthen the overall txtai platform. It’s also NeuML’s main source of revenue.
In 2024, we’ll set out to double our consulting efforts over what was done in 2023. This will be a combination of doubling down on our existing projects and seeking out new work.
Community engagement and training
NeuML and txtai have become recognized in the overall AI community. There is a solid presence on social media and other platforms. We also held a number of txtai talks with graduate-level college students, meetup groups and corporate teams in 2023. While there is still much work to do, progress is being made.
Like NeuML’s consulting work, speaking engagements and training provide immense value to our open-source projects. We’ll look to do a higher volume of this in 2024.
Wrapping up
This article covered the state of NeuML at the end of 2023 and our plans for 2024. We’re incredibly optimistic on the future of the AI space and NeuML!
Thank you for reading. Please follow along and check in on how we’re doing over the course of 2024.
Interested in NeuML’s history? Then read the recaps from 2020, 2021 and 2022.
