Since its first release in June 2017, AllenNLP has become an important library for NLP research with over a hundred academic publications referencing it and hundreds of open-source projects depending on it.
To shape the future of the library, we organized the first AllenNLP Summit, where people joined us from across the country. Our wide variety of attendees included people such as Masato Hagiwara, the soon-to-be-author of the first book based on AllenNLP, and Avneesh Saluja, who has applied AllenNLP to production systems at Netflix. It was a rare opportunity to spend a full day working directly with our key users, who provided us with feedback on our existing ideas as well as new directions to consider.
To kick off the event, we heard from key members from the AllenNLP team about possible futures. Noah Smith, the Sr. Research Manager for AllenNLP, emphasized that — while there are numerous pressures on the AllenNLP team — the key goal is to continue to drive impactful research. He presented some of the research directions we are exploring, such as solving problems that require both NLP and other AI fields (such as computer vision), improving the interpretability of models so humans can understand model decisions, and investing in efficiency improvements and computations cost reporting to encourage Green AI.
AllenNLP team members Joel Grus and Matt Gardner presented more tactical future plans during the next phase of the summit. Joel shared our plans to significantly improve the “Simple Server”, a component that lets anyone easily create a shareable web application for a new model. He plans to dramatically improve the look-and-feel of demos created for new models, as well as make it easier for people to reuse components from the main AllenNLP demo. He also highlighted the new callback-enabled trainer, which we learned over the course of the summit has already solved many pain-points. Matt Gardner announced that he is working on a free, web-based AllenNLP Course that will provide comprehensive onboarding for new users. Users can also look forward to significantly improved onboarding documentation and workflows.
Throughout the rest of the day, we heard “lightning talks” from our users who shared their particular applications of AllenNLP as well as their top complaint about the library—we wanted to hear what’s bad so we can fix it! We also participated in several small group discussions on topics such as “AllenNLP in Production” and “Expanding the AllenNLP Community”. In these discussions, our attendees also shared what they liked most about AllenNLP, which included:
- That the team is open and asking for feedback.
- Our JSON API, which makes it easy to create a server and share a new model across an organization.
- The state-of-the-art visualizations we build to make it easier to understand our systems on the public demo.
- Strong baselines across a wide variety of tasks.
- Re-usable implementations for NLP components that significantly reduce boilerplate so users can focus on their model architecture instead of low-level details like padding.
- Configuration-driven experimentation that provides easy replication of experiments as well as a convenient way to manage experiments for hyperparameter tuning.
- Strong tutorials and documentation (although people also emphasized a desire for more).
- The ability to easily save a model, reload it, and have the same results.
During the whole event, we heard many great ideas about where we could take the library, but a key takeaway is that we should make it easier for the community to be actively engaged with the AllenNLP project. One idea that emerged from this discussion was that we split out our models into community-maintained libraries. This was proposed by Matt Gardner and received such strong support throughout the day that we’ve already started working on it. Another idea was to start a Discourse forum similar to PyTorch’s, which will provide a less intimidating place to ask novice questions and will open another avenue for the community to support the AllenNLP library. I’m pleased to announce we just launched an AllenNLP Discourse forum today. Finally, in future quarters, we will better publicize our goals so that the external community knows what we are working on and can collaborate effectively.
I was continually impressed by the passion and excitement our team and our attendees brought to this event, as well as the breadth of ways people were using the library. Overall the event was a huge success, and I look forward to a second AllenNLP Summit next year!