Lessons Learned from Working as an NLP Engineer

If you have landed on this blog, you probably already know what NLP is and I will spare you a boring introduction. In case you don’t, please follow this link to enjoy an introductory read.

What I am sharing with you today is not a very technical topic but may be useful for fresh graduates or people considering a career in NLP. Being prepared for what awaits you as an NLP Engineer in the industry can greatly contribute to your success. Thus, in this blog post I want to share with you some of my insights on working in NLP and what you can expect from it in the industry.

Since my bachelor’s degree, I have been involved in various NLP projects and part-time jobs. This gave me a lot of experience but still only counted as a playground environment. Before starting a full-fledged career, I had certain expectations but no real understanding of what my career would look like. Now, after working in the industry as a full-time NLP engineer for more than three years, I have gathered many learnings that I want to share with you to keep you in the loop and adjust your expectations.

Of course, keep in mind that all NLP positions are different, and NLP is a broad field. Therefore, take all information with a grain of salt. Some jobs might focus more on machine learning, others might require more linguistic expertise, and some might not even involve text. Furthermore, requirements can vary from company to company and from team to team. Nevertheless, I will try to cover a broad range of topics to help provide a clearer picture of things that might be waiting for you around the corner.

To stay on topic, let’s start from the fact mentioned above that roles are different. This means…

Developing SOTA or cutting edge technology might not be a part of your job

Yes, there is a chance that you will land a job at OpenAI or Google Brain, where you might work on state-of-the-art technology, but usually this is a long journey which typically requires a PhD. Most of the NLP jobs are not like this. There are many different industries out there that urgently need NLP solutions and are looking for people to implement them. This most likely won’t involve brainstorming new powerful approaches for natural language understanding or training yet-another GPT, but that’s not necessarily a bad thing. There are many interesting things happening beyond the NLP frontier.

Funnily enough, even though you might not develop SOTA yourself, you will still be influenced by it.

“Yet-another-new-thing” will haunt you

We are currently in the midst of an AI boom, particularly in the field of NLP. New NLP advancements emerge almost every week, making it challenging to keep up with the extensive research in this domain. The development of NLP has been gradual, with occasional breakthroughs, and has now reached a point where the perceived quality is high enough.

As a result, these new NLP solutions are gaining a lot of attention, and many businesses are eager to adopt them simply because they are trendy. While integrating solutions like ChatGPT into products can be beneficial for certain companies, it is important to recognise that not every new solution is necessary or the most suitable option.

Therefore, be prepared to occasionally contend with these new, enticing advancements when necessary. As an NLP Engineer, you should have a background in various solutions within your field and stay consistently updated on recent advances. Gain a deep understanding of which solutions are best suited for specific tasks, rather than hastily embracing a new solution solely based on metrics or hype.

This decision-making process may involve assessing not only technical constraints and suitability for a particular NLP task, but also considerations related to cost, efficiency, safety, ownership and other factors. Even though these responsibilities may seem beyond the NLP Engineer role, in the age of language AI, they have become an integral part of the job.

Legacy system will haunt you

As an NLP Engineer, there are certain challenges that can make your job more difficult aside from new advancements in NLP, and that’s dealing with legacy systems. If you’re lucky enough to join a startup, you have the freedom to build solutions from scratch without the curse of a legacy system.

However, some companies already have their own NLP systems that they’ve been developing for years, which can become outdated due to new technology, high maintainability, and challenging development cycles. And it’s not just the NLP system itself that poses challenges. The entire company infrastructure may not be easily changed to meet the demands of one team. Facing a legacy system and going head-to-head with it is something you should be prepared for.

When a legacy system underperforms and calls for replacement, the primary challenge lies in determining how a new replacement NLP solution can seamlessly integrate into the existing tech pipeline. This challenge is rooted in the overall company infrastructure, including the degree of decoupling among different components and how well the new stack of technologies aligns with the current setup. Finding a solution that satisfies all parties involved requires collaboration and input from various tech teams within the company.

On the other hand, if the legacy system is deemed good, introducing a new solution becomes even more complex due to the need for benchmarking. You must demonstrate that your proposed solution is better in some regard, which can be a tricky task.

Typically, the development of a new solution begins with prototyping and building a minimum viable product. However, this process can be challenging when the company heavily relies on the legacy system to serve clients and is reluctant to allocate resources to uncertain solutions. Proving value becomes a real challenge.

In this case, prototyping might involve using toy data and making numerous assumptions, leading to expectedly less impressive results. Thus, rather than immediately implementing a new solution, you embark on a long journey, providing a series of milestones that gradually lead to the adoption of your newly created solution and eventually the dismantling of the legacy system.

Research and development are hard to balance

Let’s talk about research in the industry since we touched upon the prototyping topic. There are companies out there that make research a part of their job, while others focus more on serving their clients. Although research and development usually go hand in hand, for the latter, finding a balance between research and development can be a bit tricky since their main goal is to keep clients happy by making continuous improvements.

Clients expect to see fixes for technical bugs, “ML bugs”, the introduction of new features, etc. This means that at times, innovation gets neglected because there are too many development tasks that require attention.

While some companies have separate teams for research and development, more often than not, one team handles everything and you need to be ready to tackle both research and development tasks.

When it comes to research, it usually involves a significant amount of trial and error, exploring different approaches in the quest for the most suitable one. It’s important to be prepared for failures along the way, as they are part of the process of discovering hidden gems.

Non-trivial task of keeping “NLP” in the NLP Engineer

As we mentioned earlier, research in the industry can sometimes lag behind, and if you’re deeply immersed in development, it’s easy to fall behind on the latest NLP trends. It’s crucial to remain proactive and not rely much on the industry to keep you informed. Otherwise, there’s a high chance of quickly becoming “outdated.”

Another aspect to consider is that while NLP is a vast field, working as an NLP Engineer doesn’t necessarily entail working on a wide range of NLP tasks in general. Typically, companies have their specific areas of focus, which means you’ll be working on a select set of NLP tasks. As time goes on, you’ll naturally become more specialised in some particular NLP subfields, for example Sentiment Analysis, Topic Modeling, or Speech Recognition. Sometimes, this specialisation can even extend further into specific domains or languages.

However, it’s important to acknowledge that by specialising in a particular NLP task, you may inadvertently limit your exposure to other areas. Staying connected to the NLP field can be challenging which can limit the opportunity for exchanging ideas and gaining inspiration from other areas of NLP.

NLP Engineer is more than NLP

One of the most significant things I’ve learned is that working as an NLP Engineer isn’t solely about focusing on NLP tasks and becoming more knowledgeable about NLP. While it’s true that you’ll have the opportunity to enhance your knowledge and gain experience in NLP techniques and frameworks, there’s more to the role than that.

In the industry, there is a growing emphasis on being a generalist, which means having knowledge in various areas related to your field and bridging the gap between NLP and other teams within the company. You may have come across the concept of a “T-shaped specialist”, which precisely captures this idea.

It’s highly valued if you possess not only the ability to develop NLP solutions but also the skills to integrate them smoothly into the existing infrastructure using CI/CD practices. Being able to understand product requirements, anticipate clients’ needs, create robust and efficient solutions, and establish high-performance experimental environments are all valuable traits.

From my experience, I never expected to find myself working on tasks like developing APIs, deploying them, engaging with stakeholders and external vendors, practicing MLOps, or dealing with orchestration tools and more.

These learnings naturally come while working in the industry, often extending beyond the realm of NLP. It’s essential to be prepared to expand your knowledge beyond NLP and remain open to learning. These additional skills are incredibly beneficial as they enable you to better comprehend the challenges of the industry and effectively communicate with individuals from diverse technical backgrounds.

Communication + collaboration is the key

Effective communication with various stakeholders has an enormous importance. This includes being on the same page and speaking the same language with other tech teams, as well as clearly communicating with non-technical teams and clients. The ability to distill complex technical concepts for individuals from diverse backgrounds and varying levels of technical expertise will be a valuable asset.

However, the most noticeable advantage lies in being a team player. Communication within the team is vital and should never be underestimated. Team alignment is what brings the most value and fosters effective collaboration. By improving communication and sharing information within the team, many potential problems can be addressed from the outset.

It took me some time to realise this but prioritizing speedy delivery may not always be the best approach. Taking the time to ensure that all teammates are well-informed and aware of project developments can yield additional benefits. Ultimately, the goal is to empower every team member to learn and be able to deliver results. Overall, increasing the bus factor, or the number of individuals knowledgeable about a particular project, is of utmost importance.

Embrace a full product life cycle

In the blog post discussing NLP Engineers, I cannot skip talking about development. While I won’t delve into the nitty-gritty details that are extensively covered in other blog posts, it is important to highlight the broader perspective. Developing an NLP solution involves more than simply writing code. It requires considering the entire product life cycle.

Apart from developing the solution itself, you’ll also need to think about how to effectively deploy and monitor it. This encompasses various aspects such as system evaluation, testing, continuous improvement, deployment strategies, scalability, gathering client feedback, and comparing different systems in production. Paying attention to the complete product life cycle is crucial. It may even involve exploring adjacent fields like MLOps to effectively manage, maintain and monitor your solutions.

Data is still important

When it comes to development, data is a starting point. Data is often mentioned as one of the bottlenecks not only for ML solutions and this topic has been extensively covered in various online sources. However, I find it important to highlight some higher-level points.

Maybe it is counterintuitive but working with textual or speech data still requires a thorough understanding of the data itself, including its meta information and distribution across different aspects. Yes, I am talking data analysis here. Analyzing this information can reveal limitations and shortcomings in the data, leading to the decision to improve/change the dataset or even pivot a chosen model architecture. Don’t underestimate the importance of data refinement, which may involve adjusting its distribution, balancing it, or even employing data augmentation and reannotation.

Data (re)annotation is another significant aspect, as labeled data is usually not readily available. While you may be fortunate enough to avoid manual annotation, it doesn’t necessarily make things easier. Finding suitable annotation tools and vendors can be a challenge, depending on the complexity of the task. Creating clear annotation guidelines is not a trivial matter, and ensuring annotation quality can be elusive. In fact, my team had to switch annotation vendors three times before finding the right fit for our needs.

Last but not least about the data, working on a specific NLP solution often involves dealing with domain-specific data. While you are the one who is responsible for developing the NLP solution, it doesn’t hurt to become a domain expert or seek consultations from domain experts. This helps prevent the need for significant changes or pivots in the solution at later stages.

Choose your guiding light

Lastly, let’s touch upon the evaluation step. When working in the industry, it’s important to keep clients in mind and ensure that your solutions effectively meet their needs. Apart from many other important decisions that you need to take in favour of clients, this requires careful consideration of what you optimize for. Common metrics might not always accurately reflect the practicality and success of your solution. It’s crucial to align your metrics and objective functions with your ultimate goal.

Additionally, metrics should not be the sole determining factor in choosing your solutions. Other important factors such as explainability, efficiency, latency, robustness, bias and responsibility, scalability and maintainability should also be taken into account. Making decisions is often a balancing act, weighing different factors and considering trade-offs. In my own experience, more performant models were sometimes discarded in favor of faster and more robust solutions.

Of course, this is not an exhaustive list of learnings, but I have tried to capture some of the most striking ones based on my personal experience. There are other important topics to explore, such as hiring considerations and tips for managing work-life balance while ensuring efficient development processes. Perhaps, we can delve into those topics another time :)

I hope you have found something interesting in this blog post, and I haven’t scared you away. I assure you that working in the field of NLP is both fascinating and versatile and if you go this way, you will embark on your own unique journey, which may differ from mine but is equally exciting.

Till the next time! 👋

P.S. This blog post and illustrations were created by a human being!