Watson Natural Language Processing: now generally available in IBM Watson Studio Notebooks

Less beta, more languages

Since January, thousands of Watson Studio Notebooks have used the Watson NLP library for high-quality Natural Language Processing in more than 20 languages. I’m glad our users weren’t put off by my blog post on that subject.

It’s time for the next step: as of today, the Watson NLP integration is no longer in “beta” status, and ready to take on your production workload!

Deeper sentiment insights

We added a new model for sentence-level sentiment detection, based on BERT, with improved quality. We’ve also added a utility method that makes it easy to compute document-level sentiment, based on the sentiment of the individual sentences.

And my favorite: we included a new model to detect targeted sentiment. Targeted sentiment is an innovation straight out of IBM Research: it not only determines the sentiment (positive or negative) of a particular phrase, but tells you exactly what the sentiment is about. For example, the sentence “The location was great, but the rooms were dirty.” will yield two pieces of targeted sentiment: (location, SENT_POSITIVE) and (rooms, SENT_NEGATIVE) — without the need to specify targets in advance!

This allows for deep sentiment insights without any custom configuration, as the following example shows. I ran a set of hotel reviews through targeted sentiment — and the result clearly shows me the areas the hotel needs to improve on:

Targeted sentiment for hotel reviews

Support for GPUs

The Watson NLP library is now also included in all our GPU environments by default. This is specifically relevant if you want to train your own classification models based on CNN — you will see that the training time per epoch is reduced, compared to our CPU environments. Running on GPUs may also speed up inference time, i.e., when you apply your custom model to new data.

Getting Started

Using Watson NLP models is still easy as 1–2–3–4:

1. Start a Python notebook with the DO + NLP Runtime 22.1 on Python 3.9, or the GPU Runtime 22.1 on Python 3.9
2. Import the watson_nlp library
3. Load the block that contains the model you need
4. Apply the block to your text

Using Watson NLP in a Notebook

We updated the block catalog in our documentation with all models you can use out-of-the box, and instructions of how to create your own models, including code samples.

We also updated our sample Project in the Watson Studio Gallery: Text Analysis with Watson Natural Language Processing. This project contains sample notebooks that show you how to work with each model, and examples how to further analyze and visualize the results for insights. Most of these notebooks are also available outside the project, directly in the Gallery. Just search for “NLP” in the Gallery to get them all.

Migrating from the NLP beta version

The good news: most of your notebooks will continue to work “as is” once you switched to the new environments. Here’s what you need to do:

• The NLP beta version will be supported for another two weeks. To ensure your notebooks continue to work after that, switch to an environment that contains the new NLP library.
• If you use the Default NLP beta environment: Change the environment of your Notebook from Default Python 3.8 + Watson NLP XS (beta) to DO + NLP Runtime 22.1 on Python 3.9 or GPU Runtime 22.1 on Python 3.9.
• If you use a custom NLP environment: Create a new custom NLP environment, using the software versions DO + NLP Runtime 22.1 on Python 3.9 or GPU Runtime 22.1 on Python 3.9. Change the environment of your notebook to this new custom environment.
• Sentiment analysis: Use the new blocks for sentence sentiment and targeted sentiment. The document sentiment block from the NLP beta is deprecated.
• Tone classification: Use the block name ensemble_classification-workflow_en_tone-stock instead of ensemble_classification-wf_en_tone-stock.
• Emotion classification: Use the block name ensemble_classification-workflow_en_emotion-stock instead of ensemble_classification-wf_en_emotion-stock.
• Custom classification models: We recommend retraining your models with the new NLP library. This will allow you to benefit from quality improvements we made, especially in the ensemble classifier.

As always, we’re looking forward to your feedback, and are excited to bring more innovations from IBM Research and IBM Software to your data science experience!