Natural Language Processing What is NLP Google Cloud Vision?
Natural language processing (NLP), which is the combination of machine learning and linguistics, has become one of the most heavily researched subjects in the field of artificial intelligence. In the last few years, many new milestones have been reached, the newest being OpenAI’s GPT-2 model, which is able to produce realistic and coherent articles about any topic from a short input.
This interest is driven by the many commercial applications that have been brought to market in recent years. We speak to our home assistants Google or Alexa who use NLP to transcribe the audio data and to understand our questions and commands. More and more companies shift a big part of the customer communications effort to automated chatbots. Online marketplaces use it to identify fake reviews, media companies rely on NLP to write news articles, recruitment companies match CVs to positions, social media giants automatically filter hateful content, and legal firms use NLP to analyze contracts and generate contracts. Training and deploying machine learning models for tasks like these has been a complex process in the past, which required a team of experts and expensive infrastructure. But high demand for such applications has driven big could providers to develop NLP-related services, which reduce the workload and infrastructure costs greatly. The average cost of cloud services has been going down for years, and this trend is expected to continue.
I will introduce product in this blog are part of Google Cloud Services and are called “Google Natural Language API”.
Google Natural Language API
The Google Natural Language API is an easy to use interface to a set of powerful NLP models which have been pre-trained by Google to perform various tasks. As these models have been trained on enormously large document corpuses, their performance is usually quite good as long as they are used on datasets that do not make use of a very idiosyncratic language.
The biggest advantage of using these pre-trained models via the API is, that no training dataset is needed. The API allows the user to immediately start making predictions, which can be very valuable in situations where little labeled data is available.
The Natural Language API comprises five different services:
- Syntax Analysis
- Sentiment Analysis
- Entity Analysis
- Entity Sentiment Analysis
- Text Classification
Syntax Analysis
For a given text, Google’s syntax analysis will return a breakdown of all words with a rich set of linguistic information for each token. The information can be divided into two parts:
Part of speech: This part contains information about the morphology of each token. For each word, a fine-grained analysis is returned containing its type (noun, verb, etc.), gender, grammatical case, tense, grammatical mood, grammatical voice, and much more.
The commonly used Python libraries nltk and spaCy contain similar functionalities. The quality of the analysis is consistently high across all three options, but the Google Natural Language API is easier to use. The above analysis can be obtained with very few lines of code. However, while spaCy and nltk are open-source and therefore free, the usage of the Google Natural Language API costs money after a certain number of free requests. I prefer to use spacy and nltk on the projects that have a lot of data and need custom projects requirements.
Sentiment Analysis
The syntax analysis service is mostly used early in one’s pipeline to create features which are later fed into machine learning models. On the contrary, the sentiment analysis service can be used right out of the box.
Google’s sentiment analysis will provide the prevailing emotional opinion within a provided text. The API returns two values: The “score” describes the emotional leaning of the text from -1 (negative) to +1 (positive), with 0 being neutral.
Entity Analysis
Entity Analysis is the process of detecting known entities like public figures or landmarks from a given text. Entity detection is very helpful for all kinds of classification and topic modeling tasks.
The Google Natural Language API provides some basic information about each detected entity and even provides a link to the respective Wikipedia article if it exists. Also, a salience score is calculated. This score for an entity provides information about the importance or centrality of that entity to the entire document text. Scores closer to 0 are less salient, while scores closer to 1.0 are highly salient.
When we send a request to the API with this example sentence: “Robert DeNiro spoke to Martin Scorsese in Hollywood on Christmas Eve in December 2011.” We receive the following result:
As you can see, all entities are identified and classified correctly, except that 2011 appears twice. Additionally to the field in the example output, the entity analysis API will also detect organizations, works of art, consumer goods, phone numbers, addresses, and prices.
Entity Sentiment Analysis
If there are models for entity detection and sentiment analysis, it’s only natural to go a step further and combine them to detect the prevailing emotions towards the different entities in a text.
While the Sentiment Analysis API finds all displays of emotion in the document and aggregates them, the Entity Sentiment Analysis tries to find the dependencies between different parts of the document and the identified entities and then attributes the emotions in these text segments to the respective entities.
For example the opinionated text: “The author is a horrible writer. The reader is very intelligent on the other hand.” leads to the results:
Text Classification
The Google Natural language API comes with a plug-and-play text classification model.
The model is trained to classify the input documents into a large set of categories. The categories are structured hierarchical, e.g. the Category “Hobbies & Leisure” has several sub-categories, one of which would be “Hobbies & Leisure/Outdoors” which itself has sub-categories like “Hobbies & Leisure/Outdoors/Fishing.”
This is an example text from a Nikon camera ad:
“The D3500’s large 24.2 MP DX-format sensor captures richly detailed photos and Full HD movies — even when you shoot in low light. Combined with the rendering power of your NIKKOR lens, you can start creating artistic portraits with smooth background blur. With ease.”
All three of these categories make sense, even though we would intuitively rank the third entry higher than the second one. However, one must consider that this input segment is only a short part of the full camera ad document and the classification model’s performance improves with text length.
After trying it out with a lot of documents, I found the results of the classification model meaningful in most cases. Still, as all other models from the Google Natural Language API, the classifier comes as a black-box solution which cannot be modified or even fine-tuned by the API user. Especially in the case of text classification, the vast majority of companies will have their own text-categories that differ from the categories of the Google model and therefore, the Natural Language API text classification service might not be applicable for the majority of the users.
Another limitation of the classification model is that it only works for English language texts.
Natural language processing (NLP), which is the combination of machine learning and linguistics, has become one of the most heavily researched subjects in the field of artificial intelligence. In the last few years, many new milestones have been reached, the newest being OpenAI’s GPT-2 model, which is able to produce realistic and coherent articles about any topic from a short input.
This interest is driven by the many commercial applications that have been brought to market in recent years. We speak to our home assistants who use NLP to transcribe the audio data and to understand our questions and commands. More and more companies shift a big part of the customer communications effort to automated chatbots. Online marketplaces use it to identify fake reviews, media companies rely on NLP to write news articles, recruitment companies match CVs to positions, social media giants automatically filter hateful content, and legal firms use NLP to analyze contracts.
Training and deploying machine learning models for tasks like these has been a complex process in the past, which required a team of experts and an expensive infrastructure. But high demand for such applications has driven big could providers to develop NLP-related services, which reduce the workload and infrastructure costs greatly. The average cost of cloud services has been going down for years, and this trend is expected to continue.
The products I will introduce in this article are part of Google Cloud Services and are called “Google Natural Language API” and “Google AutoML Natural Language.”
Google Natural Language API
The Google Natural Language API is an easy to use interface to a set of powerful NLP models which have been pre-trained by Google to perform various tasks. As these models have been trained on enormously large document corpuses, their performance is usually quite good as long as they are used on datasets that do not make use of a very idiosyncratic language.
The biggest advantage of using these pre-trained models via the API is, that no training dataset is needed. The API allows the user to immediately start making predictions, which can be very valuable in situations where little labeled data is available.
The Natural Language API comprises five different services:
- Syntax Analysis
- Sentiment Analysis
- Entity Analysis
- Entity Sentiment Analysis
- Text Classification
Syntax Analysis
For a given text, Google’s syntax analysis will return a breakdown of all words with a rich set of linguistic information for each token. The information can be divided into two parts:
Part of speech: This part contains information about the morphology of each token. For each word, a fine-grained analysis is returned containing its type (noun, verb, etc.), gender, grammatical case, tense, grammatical mood, grammatical voice, and much more.
The commonly used Python libraries nltk and spaCy contain similar functionalities. The quality of the analysis is consistently high across all three options, but the Google Natural Language API is easier to use. The above analysis can be obtained with very few lines of code (see example further down). However, while spaCy and nltk are open-source and therefore free, the usage of the Google Natural Language API costs money after a certain number of free requests (see cost section).
Apart from English, the syntactic analysis supports ten additional languages: Chinese (Simplified), Chinese (Traditional), French, German, Italian, Japanese, Korean, Portuguese, Russian, and Spanish.
Sentiment Analysis
The syntax analysis service is mostly used early in one’s pipeline to create features which are later fed into machine learning models. On the contrary, the sentiment analysis service can be used right out of the box.
Google’s sentiment analysis will provide the prevailing emotional opinion within a provided text. The API returns two values: The “score” describes the emotional leaning of the text from -1 (negative) to +1 (positive), with 0 being neutral.
The “magnitude” measures the strength of the emotion.
Google’s sentiment analysis model is trained on a very large dataset. Unfortunately, there is no information about its detailed structure available. I was curious about its real-world performance so I tested it on a part of the Large Movie Review Dataset, which was created by scientists from Stanford University in 2011.
I randomly selected 500 positive and 500 negative movie reviews from the test set and compared the predicted sentiment to the actual review label.
Entity Analysis
Entity Analysis is the process of detecting known entities like public figures or landmarks from a given text. Entity detection is very helpful for all kinds of classification and topic modeling tasks.
The Google Natural Language API provides some basic information about each detected entity and even provides a link to the respective Wikipedia article if it exists. Also, a salience score is calculated. This score for an entity provides information about the importance or centrality of that entity to the entire document text. Scores closer to 0 are less salient, while scores closer to 1.0 are highly salient.
When we send a request to the API with this example sentence: “Robert DeNiro spoke to Martin Scorsese in Hollywood on Christmas Eve in December 2011.” We receive the following result:
Originally published at Kageera.
