Tokenize String using Natural Language Processing in SwiftUI
Starting iOS 12, Apple introduced many APIs to bring Natural Language Processing native to Apple’s ecosystem.
Today, we will explore NLTokenizer which helps separate text into desired units leveraging power of Natural Language Processing.
Tokenizing a string
Tokenizing a string simply means that we separate a string into semantic units and analyze it for various use cases. We may want to divide string into units so we can understand the Names, addresses or Locations mentioned in the given text or we may want to understand the overall sentiment of the written text for positive or negative statement.
If the text is written in english language, we may opt to split the string using a separator but this approach may not work for the languages such as Chinese, Japanese or Korean to name a few where spaces are not used to separate their words so for these kind of use cases we can leverage power of Tokenization from Natural Language Processing.
NLTokenizer
NLTokenizer creates individual units from natural language text. This class is defined inside NaturalLanguage framework and we start by creating an object for the class.
