NLP to LLM from Basic Principles: NLPlanet NLP and ML pt.3

Photo by Ales Nesetril on Unsplash

Preface:

I am undertaking a Natural Language Processing and Machine Learning course that uses NLTK and HuggingFace. I have taken notes on their lessons and would like to share them. Although I tried to be in-depth and transformative with my notebook, I strongly encourage following their course as it is detailed and well-formatted.

Credit:

Course Link:

Welcome to the Course — Practical NLP with Python (nlplanet.org)

(Section link provided further down)

Author:

Fabio Chiusano — Medium

Note Information:

This is a Jupyter Notebook uploaded to my Github as provided in the link further down.

Notes include definitions, examples, and explanations.

Uses HuggingFace Hub, NLTK, pandas, Plotly, collections, and RegEx.

For more in-depth information on NLTK, consider this NLTK course: NLP to LLM from Basic Principles: NLTK | by Brandon Gomes | Medium.

Title:

Section 1.9

Representing Texts as Vectors: TF-IDF

Section Link:

1.9 Representing Texts as Vectors: TF-IDF — Practical NLP with Python (nlplanet.org)

Information Covered (All Defined in Notes):

• Zipf’s Law
• Use Case: Medium Articles
• HuggingFace Datasets
• DataFrames
• Plotly Visualizations
• Use Case: Brown Corpus
• Use Case: Stop Words
• TF-IDF

Note Link:

NLPlanet Lesson 1.9 Notes: Representing Texts as Vectors TF-IDF