Using pyLDAvis interactive visualization in Python

I’ve used LDA Topic modeling in an NLP project of mine and discovered a great visualization library for topic modeling. This Python Library is called pyLDAvis. The original documentation can be found here.

To install the library:

pip install pyldavis

And a few lines of code to have an interactive visualization:

import pyLDAvis
import pyLDAvis.gensim
vis = pyLDAvis.gensim.prepare(topic_model=pickled_lda,
corpus=bow2doc_corpus,
dictionary=dictionary)
pyLDAvis.enable_notebook()
pyLDAvis.display(vis)

You can check-out a project of mine called dreamjobber, in which I used LDA Topic Modeling and utilized pyLDAvis for better interpretation of topics.


Window Functions

Hey Readers! This blog will cover what I’ve learned with Window Functions… Getting into a little more advanced SQL! 🤓

I will be using the Chinook database as example data.

What are Window Functions?

The easiest way to understand window functions, for me at least, was to compare it to Aggregate Functions (SUM, COUNT, AVG, MIN, MAX). “Aggregate functions calculate a set of rows and return a single output row.” For example:

SELECT *,
SUM(Quantity) AS total_quantity
FROM InvoiceLine;

As you can see the aggregate function SUM caused the query to output a single row, which is the very first row from…


Subqueries

More SQL wooohooo! This blog I will go over subqueries.

First off what in the world is a subquery?

You guessed it! It is basically a query inside a query. In technical terms: “It is a regular query nested inside another query to form a complex query.” Now, why would we want to make things complicated by adding queries inside of a query? Well turns out it’s quite useful when trying to accomplish more complex data analysis tasks.

Basic Syntax

  • Using the Chinook Database as an example. ERD can be seen here.
  • The following query returns the managers/supervisors using a subquery and the IN operator.
SELECT EmployeeId, FirstName, LastName…

SQL JOINs

This is part 3 as I continue learning SQL and share some concepts I’ve learned. Check out my previous blogs as well!

I’ll be touching on Relational Databases and JOINs for this blog.

Relational Database

Chinook Database Entity Relationship Diagram

The image above is an example of a Relational Database.

  • Contains multiple tables to keep data organized with unique instances
  • Each table contains a Primary Key which is a unique identifier for the table
  • Primary Keys also serve as links between tables
  • A Foreign Key in a table is the Primary Key of another table, serving as the link in which we can conduct JOINS
  • Data is structured this way to eliminate duplicate data and ease…

SQL basics

Continuation of SQL basics… If you haven’t seen my initial blog on SQL basics, check it out here.

SQL Aggregate Functions are a way of aggregating data that is used often. It will perform a calculation that will return a single value. It is often used in the SELECT statement.

COUNT |  Returns the number of rows 
SUM | Returns the sum of all the values in a numeric column
MIN | Returns the lowest value in a column
MAX | Returns the highest value in a column
AVG | Returns the average value of a numeric column
Example Syntax Use:SELECT COUNT(ColumnName), SUM(ColumnName), MIN(ColumnName), MAX(ColumnName), AVG(ColumnName)
FROM TableName;
  • Note on NULLS: COUNT( ) and AVG…

Context:

In my pursuit of mastering SQL, I would like to share concepts I’ve learned through some blogs. Thanks for checking this out and I hope this helps others learning SQL as well! Happy SQL!

This blog will cover the basic READ operations for a SQL server. Before I get into this I would like to quickly go over the concept of CRUD.

CRUD is an acronym for the basic database operations:

CREATE operations: Performs the INSERT statement to create a new record.

READ operations: Reads the table records based on the primary keynoted within the input parameter.

UPDATE operations…


A high-level look into the article of Human Decisions and Machine Predictions by Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, Sendhil Mullainathan

(Public Domain Image)

In the United States, over 10 million people are arrested each year. In the year 2006, over 14 million people were arrested. (For more in-depth on these statistics, check it out here.) After an arrest is made, a very important decision must be made by the Judge.

Will the defendant await trial at home or in jail?

This very important decision provides a good test case to implement Machine Learning to see if it will improve human decision making.

During a bail hearing a judge has three…


A simple and brief tutorial on hypothesis testing using Python

Image from: http://www.advanceinnovationgroup.com/blog/median-based-hypothesis-testing

In this blog, I will give a brief tutorial of Hypothesis Testing using Statistical methods in Python. Hypothesis Testing is part of the Scientific Method we’re all familiar with, something we probably learned in our early educational years. However, in statistics, many experiments are done on a sample of a population.

“Determining what a sample set of observations tells us about a proposed explanation, in general, requires us to make an inference, or as we statisticians call it, to Reason With Uncertainty. Reasoning with uncertainty is the core of statistical…


“Each evening we would scan our charts for the pattern we wanted to trade in the current market environment and we would know what trades we needed to place the following morning.” — Llewelyn James

In this blog post I would like to talk about the Candlestick chart. So what is a Candlestick chart? It is a financial chart used to display the price of a stock and price movements within a trading day. Some history about the Candlestick chart: It was developed in Japan by Munehisa Homma, a businessman who traded rice contracts in the 1800s. “Stories claim that…


Why I’m learning data science is a question to be answered by my story leading up to my decision to enroll at the Flatiron Data Science bootcamp. During my childhood I enjoyed sports and just being outside hanging out with other kids, just like most kids back in the early 90’s when that was our form of fun. Now we have video games of all kinds. Every kid has a dream, a dream that usually shoots for the stars. Mine was to be a professional golfer just like Tiger Woods; like I said shoot for the stars. Golf was the…

Mando Iwanaga

Data Fanatic

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store