Ali Ghodsi’s U Waterloo AI Courses
I came across Ali Gholdsi’s lectures by chance, and some of the YouTube comments were so positive that I had to watch his lectures:
“These lectures are way more informative than the Stanford ones”, said one.
Another said: “One of the patient explanations, almost ‘a conceptual derivation’ of the GAN. Talks few words and slowly so you can process what he is saying and gets foundations clear first. But the depth of his grasp and sharp thought can be seen in many places [for example where he] answers the question ‘if the function needs an inverse then does that mean you cant use Relu in the network?’ brilliantly! Happy to know this resource.”
After watching his lecture on GANs (above), I sought all of his lectures, and attempted to collect them in one place. Some were already listed on the U Waterloo data analytics site, but it was not a complete list.
So here they are:
STAT 946 Topics in Probability and Statistics: Deep Learning (Fall 2015) all videos and slides
- Course Outline, list of papers
- Lec 1.1: Introduction, slides
- Lec 1.2: Perceptron, Feedforward Neural Network, Back propagation
- Lec 2.1: Regularization, slides
- Lec 2.2: Regularization
- Lec 3.1: Word2vec, slides
- Lec 3.2: Word2vec
- Lec 4.1: Sum-Product Networks, slides
- Lec 4.2: Sum-Product Networks
- Lec 5.1: Recurrent neural network, slides
- Lec 5.2: Recurrent neural network
- Lec 6: Convolutional network, slides
- Lec 7: Restricted Boltzmann Machine (RBM), slides
- Theano Tutorial, example.ipyng, lasagne_example.ipynb, lstm.ipynb
- Keras Tutorial, slides
STAT 441/841, CM 763: Statistical Learning Classification (Fall 2015) all videos and slides
- Lec 1: Machine Learning, Introduction
- Lec 2: Formal definition of classification, Linear discriminant analysis (LDA), Quadratic discriminant analysis (QDA)
- Lec 3: QDA, Principal Component Analysis (PCA)
- Lec 4: PCA,Fisher’s Discriminant Analysis (FDA)
- Lec 5: Logistic Regression
- Lec 6: Logistic Regression, Perceptron
- Lec 7: Backpropagation
- Lec 8: Radial Basis Function Networks
- Lec 9: Stein’s unbiased risk estimate (sure)
- Lec 10: Weight decay
- Lec 11: Hard margin svm
- Lec 12: Soft margin svm
- Lec 13: Dual PCA, Supervised PCA
- Lec 14: Supervised PCA, Decision tree
- Lec 15: Decision Tree, KNN
- Lec 16: Boosting
- Lec 17: Bagging, Convolutional Networks (part 1)
- Lec 18: Convolutional neural network (part 2)
- Lec 19: PAC Learning
Deep Learning (2017) all videos and slides
- Sep 7: Introduction (no video), slides
- Sep 12: Perceptron, FFNN, Backpropagation, slides
- Sep 14: Overfitting, Regulatization
- Sep 19: Weight Decay, Introduction to Keras, slides
- Sep 26: Regularization, Dropout, slides
- Sep 28: Batch Normalization, CNN, slides
- Oct 3: CNN, slides
- Oct 5: RNN, slides
- Oct 12 Part 1: Variational Autoencoder
- Oct 12 Part 2: Variational Autoencoder
- Oct 17: Sum Product Network, slides
- Oct 19: Deep Reinforcement Learning, slides
- Oct 24: Generative Adversarial Networks, slides
STAT 441/841: Statistical Learning — Classification (Winter 2017), playlist
- Lec 1: Intro to classifiers, Bayesian classifiers, LDA and QDA, slides
- Lec 2: QDA, PCA
- Lec 3: FDA
- Lec 4: Logistic regression
- Lec 5: Model selection, Neural Networks
- Lec 6: Spectral Clustering, Laplacian Eigenmap, MVU
- Lec 7: Back Propagation, RBN
- Lec 8: Complexity control for RBN
- Lec 9: Regularization, Hard Margin SVM
- Lec 10: SVM, Kernel SVM
- Lec 11: Soft Margin SVM
- Lec 12: Metric Learning
- Lec 13: SPCA, Naive Bayes, K-nearest neighbour
- Lec 14: Convolutional Neural Networks
- Lec 15: Random features, Tree
- Lec 16: Tree, Boosting method
- Lec 17: Boosting method
- Lec 18: Bagging
STAT 442/842: Data Visualization, a course on unsupervised learning (2017)
- Lec 1: Principal Component Analysis
- Lec 2: PCA (Ordinary, Dual, Kernel)
- Lec 3: FDA
- Lec 4: MDS, Isomap, LLE
- Lec 5: LLE, Spectral Clustering
- Lec 6: Spectral Clustering, Laplacian Eigenmap, MVU
- Lec 7: MVU, Action Respecting Embedding, Supervised PCA
- Lec 8: Supervised PCA
- Lec 9: SPCA, Nystrom Approximation, NMF
- Lec 10: NMF via R1D algorithm
- Lec 11: Sum-Product Networks
- Lec 12: Neural Networks, Autoencoders, Word2Vec
- Lec 13: Word2Vec Skip-Gram
- Lec 14: Autoencoders, Clustering, Mixture of Gaussians
- Lec 15: t-SNE
- Lec 16: Variational Autoencoders
Deep learning (Fall 2020), only two available:
New videos will get posted on Ali Ghodsi’s YouTube channel. One nice feature of his videos is that each is more or less self contained. That is — they depend on earlier material, but it is relatively easy to navigate and find these dependencies.
Among his publications, I am collecting below his recent Tutorials and Surveys. They accompany and expand some of the materials presented in his videos.
Tutorials and Surveys
Here is Ali Ghodsi on Google Scholar. Not to be confused with Ali Ghodsi from U. Berkeley, CEO and founder of Databricks.
- Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey, B. Ghojogh, A. Ghodsi, F. Karray, M. Crowley (2021)
- Restricted Boltzmann Machine and Deep Belief Network: Tutorial and Survey, B. Ghojogh, A. Ghodsi, F. Karray, M. Crowley (2021)
- Unified Framework for Spectral Dimensionality Reduction, Maximum Variance Unfolding, and Kernel Learning By Semidefinite Programming: Tutorial and Survey, B. Ghojogh, A. Ghodsi, F. Karray, M. Crowley (2021)
- Laplacian-Based Dimensionality Reduction Including Spectral Clustering, Laplacian Eigenmap, Locality Preserving Projection, Graph Embedding, and Diffusion Map: Tutorial and Survey, B. Ghojogh, A. Ghodsi, F. Karray, M. Crowley (2021)
- Attention Mechanism, Transformers, BERT, and GPT: Tutorial and Survey, B. Ghojogh, A. Ghodsi (2021)
- Locally linear embedding and its variants: Tutorial and survey, B. Ghojogh, A. Ghodsi, F. Karray, M. Crowley (2020)
