18 PyTorch Tips You Should Know: How and Why They Work — Why should you read this post? The training/inference processes of deep learning models are involved lots of steps. The faster each experiment iteration is, the more we can optimize the whole model prediction performance given limited time and resources. I collected and organized several PyTorch tricks and tips to maximize the efficiency of memory usage and…

Are the deeper networks always good in practice? What can be the potential issues of deep neural networks? The biggest issue of deep networks is the vanishing gradient problem. This is because, through the backpropagation, we need to calculate the gradient for each hidden layer by the chain rule starting from the output layer. If there are n layers, the gradient of the first hidden layer may be…

Work Smarter, Not Harder by AI-Powered Productivity Tools for Writing, Drawing, Creating Content, and more — Motivation ChatGPT, the widely recognized and highly successful AI-based tool, has only been available for less than two months but it has already made a significant impact. It has fundamentally changed people’s perceptions and expectations of AI and also signified that AI will transform the way people produce content. This is…

What is it? How was it trained? What does it mean to us? — You may have been overwhelmed by lots of text/conversation screenshots with a little green icon at the top left corner on all the social networks and the internet. Those were all generated by the latest language model ChatGPT from OpenAI. …

Let’s unleash the power of the internal GPU of your Macbook for deep learning in Tensorflow/Keras! — Why this article? Many Macbook laptops come with AMD GPUs. Unfortunately, AMD GPUs have been notoriously nearly incompatible with most major deep learning frameworks because Nvidia and CUDA have been developed a gigantic deep learning ecosystem. Luckily, Apple recently has released Tensorflow Metal to enable AMD GPU for Tensorflow. Without a desktop with…

Why is a deep neural network better than a shallow neural network (given a similar amount of parameters and training data)? Why depth? Universal approximation theorem [1] says, “Neural network with a hidden layer can represent any functions” (with enough and proper parameters). However, we don’t have enough data for our problems most of the time. If we do, we may solve those problems with a look-up table. …

Data scientist was named as the sexist job in the 21st century and the need is clearly growing. Are you looking for a data science job? Do you know what programming languages and skills are needed on the job market in 2020? Do you know your current skills may fit…

A journey to applying GTX 1080 Ti — I have been wanting to build a deep learning machine for more than one year. During one of my past image recognition projects, my Macbook Pro (of 2.5 GHz Intel Core i5) took more than 8 hours to train one simple convolutional neural network (CNN). Although I spent only one…