Neural Ordinary Differential Equations and Dynamics Models

This post was written by Aidan Abdulali.

In this post, we explore the deep connection between ordinary differential equations and residual networks, leading to a new deep learning component, the Neural ODE. We explain the math that unlocks the training of this component and illustrate some of the results. From a bird’s eye perspective, one of the exciting parts of the Neural ODEs architecture by Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud is the connection to physics. ODEs are often used to describe the time derivatives of a physical situation, referred to as the dynamics. Knowing…

Machine Learning Crash Course: Part 5 — Decision Trees and Ensemble Models

26 Dec 2017 | Shannon Shih and Pravin Ravishanker

Trees are great. They provide food, air, shade, and all the other good stuff we enjoy in life. Decision trees, however, are even cooler. True to their name, decision trees allow us to figure out what to do with all the great data we have in life.

Like it or not, you have been working with decision trees your entire life. When you say, “If it’s raining, I will bring an umbrella,” you’ve just constructed a simple decision tree.

It’s a pretty small tree, and doesn’t account for all situations. Likewise…

Machine Learning Crash Course: Part 4 — The Bias-Variance Dilemma

13 Jul 2017 | Daniel Geng and Shannon Shih

Here’s a riddle:

So what does this have to do with machine learning? Well, it turns out that machine learning algorithms are not that much different from our friend Doge: they often run the risk of over-extrapolating or over-interpolating from the data that they are trained on.

There is a very delicate balancing act when machine learning algorithms try to predict things. On the one hand, we want our algorithm to model the training data very closely, otherwise we’ll miss relevant features and interesting trends. …

Machine Learning Crash Course: Part 3 — Neural Networks

04 Feb 2017 | Daniel Geng and Shannon Shih

Neural networks are perhaps one of the most exciting recent developments in machine learning. Got a problem? Just throw a neural net at it. Want to make a self-driving car? Throw a neural net at it. Want to fly a helicopter? Throw a neural net at it. Curious about the digestive cycles of your sheep? Heck, throw a neural net at it. This extremely powerful algorithm holds much promise (but can also be a bit overhyped). …

Machine Learning Crash Course: Part 2 — SVMs, Perceptrons, and Logistic Regression

24 Dec 2016 | Daniel Geng and Shannon Shih

In this post we’ll talk about one of the most fundamental machine learning algorithms: the perceptron algorithm. This algorithm forms the basis for many modern day ML algorithms, most notably neural networks. In addition, we’ll discuss the perceptron algorithm’s cousin, logistic regression. And then we’ll conclude with an introduction to SVMs, or support vector machines, which are perhaps one of the most flexible algorithms used today.

Supervised and Unsupervised Algorithms

In machine learning, there are two general classes of algorithms. You’ll remember that in our last post we discussed regression and classification. These two methods…

Machine Learning Crash Course: Part 1 — Regression/Classification, Cost Functions, and Gradient Descent

06 Nov 2016 | Daniel Geng and Shannon Shih

Machine learning (ML) has received a lot of attention recently, and not without good reason. It has already revolutionized fields from image recognition to healthcare to transportation. Yet a typical explanation for machine learning sounds like this:

“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.”

Not very clear, is it? This post, the first in a series of ML tutorials, aims to…

10 Jan 2018 | Daniel Geng and Rishi Veerapaneni

Assassination by neural network. Sound crazy? Well, it might happen someday, and not in the way you may think. Of course, neural networks could be trained to pilot drones or operate other weapons of mass destruction, but even an innocuous (and presently available) network trained to drive a car could be turned to act against its owner. This is because neural networks are extremely susceptible to something called adversarial examples.

Adversarial examples are inputs to a neural network that result in an incorrect output from the network. It’s probably best to…

Demo Day: September 2016

One of our main goals here at ML@B is to help students understand how to use machine learning in real-world situations. This semester, we’ve teamed up with Github, Grand Rounds, SAP, and Intuit to work on solving some of their problems through machine learning. In addition, we have members working on their own independent research projects with groups such as the International Computer Science Institute.

Just this Friday, we had our very first demo day — a day where project members got to present what they’d been up to. Here’s a brief summary of what they had to show:

Code…

Hello World!

One of the hottest and most exciting topics floating around these days is machine learning. People have created amazing things through machine learning, such as self-driving cars, mind-controlled prosthetics, and actual, readable dialogues in the style of Shakespeare. But just what is machine learning? Surely it’s something obscure and esoteric. Right? And surely the subject is so entangled in computer science and mathematics that it would make any reasonable student run the other way. Right?

That’s where Machine Learning at Berkeley comes in. ML@B is the first undergraduate machine learning club at UC Berkeley. We were founded with the goal…

Machine Learning @ Berkeley

A student-run organization at UC Berkeley working on ML applications in industry, academic research, and making ML education more accessible to all

Get the Medium app