Random projections did it again!

R: “I like it because it is linked with having many classifiers ensembled!” A: “I will pretend I thought about that. Behind many artists maybe there was simply a visionary art critic.” — Just a normal day at work.
Figure 1: Time and energy spent on computing a matrix multiplication (n,D) × (D,D). The batch size n is 3000 (solid line) or 1000 (dotted line). The OPU is compared to an NVIDIA P100 GPU. Plot taken from [6].

Double trouble in double descent

Figure 2: Synthetic data generated from a standard Gaussian. The labels are assigned using a ground truth corrupted by Gaussian noise with increasing variance from left to right.
Figure 3: Decomposition of the test error.
Figure 4: The effect of ensembling on the double descent curve. The peak decreases with K until it disappears in the limit of infinite K. Plot taken from [4].

Fulminate the double descent curve

Figure 5: the random feature regression implemented with an OPU.
Figure 6: We can project our data on the number of random projections * max(K). We can obtain the different values of ensembling by using slices of this matrix.
Figure 7: The ensembling effect on the double descent curve on a sub-sample of MNIST.

Conclusion

About us

The author

Acknowledgement

References

--

--

--

We are a technology company developing Optical Computing for Machine Learning. Our tech harvests Computation from Nature, We are at lighton.ai

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

DATA PREPROCESSING

Light FM Recommendation System Explained

Chapter 4: Decision Trees Algorithms

Building a real time Face Recognition system using pre-trained FaceNet model

Machine Learning Algorithms: Surprises at Deployment?

Land Cover Classification Using Keras

Exploring Deep Embeddings

TASK 6- Face Recognition

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
LightOn

LightOn

We are a technology company developing Optical Computing for Machine Learning. Our tech harvests Computation from Nature, We are at lighton.ai

More from Medium

Leveraging Commercial Eyes in Space

Should We Be Worried About Stuxnet Redux?

Task Generality, or, Sex with Robots

How to find your rubber duck: Using machine learning to understand a changing sea