Reducing Dimensionality from Dimensionality Reduction Techniques
Elior Cohen

I have a question:

If I have a 3mio rows dataset with non linear structure

  • I cannot use pca because of it’s limitation to only linear transformation
  • I cannot use tsne because my conputer can’t hold 3mio/3mio distance matrix
  • basically autoencoder would catch non linearity and work with 3mio rows but it will not use the proximities as input before reduction (like tsne)

How would you recommend to approach this ?

Many thanks,