Torrenting the galaxy

Image source: ESA/ATG medialab

Gaia

,,,,,,,0.35475,,,,,,,,,,115,0,115,0,0,0,,0.6696595881966159,1.4008253634028593,false,1.2046634,0.3231782,,2,25,false,0.33076313,0.15289937,0.3994712,0.3902733,-96.040985,-9.364204,-10.255349,-21.1315,120,645.2563508174401,2.6361056560487532,18.500439344588465,NOT_AVAILABLE,59.27191935938144,3.8225596506032447,299.3146621937641,46.346717555917245

Determining a star’s distance with parallax

or from the star’s perspective
image courtesy of Michael Richmond

In the weeds with a few hundred gigs of CSV files

We should have some ways of connecting programs like garden hose — screw in another segment when it becomes necessary to massage data in another way. This is the way of IO also.

(full gist available here)
node processStars.js GaiaData_000.csv.tar.gz > GaiaData_000.processed.csv

How to throw a few dozen computers at the problem

(full gist here)

Packing 2 million stars for a trip to your GPU

111.236031,111.369408,0.646683
              byte0    byte1    byte2    byte3
-------- -------- -------- --------
111.23603 = 01000010 11011110 01111000 11011001
111.36941 = 01000010 11011110 10111101 00100011
0.646683 = 00111111 00100101 10001101 00000100
              byte0    byte1    byte2    byte...n
-------- -------- -------- --------
111.23603, = 00110001 00110001 00110001 00101110
00110010 00110011 00110110 00110000
00110011 00101100
111.369408, = 00110001 00110001 00110001 00101110
00110011 00110110 00111001 00110100
00110000 00111000 00101100
0.646683 = 00110000 00101110 00110110 00110100
00110110 00110110 00111000 00110011
Damnit, close enough.

Torrenting the galaxy

The Final Result

--

--

Programmer, amateur astronomer, occasional clock maker, co-creator of The Great Gatsby for NES.

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store