Earlier this year I received an e-mail from Andrew Siemion, Director of the Berkeley SETI research centre. He requested my assistance in improving their software deployment strategy on HPC infrastructure.

Image for post
Image for post
searching for aliens at home

How ridiculously awesome is that! Back in the time when I was still young and innocent and I just started to explore the possibilities of computers I have spent significant amounts of time staring at the hallucinating and psychedelic colours of the seti@home software. This software was revolutionary and, I guess naively, the inspiration for many distributed data sharing technologies popular now. But unlike being a platform for downloading audiovisual…

We have used CWL, KERN, Docker, Singularity, and Slurm to build flexible and dynamic pipelines used for analysing radio astronomy data.

Wouldn’t it just be perfect to create a data processing pipeline on your laptop and then deploy it in a cloud environment or supercomputer, without any modifications? I think we have found a combination of technologies that help us realise this geeky dream. For the last couple of months, Michael Crusoe and I have been working closely together with ASTRON and SURFsara on a series of prototype data reduction pipelines for the LOFAR science demonstration as part of the…

Image for post
Image for post
The MeerKAT array in South Africa

It happened already a couple of weeks ago, but the SKA South Africa team has made a new KERN release. KERN is a bi-annually released set of radio astronomy packages, targeted at the Ubuntu LTS platform. These packages are used to analyse and process data coming out of radio telescopes like MeerKAT and LOFAR. We bundle a fixed set of these tools together so it becomes easier to set up a consistent and tidy data reduction environment. Next to the packages we supply tooling for easy creation of containers for for example Docker and Singularity.

Gijs Molenaar

Software composer, music brewer, beer programmer.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store