My experience in GSoC 2019

Barbiegaldor
Aug 30, 2019 · 5 min read

In this blog I will discuss my time in GSoC, my experience with CROSS and my contribution to the Popper project.

Popper is a workflow execution engine based on GitHub Actions, but my work wasn’t exactly working on new and exciting features for Popper, my work was Blackswan which is a web interface that will allow users with projects that implement Popper workflows to keep track of their automatic workflow executions on the CI services. In order for a project hosted on GitHub to be tracked, users need to enable them on the Blackswan interface.

View for managing GitHub and Blackswan Projects

Before officially coding

In preparation for making a proposal I started working with Popper to get familiar with the code and how it works, I worked on some PRs and they got merged. (#539, #483, #521)

So, if you haven’t contributed to open source and learned how it works, I would push you to start NOW!

The first month

From May 23 to June 28

Because I had the privilege of starting the project, i had to work back to back with the frontend (Vue, Vuex, Vue material, Javascript) and backend (Django, Python). I did all the work from designing and writing all views to API configuration and database modeling, and this is reflected in the issues for this month. I also made data to use for testing the database and implemented a js library for graphical view of the workflow (the format we picked didn’t work out for our project in the end but oh well)

The most challenging and new of the issues I worked with in this period, was the use of technologies that i had a basic knowledge of like Javascript and Django, and also, integrating with other services like GitHub using OAuth to extract repositories was my favorite, yet, difficult.

One of the views made during this month, you visualize your projects and the last executions of them.

The second month

From July 1st to July 26

For this month i focused mainly in the backend part, i made changes to the models like allowing a project to have more then one user and let the user able and disable. I made changes to the API to allow more features like showing your secret token, showing your GitHub repositories and enabling a GitHub repository in Blackswan, allow unauthenticated users to visualize executions from public repositories, a debug login screen that doesn’t requires an authorization from GitHub to test fixtures and the start of the Blackswan documentation.

I created unit tests for Travis, and also cleaned the repository of auto generated data

For the frontend part I created a profile view and views for GitHub repositories and Blackswan projects.

View of executions from a project.
View of the log and workflow results from the execution

Final month

From July 29 to August 23

This was the hardest month of all, where the most challenging issues come in front of you with a moth in hand ready to destroy your work and you only have stackoverflow as your shield.

I worked in making the API good again. Fixed issues with calls, issues with the OAuth implementation and the data is saved, making a view for bad urls, changes to routing and the cutest change of them of all: changing the theme color for the page 😁

The most challenging and frustating issue ever: Making a deployment pipeline for Google Cloud Platform so we can start working with production, saving information from branches, inserting code to GitHub, etc.

Google Cloud Platform (left), me (right)

Usage

It is possible to run a local test version of Blackswan. This only requires Git, Docker, and Docker Compose:

git clone https://github.com/blkswanio/blackswan.git
cd blackswan
docker-compose up --build

You can run the API unit tests inside it’s respective docker container using

docker-compose up -d backend
docker-compose exec -it backend python manage.py test

To run the Popper pipeline that deploys the current version of Blackswan to Google App Engine, make sure the GCLOUD_AUTH environment variable is set and run these commands inside the project directory:

popper run --wfile ./ci/workflows/gcp-deploy/main

Conclusion

To finish this long post, I want to point out how rewarding these short three months have been, it helped me grow as a developer (A LOT), on things like: how to start a project, the fundamentals of planning step by step the route of the implementations you have to do, how important testing is and how to contribute to open source, I had the pleasure to work with awesome open source tech that made my life easier. I personally feel a lot more prepared and comfortable with the technologies I used (It required a lot of reading and patience).

I also think it made me better at working with others, expressing my problems and at not getting frustrated with little things. I’d like to take the time to thank my mentors for the help and time they provided me, specially Ivo Jimenez, he is a great teacher, great person, respectful and very patient.

I am very pleased with the time I spent in the meetups with the CROSS crew (even though it was short) and I’d also like to thank to you, the reader, for reading till the end of this post.

getpopper

Topics on practical aspects of reproducibility in computational and data science: SciOps (DevOps in Science), OpenScience, OpenData, and more.

Thanks to Ivo Jimenez

Barbiegaldor

Written by

getpopper

getpopper

Topics on practical aspects of reproducibility in computational and data science: SciOps (DevOps in Science), OpenScience, OpenData, and more.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade