Big Data, Big Challenges: My Experience at My First Hackathon
Five students had a great time at the Big Data Hackathon and learned much about thinking on our feet and working until the end to come out with a product.
I’ve provided the links below to the actual project, the description of the project, and the repository for the project (all in that order). If you are interested in hearing more or have any words of wisdom you would like to give to improve the project or to just get your two cents in, don’t hesitate to comment or contact me.
— start blog post —
I ’ve discovered the joys of incorporating drop caps in a Medium article, so I apologize if I seem to overuse it. It looks so wonderful to me for whatever odd reason.
This past weekend (Oct. 8–9, 2016), I had the opportunity to attend the Midwest Big Data Hackathon over at The University of Iowa in Iowa City, IA. Thanks to the graciousness of the Computer Science Department (thanks, SamR!), four other Grinnell College students (Trang Nguyen, Krit Petrachaianan, Hanh Pham, and Linh Pham) and I were able to drive down there. We originally had two other students who were planning to tag along with us, but unfortunately, they had other commitments that they had to make and meet.
Let me preface this post with the fact that all five students, including myself, have never attended a hackathon before, nor have we had more than a year’s worth of computer science classes. I will note that all five students have had some experience with statistical programming. Could you imagine how daunting it was for five students to go out to a competitive event where top graduate students, employers, and other intelligent undergraduate students are also in the running to create the best big data project? I can; I experienced it all.
As a General Sciences-Biology major with a concentration (minor) in Statistics, I probably have had the least amount of exposure and experience to this realm compared to the rest of my peers. I have a great background in research, marketing, and networking — the kind of networking where you meet people and develop connections, not the kind where circuits and wires are involved. I was paired up with Krit, who has had the most experience and exposure to programming in general. Both he and I were scared of him leaving me in the dust to complete the project on his own while I would be unable to provide anything insightful or impactful.
However, our fears slowly disappeared as the hackathon proceeded onward.
The two groups of Krit and I and of Trang, Hanh, and Linh both decided to create a network visualization using the R Programming Statistical Language, a language we all felt near comfortable using. Many of the products that are made at hackathons are typically in the form of some mobile app or website, so I felt that creating something in R would add a unique perspective to the list of submissions.
As a team, Krit and I worked together to efficiently create the project with our strengths in mind. I had much experience in wrangling with data and cleaning them up. Krit had an extensive background of the
shiny package in R, so I was tasked with organizing the data while Krit was assigned to creating the framework.
The thing I did not want most to happen kept replaying in my head: “Don’t weigh down Krit with your inexperience.” I mentioned previously how Krit was the one with the most experience out of all of us in terms of R and statistical programming in general, so I kept a motivated mindset while working so that I would not pose as a burden. This allowed me to stay on task and to complete my assigned work for the project (with a little help from Krit as well). Krit and I spent a majority of the night afterwards eating slices of pizza, attending a spontaneous karaoke event, and pair-programming to ensure that the code was working well.
I picked up pair-programming and learned about the concept when I took my first computer science class at Grinnell College last spring, CSC-151: Functional Problem Solving in Scheme. Pair-programming is the process of developing code with a peer, which creates a communal learning environment and improves coding since there are two people to catch mistakes in the code. As a novice, I felt that this learning style was godsend since it promotes a healthy learning environment (as long as the pairs of programmers are civil with each other and receptive of criticism).
There are definitely skeptics that believe pair-programming is obsolete. Knowing Krit — he has been a friend of mine since first year — I knew that he belonged in this category since he is quite self-sufficient in terms of coding, and perhaps in terms of real-world survival. We had to pair-program from time to time in order to complete tasks such as finalizing the code for cleaning the datasets and finishing up the structure of the server.R file for the Shiny app.
I thought we would not be able to mesh well with the circumstances given: I am not an expert an R and did not have much insight about how to form code in the
shiny package, while Krit was efficient and was capable of completing work very quickly. However, this what we thought wouldn’t work was actually the best environment for pair-programming to excel and prosper. Because I had to take things slow, I had to ask Krit about how the syntax, input, and output was structured in the
shiny package. By doing this, Krit and I were able to both catch mistakes that were in the underlying code and figure why some parts of our app was not working as intended. We did this for quite a bit throughout the night.
At a hackathon, I should mention that the expectation is set that you get only a few hours of sleep, if not any. Most of our group members slept for about an average of 3.2 hours in the van. Obtaining sleep, despite how little of it we did get, was very crucial in being able to function the next day because the last 6–8 hours of the hackathon is dedicated to finalizing your project — no matter how unfinished it may be — and developing a way to present your product in a way that is both easy-to-understand and that fits with the theme — in our case, Big Data.
Krit and I were able to create a functioning prototype with an hour left to spare. I developed a way to market the Shiny App we created to be used for higher education. The network visualization allows you to observe how individuals are connected in a student dormitory and how they interact with one another. This type of information would be crucial for Residence Life at universities and colleges in developing community and improving student programming.
Judges came around from table to table to listen to presentations, similar to how a fifth grader presents their volcano at a science fair. Before they came to our area, I told Krit to come in with no expectations for the presentation and the end result because we worked very hard to come out with a usable product, and we should be proud for that sole reason and not be affected by any other external forces. We should be happy with what we’ve done and go in with an open mindset when we present.
After we presented, we waited patiently for the award ceremony to start. As the event committee set up, we noticed prizes of sorts being brought up to the stage such as two flat-screen TVs, drones, mini projectors, and others. The five of us had no idea there was supposed to be prizes up for grabs, so we were a bit thrilled to think about the possibility of bringing home one of the prizes.
Finally, the closing ceremony began and they started off with a big thank you to The University of Iowa and their sponsors including but not limited to Pariveda Solutions, Leepfrog Technologies, and the Iowa Flood Center. They followed up with the presentation of awards. They announced third place, which was a student who created a web application that gave information about different compositions of music using a classical music dataset.
As they announced third place, they mistakenly called out Krit’s name and my name, which did not make sense since that wasn’t even our project. However, both of us had a strange feeling about the mistake they had made; why did they call our names out when the third-place project was nowhere near our vicinity? The answer to this question would inevitably surprise us since they called our names to receive the second-place award for the competition. We received two digital cameras (Sony DSC-H300) for our recognition (see below).
Krit and I were awestruck, flabbergasted, shocked; any of the above words will suffice to describe how we were feeling. We were competing with students from some great institutions and backgrounds that included both software developers and graduate students, and we came out as one of the top contenders using R …that in itself was a very pleasant surprise for us. As other names were being called for honorable mentions and company-specific prizes, we were encapsulated by our own disbelief that we had won.
We both thought that we should ask judges why we stood out as a quality project compared to others, so after the ceremony we sought out the sponsors and other committee members. The organizers took no part in the judging, but they mentioned that judges scored based on a set of criteria.
We were able to meet one of the judges, Caglar Koylu of The University of Iowa, who told us that this project was one of his favorites and was well communicated. He also does work with spatial analysis and visualizations, so I felt that it was no surprise that he took a liking to our presentation. Additionally, he informed us that he works at the Iowa School of Public Health where I’ll be attending after I graduate from Grinnell College. This interaction not only gave us something to be proud about, but also gave me a future connection to possibly move forward with and develop this project further.
Aside from these remarks, we were unable to find others and felt satisfied with what we had heard. From there, we took off, enjoyed some Vietnamese food, and arrived back home.
If you’ve read this far, you should know that this marks the end of my blog post. If I was asked what was the most important lesson I learned from this experience, then I say this: “Don’t be afraid to take on a challenge head-on, especially if the worst thing that can happen is that you learn something new.” Although I was a bit hesitant to attend a hackathon without any experience in software development, I ended up learning so much in one weekend from the usefulness of code readability to the importance of getting rest, no matter how short of a rest period.