It’s been ages since I caught a glimpse of the sunrise but thanks to my early morning flight to PDX, I was able to appreciate it again. The excitement of traveling alone to a new place brought back good old memories.
The flight time was pretty much a power nap time, the row of empty seats begged me to lay down and the bright sun forced me to shut the windows. A nap to remember!
I pictured Portland to be similar to Ithaca, NY. Drab, boring, yet green and gorgeous. Well, part of Portland near the airport met my expectations but as soon as my uber driver reached the highway where traffic seemed to be obvious at that point of the day, I was completely shocked. It was a good mix of Chicago and NYC. The old bridges, roads, crowdy streets and pavements and hot trucks and cars. Bustling city!
Quickly, I checked into my hotel, dropped off my baggage and got a ride to the conference. Entered through the doors, checked in at registration, made up my mind on what sessions to attend and then I caught a glimpse of…..stickers!
The first session I attended was on Data visualization. It was presented by Silicon Valley Data Science Group who had researched immensely on how and why color choices matter and how we could leverage emotions and properties of color to better visualize data. This was an area I was completely new to and it was very inspiring to meet fellow data scientists who cared so much about this. However, this was not my forte so I switched to something more hands-on after break.
After break, it was time for some hands-on IoT. Using almost 9 different open source technologies, this session let me access data from devices (MAC addresses) connected to a particular wifi, allow this data to be batch processed and present it visually using interactive charts. Pivotal lab engineers walked us all through a tutorial for all of these open sourced technologies but, what was interesting was that it mostly did not work. Regardless, this idea of being able to grab data from devices connected on a common platform is something new that Spring XD brought into this world.
The next session I attended after lunch was called, “Becoming friends with Cassandra.” Two Datastax engineers managed to make this session more fun than I was anticipating it to be. They went over some basics about Cassandra, that I already knew but, one important point they made that helped me understand Cassandra better was that, “In Cassandra, the queries drive the data model where as in RDBMS world it is vice versa.” They also did a great job of showcasing how easily it could be integrated with Apache Spark. At the end, Cassandra really did become a friend. I was never a fan of NOSQL databases, but wow now I have to say bye to Relational World!
The day ended with a reception sponsored by PayPal. I got to meet developers from different companies and learn about how their technologies might help us. SauceLabs, Elastic, RedHat, GitHub, NewRelic and Erlang were some of the companies I got to interact with. Leaving the hall, I noticed an empty spot on a big black board where people were sketching the logos of their companies.
I happily left our mark as well!
Le` Cloudy day in Portland. Today was interesting because I started noticing a theme for all the technologies being presented in this conference. Most open sourced software out there can be divided into the following three main categories:
Infrastructure & scaling
Data processing and analytics
Tools and technologies help us solve problems but there are just so many of them out there. And visibility is exactly the main problem that most of these open source tools have been facing.
The opening session went over why open source was important. Facebook presented its open source case study on how they manage to contribute regularly. Pretty interesting!
I went to a session on visualizing program execution after this. Essentially, we need better debuggers but also a strategy to debug efficiently. The speaker introduced a new concept called “omniscient debugging.” Can you imagine your program being able to log everything so that when you just take a look at it, all you have to do is hypothesize what the bug is and solve it? No need to reproduce bugs as you have all the output right there! Cool idea but this field of omniscient debugging is still being researched.
Next, I attended a session on how Twitter used Scala to scale. This was not as impressive as I thought it would be. The speaker mentioned that in the 10 months he worked there, developers at Twitter have been switching to a newer version of Scala and have been trying to make Java and Scala compatible. They have a lot of internal libraries for Scala which they were not intending to make open source.
Apparently, Twitter maintains one of the largest codebases for Scala on the planet. They are now trying to document and spread awareness on Scala by offering courses on Twitter University. Um cool?
Next up was Docker! Well Docker is interesting because it became widely adopted within a few months. The speaker gave us a brief tutorial on what micro services were, how using different stacks for different services makes refactoring individual services easy and how very less coordination is needed when you deploy (deploy more often, less risk, more agility and speed). Docker uses a combination of deploy scripts and config management for building and deploying. The speaker also introduced us to how Docker worked at scale with Docker compose + swarm.
The next two sessions I attended were mostly about Uber. The first talk was mind blowing! It was given by Matt Ranney, the chief systems architect and it was about scaling Uber by breaking everything yet facing this constant challenge to maintain quality. The important takeaway from this was that sometimes, “you have to admit that you are wrong.” And by admitting so you can design systems that are capable of breaking randomly yet know how to respond to such failures. Available systems are better than consistent systems!
He also went over some Uber stats and their current system architecture (a very simple view of the complex giant).
The day ended by accumulation of more swag. I think this hoarding habit will kill me one day. I got so many books, t-shirts and stickers that I was unable to carry my bag around the conference.
And it only gets better! Sunny, bright and hot, I ❤ Portland. I was off to the conference this morning after I finished the 10 puzzles they gave us out. I got an OSCON hoodie for completing them correctly (the swag hoarding continues….). Today, I have decided to switch it up a bit and attended sessions that were slightly off topic. Doing this made me appreciate open source even more!
The first session I attended was called “Accelerating the core of the cloud,” presented by an Intel engineer. The name is such a misrepresentation of what the session actually entailed. In two words it was about compilers and performance. They went over limitations of existing compilers and how Intel is actively working on a new strategy called: Core Performance optimization primarily targeting emerging languages like python, node.js, php, hhvm, and GO.
Don’t speed up the program, speed up the compiler or runtime at the core.
This idea is an interesting cross-collaboration between a corporation and open source world. Anyway, next up was collecting analytics from Graphs. The core of this session focused on utilizing existing technologies to do some interesting analytics on today’s social graphs. To put it in a sentence, this is super duper complicated to do at a large scale. The speaker mentioned technologies like Graphx and Giraph sitting on top of a Hadoop ecosystem. I think this is the next area that data analytics will probably move towards. Very futuristic but still a lot of work to be done! So next time I complain about why Facebook is suggesting me friends who I do not know, I will be thinking about how difficult it is do scalable graph analytics.
The next session I attended is definitely the highlight of the day. It was called “How to Game Design.” Presented by three Australian mates, the session was not only super fun, covered mechanics of game design but also enlightened me on how open source software helped make games better.
Cassandra 3.0 was announced at OSCON today. The talk gave a brief history of why there was a paradigm shift from using relational DBs towards NoSQL and how Cassandra gained momentum in this process. The CTO of Datastax, Jonathan Ellis also presented a few notable features of the new version. To be released in September, this version will be offering materialized views and role based authorization. He also mentioned that they were shifting the way they developed code. Inspired by Intel’s tick tock strategy, Datastax is now going to release few features, follow them up with bug fixes and then release more features, again follow them up with bug fixes and so on until the version is stable enough.
The last session of the day was about the learnings of an Amazon AWS Engineer who contributed towards two different query languages. It focused on a systematic way to write specifications when designing the code. Much of the talk included tid bits of AWS SDK which just brushed through my head but overall, the takeaway from this was to write specifications that could adapt to use cases of tomorrow. And that was it, end of the day.
Not so soon…..I ventured out and explored Portland today. Pictures speak more than words so here is a snapshot of Portland, OR.