Twitter Analytics — Part I

Using The Cloud as an excuse to stalk the Tweeters of Melbourne

While maybe just somewhat as this post’s title suggests, the proliferation of social data across the internet, among friends and data-voyeurs alike, proves quite provocative — and definitely is so from an analyst’s perspective.

From Public Datasets to Social Media, not only is our most innocuous and sensitive of information now so portable, discoverable, and undeletable, but it is so latently meaningful. Data is valuable. And as I’ve discovered in my own time, this is true both from a research and technological perspective; As we, as a society, approach quickly the horizon of Big Data and Cloud technologies.

An Introduction — How Well Do We Know Big Data?

It is without doubt (but maybe without such hyperbole) that our personal descriptors — including our name, email, buying habits, colour preferences, shoe size, and all other superfluous, identifying tidbits — are now permeant online. Access to this data is currently being passed virulently through your expanding network of Facebook friends, or exchanged between your Linked In business connections, or probed by zealous bots and adware. It is recorded further along the way with each instance that it is passed down, again and again, with each sign-up, cascading down the unending chain of clandestine 3rd party provocateurs.

Our data is valuable to those who fetish over it; Those who can find business in the unbounded worth of our collective data.

You know which data I am talking about too. In a Twitter sense, it is the kind of data we frivolously released as we made yet another tweet on our #Love4Bieber, or on the ludicrous display we saw last night, all without disabling our location settings — A Tweet I have come to notice is recognisable by the millions.

Countries with Trending Twitter Topics — Source: Creative Commons
But why is this significant? I would argue it’s because no data is inherently meaningless, especially when it comes in droves.

And that’s because it is ‘Big’ Data.

Big data, and that which positions organisations and groups to deify and cultivate the vast expanse of virtual information that now floats on about our tiny, blue orb, presents something that codifies our very existence. And, with it, it brings a unique tool-set for furthering our own social understanding. It allows us to quantify the underlying qualities of the world; those which we could garner a general picture of, but couldn’t finitely know due to their near-infinite volume and veracity. Whether this is into pixels the transforming of every avocado craft beer once gleefully photographed at our favourite post-neo-anarchist hipster cafe, allowing us to construct grand image recognition software, or the what-must-be-now petabytes of website URL’s which ever-extend your email’s infinitely-long ‘subscribed-to’ list, allowing us to tailor your online shopping campaign by accommodating to your preferences.

Big data is not just the extraction and archiving of important information, but also the endeavour of finding worth and enterprise in the masses of irrelevant data that is already available to us; that which has spawned simply as a consequence of being a part of the 21st century.

The concept of Big Data has amassed itself into plenty of superficial outcomes as well. For example, as the prime solution for all your business needs — “now presenting Google’s new service: the Entrepeneur’s New Clothes. Track your users and lump their behaviour across the Internet of Things in a way you’ve never seen before, bringing you The Cloud’s No.1 Open Source business platform. Able to streamline your IT obsolescence, optimize and automate your redundancy workflow, and hybridize your digital assets! Version 2.0 now streaming irrelevant user analytics directly to the smart fridge in your virtual tea room, all by way of ‘The Cloud’ ”.

But, Big Data nevertheless has amassed itself as an exciting foreground for us tech-savvy nerds; those compelled not to lose their chances to compete in the technological footrace of the 21st century’s ‘Information Big Bang’.

The Growth of Data Storage: 1986–2007

Where Does This Take Us?

I argue that yes these are things that can extend our knowledge of society and human interaction — and it has without doubt been enjoyable exploring them. But don’t forget, every opportunity to release and collect data en-masse is an extension to our marketability, and with more encompassing technology becoming available it is also a source for sinister points of exposure to our privacy and personal detail. This is merely a caution. It was one of the considerations that we noted with our project.

The following project is an example of my own foray into this kingdom. Into the realm of Data Analysis and Cloud Computing, and what it means to capture within these large swathes of data and tidbits interesting insights about the collective sources and suppliers from which they came, all the while cautious of what I am taking part in and what I’ve seen the technology can portend.

To see the next chapter in this story, see Part II

Twitter Analytics — Part II