CoolHunting in Cambridge
A few weeks ago I had the good fortune to attend a CoolHunting Academy at MIT in Cambridge. The week-long workshop marked the kickoff of an exciting collaborative research initiative between MIT, Savannah College of Art & Design, University of Helsinki and University of Cologne.
CoolHunting operates by the principles of swarm creativity and collective intelligence. No one person knows everything about any single topic, but together our collective knowledge about a topic is immense. By observing and analyzing the emergent patterns of communication within social networks we can identify and track trends as they ripple through society. We can also scale the centrality of actors within a given network, meaning the ease of access any particular node has with any other node within the network. If I mention several people in an article, that really doesn’t mean too much for my own centrality. But if several people each mention me in their own articles, I must be a significant person and my centrality is increased. Those who are more central are considered to be trendsetters, thought leaders, or gatekeepers within a particular network.
Condor is an exciting program developed under Peter Gloor at MIT that allows us to troll the vast ocean of content that is the active conversation of the web and then aggregate, analyze and visualize the massive amount of information that is pulled back. As we learned during our week in Cambridge, the best way to understand this software is to see it in action. And so, I will attempt to demonstrate a quick application of Condor in a market very near and dear to my heart.
Exploring the Mobile Applications Market
In the long run, every market becomes a two-horse race. — Al Ries & Jack Trout. authors of The 22 Immutable Laws of Marketing
In June of 2007 Apple released a revolutionary new device that forever changed both the mobile telecommunications and digital media download industries. While the the device itself has become the focus of much salivating and shameless imitation, the real success of the artifact can be largely attributed to the explosion of applications brought forth by enterprising young developers the world over. A whole new frontier was opened up, and naturally many of the best-suited competitors scrambled to grab what little scraps of market-share Apple left available. Behind the iPhone platform we find Google Android, BlackBerry, and the Palm Pre. If it is true that “every market becomes a two-horse race,” then perhaps we can catch a glimpse of the tone of the web to see how this market will play.
Condor in Action
Step 1: First I want to see the relative scale of activity of the terms “iPhone App,” “Android App,” “BlackBerry App” and “Palm Pre App” on the web. CoolTrend.ch is designed to do just that. CoolTrend.ch lets visually maps out the relationship between results of search queries, allowing me to very quickly see the most central hubs of information within the network. This alone could turn the world of web advertising appraisals on its head, but let’s bench that discussion for another time.
Figure 1a shows the output from this process. Surprisingly, iPhone — the untouchable market leader — owns just 31% of the web buzz-share, trailing Google’s 33%. BlackBerry App follows with 25%, and Palm Pre App with only 11%. The iPhone is certainly not going anywhere, having recently announced its one-billionth app download from its App Store, but removing it from the analysis yields another interesting perspective.
Figure 1a. CoolTrend output for “iPhone App” “BlackBerry App” “Palm Pre App” & “Google Android App”
Figure 1b shows the aggregate web buzz of “Google Android App,” “BlackBerry App,” and “Palm Pre App.” Google weighs in with 44%, BlackBerry takes 39%, and Palm picks up the last 17%. One could now argue that Google is shaping up to be a pretty solid “second horse,” but BlackBerry is keeping a fairly close margin.
Figure 1b. CoolTrend output for “BlackBerry App” “Palm Pre App” & “Google Android App”
Step 2: I want to see what key terms are being used within the blogosphere regarding “iPhone App,” “Google Android App,” “BlackBerry App” and “Palm Pre App.” I could probably sit down and write out a nice long list of what I think are key terms within the industry, but there’s no way to ensure I’ll catch every single relevant word. I fire up MySQL, launch Condor, create a new database, and set the “One Degree Collector” loose on Google blog search. Condor hits Google up for the returned snippets of 20 results for each query and creates a new data-set for each term. Next I run “Content Process” to calculate and extract the most frequently used terms for each data-set. Those terms are then compiled together to create a pretty accurate snapshot of the key terminology used in reference to my original four queries.
I want to visualize the relative association of these terms, so merge my four data-sets together (after making a backup, of course) and run the Content Process again against the new single data-set. This time I also import my list of industry terms. Figure 2 shows the resultant Static View of Terms output for my merged data-set. Terms appear as nodes within a network, connected by edges (blue lines) and spatially arranged to demonstrate how frequently they occur together.
Figure 2. Static View of Terms for Industry Terminology
Device names and related terms cluster together in different regions of the map — Palm Pre appears far left, Google Android appears top just left of center, Apple appears far top right, and BlackBerry below and just right of center — all orbiting around the golden item of the day: the word “app.”
Something very interesting is already visible in the area of Palm Pre. Notice the other terms clustering around Palm’s related terms. “itunes” is the big anomaly, as well as “syncing,” “upgrade,” “coming,” “pre owners,” etc. Palm and Apple have recently been locked in a “Sync War” as Palm attempts to allow the Pre to sync with iTunes. Apple responded by pushing updated versions of iTunes to disallow this behavior. Apparently they have been going back and forth for awhile now and we can clearly see this info resonating through the blogosphere. It’s possible this contributed to Palm’s 11–17% of web buzz-share from earlier.
Another interesting detail from the Palm cluster is that a few outlier terms such as “delayed,” “waiting,” “Lonely” connect with terms like “application” and “app catalog.” I run a quick google search with those terms plus “Palm Pre App” and find a plethora of articles citing serious development problems when authoring apps.
Step 3: I need to get a little deeper into these devices so I create a new database and run the “Blog Collector” tool. This is essentially the same component as the One Degree Collector, except it submits an addition query for each result for each additional degree you request. For example, if I request 20 results at a degree of 2 I could potentially pull back 400 articles for each of my four queries: “iPhone App,” “Google Android App,” “BlackBerry App” and “Palm Pre App.” I also opt to include the full content of the results, which takes a considerable amount of time but will give me the depth of information that I need.
My results are first returned as four individual data-sets, so I can analyze each dataset individually. I want to better understand the negative sentiments behind the Palm Pre, so I open text pad and start assembling a list of negative experience terms such as “problems, issues, headache, annoying, terrible, hacking, confusing, errors, crashes, breaks, fails, tired, frustrated” in various forms. I manually merge this with the term list from my previous One Degree Collection of Palm Pre App terms. I add a few phrases that build off of those previous outlier terms and contextualize the situation a little more, such as “tired of waiting” or “can’t wait for,” as well as “development” and “sdk” (software development kit). I save this list, return to Condor, select the “Palm Pre App” data-set and run a Content Process, importing the new term list. If my hypothetical phrases do actually exist in the content with any real frequency it will be demonstrated in the Static Term View rendering.
Figure 3 shows the rendered output, and demonstrates exactly what I had suspected. There is a strong clustering of terms like “waiting, can’t wait for, app, more apps, apps, store” with nearby terms such as “development, developer, fails, failing, problems, crash, not enough, sdk.” What we can see here is that the nature of the collective conversation about regarding “Palm Pre App” is that there are big problems with the SDK which are inhibiting developers from pushing out any respectable number of apps. Users are getting tired of waiting for something that may honestly never come. Upon further research I found that the Palm Pre App Catalog currently features 40 apps with total downloads at around 1 million. I think it’s safe to say Palm needs to start thinking about finding its own category.
Figure 3. Static View of Terms for “Palm Pre App” terminology plus negative experience terms
Google’s Android Marketplace features 4,900 apps with total downloads around 40 million. Additionally, the service has direct ties to leverage the incredible might of the Google infrastructure, which will be hard for anyone to compete with. BlackBerry, on the other hand, features only 1,100 apps and has not yet disclosed total downloads. BlackBerry Apps can be acquired any one of a number of ways, including direct-from-web download, 3rd party app stores, and network operators such as Verizon, T-Mobile, and Verizon, making it nearly impossible for the folks at Research in Motion to measure the adoption of this feature. If the old adage “what gets measured gets managed (Peter Drucker)” holds true, I think we can see where this one is heading.
There are also some serious UX snags standing in RIM’s way of pushing respectable adoption by its user base. I own a BlackBerry Pearl (8100) and recently partook in the full experience. Since my phone is serviced by AT&T, BlackBerry “App World” did not come preloaded. Instead I had to go online and request that a web link containing the downloadable program be sent to my phone. I was attempting this step on the phone itself but unfortunately I was using the Opera Mini browser, which would not work with the BlackBerry website. I had to close Opera Mini and start over using BlackBerry’s wretched default browser. Twenty minutes later I had successfully downloaded the “App World” app and was required to reboot to install the program. The reboot process took a lifetime of its own but soon I was in the store and ready to download my first app. I had no intention of buying an app so I looked for the highest rated freebie apps to download. My first app was Pandora. I loved Pandora for iPhone but the BlackBerry lacks the basic capabilities to maintain a stream of music, let alone play it at any enjoyable volume or clarity. Bust. I also downloaded BattleShip. It’s no longer on my phone either.
The Wrap-Up
As I demonstrated earlier, Condor is an incredible tool that allows us to aggregate, analyze and visualize the vastly complex exchange of information and ideas within the web. However, like any great technological innovation, it is only as effective as the operators’ interpretation of what it yields. There is an immense amount of noise resonating through the web, so the real trick is to figure out which signals to listen for to assemble any sort of reliable image of what is happening out there in the expanse.
