Where are the well paying jobs in the Bay Area?

They are pretty much where you think they are*.

Alright, this story is not really the story about well paying jobs in the Bay Area. It is the story of how I went about writing (…um actually coding) a story about affordable jobs in the Bay Area.

The project was an experiment of sorts in what I’ll call interactive data visualization and storytelling. Many of you, no doubt have come across versions of these in different outlets. The New York Times has a great series of these. As such they are not new types of stories. They are stories news outlets have always covered such as stories about income disparity or obesity rates in different parts of the country. The difference is, instead of the usual static map representing one aspect of the data, we have a dynamic map that can load all of the data, giving readers the chance to see multiple aspects of the data in a single map.

Data vs. Narrative

For a while, these sorts of data driven interactive stories were seen as the future of journalism until about a few weeks ago when Nate Sliver launched his revamped version of Five Thirty Eight. The larger world of journalism was not happy with results and instead of elevating the profile of data driven journalism, as Sliver had sought, data journalism took a hit. In case you missed it you can read more about this ongoing debate, here, here, here and here.

Personally, I fail to see how we can avoid data driven stories, simply because data is central to gaining a comprehensive understanding of many topics relevant to lives. Silver’s critique of political punditry is instructive. He has argued, and rightfully so, that political narratives are often unsupported by the data, and can thus mislead readers. On the other hand, it is easy to find numbers and statistics of all sorts on any topic. Such numbers, when presented without a narrative context, feel uninteresting and fail to enhance our understanding in any meaningful way.(A large part of the criticism against Silver’s site fell into this category.) More seriously, just as narratives can mislead, so can data driven stories if the methodology behind the data used is unsound. In other words, all data are not made equal. There is good data and bad data.

In one way or the other stories of the future must somehow merge data and narrative into some elegant form. But figuring out what this elegant form will take, as I discovered is a lot harder than it looks.

Digging for Data

Not long ago, I was listening to an NPR interview with one of the lead authors of a large Harvard study on social mobility. In his conversation with the host, Nathan Hendren, explained how regions with large urban sprawl, had very poor social mobility. The reason was the lack of reasonably priced transportation. Even if a region was able to generate many jobs, people could remain stuck in suburbs and exurbs unable to get to areas with better paying jobs, limiting their chances of upward social mobility.

This got me wondering about transportation and jobs in Bay Area. I wondered as I read and listened to the stories about the increasing hostility between tech workers and activists blocking the Google buses, how the Bay Area fared in transportation costs. I was partly inspired by this visualization of income inequality and public transportation, which itself was based on The New Yorker version of income inequality and subway stops in New York city.

The idea was to combine jobs data and transportation cost data, in a similar map of the region. The visualization would then be set up to look at where the well paying jobs in the region were and how much it would cost to get those jobs from different parts of the region. The original idea for the story was, “How much would it cost to get to well paying job in the Bay Area.”

Since the transportation data already existed, as publicly available information about ticket prices for public transit systems, all I had to do was find the jobs data. After a fair bit of digging I found the closest thing I needed from Bureau of Labor Statistics. But there was a problem.

What is the story?

I realized as I looked at the BLS data, that the story as I had originally conceived it would not work. Though I had the jobs data, it was categorized by metro region, not counties. Metro regions combine multiple counties. For example San Francisco metro region combines Marin, San Francisco and San Mateo counties. For the story to work I needed the jobs data in these counties separately so that I could show what it would cost to get from Marin county to a well paying job in San Mateo county.

It was time to rethink the story.

There was another thread of thought that looked promising. The jobs data included information about hourly wages and annual salaries for different occupations in the region. If I combined jobs data with cost of living data, I would be able to answer another question that had been nagging me.

The inequality issue in the Bay Area, especially in San Francisco has largely played itself out as a housing affordability issue, with well earning tech workers driving up rents in many neighborhoods and displacing long-term residents. Not surprisingly, the discussions about potential solutions to inequality related issues have foused on creating more affordable housing. This certainly sounds like a meaningful solution, but what if the region does not create enough good paying jobs to being with? If this is case, a housing solution is a Band-Aid at best.

To explore this story line, I had to define or in some way quantify “a good paying job.” Good, after all is really a relative term. I could use the official poverty threshold numbers as comparison figures. As an example, annual salary of $20,000 sounds pretty good, compared to the official national poverty for single person as $11,600 for the year. These national numbers though don’t reflect regional realities. I could however make this story work, if I could find cost of living data for the region.

Fortunately the cost of living data was something I already had. In an earlier experiment, I’d extracted county by county cost of living data from the MIT Living Wage calculator project. Now all I needed was data for creating the map. I needed to highlight the counties that represented a metro region when it was selected. A bit of digging and I found the files I needed from the census site. All the data for the story was now in place.

Coding vs Writing

Writers of nonfiction essays or classic narrative journalism, learn early on that to hold their readers’ interest their writing must be cogent. We can look to our own experience as readers to see why. As readers we find it hard to keep pace with a writer, who meanders, and explores endless asides. (There are though writers who have manged to make asides and into an art form unto itself. David Foster Wallace comes to mind.) As we read, we can’t help wonder, where is this going? What are you trying to say? What is your point? We expect some kind of convergence, a nugget, a well thought out argument or a thesis of some sort.

The overarching structure of a narrative project is, you could say, linear. The overarching structure of a coding project, in contrast is entirely non-linear.

It has to be. When you create an application, whether it is a stand alone application or web based application, you are creating something that your users can interact with in multiple ways. As the person creating the application, your job is to anticipate “all possible states.” As users, unlike as readers, our expectations when it comes to using an application is unfettered exploration. It is our prerogative to use the features of the application as we choose, in whatever sequence that suits us. We expect the developers to give us something that does not break when we use it.

I found myself struggling between these opposing instincts of coding and writing, as I was putting this piece together. My coding instincts wanted me to provide, as many features and possibilities for exploring the data in multiple ways. My writing instincts wanted me to narrow things down, remove features and provide a conclusion.

At least in this experiment, I think the coder won, and I left things open for exploration, instead of converging on a thesis. This was partly because I came to a point in the process, where I realized I had lost my ability to make decisive choices to reshape the project for the better. I had to stop.

Questions for future experiments

There will be more such interactive story experiments to come. In an attempt to make incremental improvements with each here are a few questions, I’d love to hear your thoughts on.

1. In your experience of interactive stories on the web, can you share one that you felt did a reasonable job of balancing the interactive experience and narrative? What was it about this particular story that worked for you?
2. How/ what could I have done differently with the jobs and wages story to enhance your experience of the story?

*If you must know, the best paying job are in the Valley. The San Jose metro region has the greatest proportion of well paying occupations. And, as you might guess most of these are software jobs of one kind or another.

If you are interested in collaborating on interactive storytelling experiments, get in touch. (Handle = ajmobidesign / Handle + gmail or twitter + Handle)