This Week in Data #5
My thoughts on what’s timely, interesting or quirky in the world of data. If you’d like to receive this in your inbox every week, subscribe here.
Kick Off
If you ever wondered whether coders have a moral code, you got an answer in the news this week that Apple engineers would quit rather than comply with government orders to unlock the iPhone. This, in fact, is allowed by the professional organization called the Association for Computing Machinery, which says that programmers should obey laws unless there is a “compelling ethical basis not to do so” and that they should respect the privacy of others. I found that interesting, as well as this Wired story on a founder of a company that specialized in secure email. He shut his company down a few years ago rather than comply with an order from the government. An important issue for us all to watch.
In the News
Scientists at the University of Texas introduced a new supercomputer, called The Wrangler, that specializes in dealing with lots of data. This marks an important shift in supercomputing. Traditionally, supercomputing has been used for scientific and engineering applications that involved simulations, but not large amounts of data. “The Wrangler” moves supercomputing closer to industries, like e-commerce, where working with lots of data has become standard. It’s a bit of a convergence between science and industry, tackling both big compute and big data at the same time.
In tech, we throw around the words “open” versus “closed” a lot, but there’s a difference between “open source” in terms of a program’s codebase and “open” in terms of the ability to tinker with a product. Whether you’ve thought about this or not, here’s a great column on it by re/code’s Walt Mossberg.
In Industry
Google is getting into the municipal data business. The company’s subsidiary, Sidewalk Labs, announced a partnership this with the federal Department of Transportation to track traffic in U.S. cities. Municipal data analysis has been a bright spot for IBM, with itsSmarter Cities initiative, and this signals Google taking on IBM in this area.
There was an interesting panel at SXSW Interactive about hacking hospital web sites, and it provides a good overview of how web site hacks tend to play out in general.
The cloud computing war is heating up. Amazon, with its Amazon Web Service, has been the leader, but increasingly Microsoft’s Azure and Google’s Cloud Platform are trying to pick up big new clients. There’s a lot to evaluate when you choose which cloud to use for your data. Given how much data we run in the cloud, we have thought a lot about the various options and I will write more on it soon.
Quirky Corner
Sometimes being in the data world is just fun — there are so many cool things bubbling up all around us. Here are a few:
There’s now a machine learning algorithm that can figure out which tweets posted on Twitter are “drunk tweets.”
Second, Twitter has facial recognition tools that reveals the demographics of people following various political campaigns on Twitter.
Last, Google just launched Unfiltered.News, a tool that shows what stories are bubbling up all over the world. Great write up about it at NiemanLab.
What’s happening at Ufora
Based on requests from our consulting customers, we’ve decided to offer customers a hosted version of TestLooper, the statistical testing framework we use at Ufora. It lets us make changes to the code with confidence that we aren’t introducing unintended side-effects, and has helped us track down some really hard-to-find bugs. We built TestLooper to work with the cloud, so when we run tests on new code changes, they run in 10 minutes, not 10 hours. It’s greatly sped up our development process and allowed for much more sophisticated tests than any other testing platform on the market. TestLooper can be used on most people’s code bases and is not tied uniquely to Ufora. Drop me a line if you’d like to talk about how it might help you.
Thanks for reading, and have a great weekend!
