The Fire Phone at the farmers market

Can it see kale?


The first half of this tab is going to sound somewhat negative: the familiar stance of a humanistic critic unimpressed by the latest whiz-bang technology because it falls short in some essential way. The tab will however quickly resolve into geeky enthusiasm. Please don’t miss the enthusiasm.

The most provocative feature in Amazon’s new Fire Phone, by a wide margin, is something the company calls Firefly. The premise is potent: Point your phone at something, anything, and the phone will recognize it.

https://twitter.com/fmanjoo/status/479325524175892480

Among the “items” used as examples in the Fire Phone’s unveiling, there were: a book, a song, a TV show, and a jar of Nutella.

This is, on one hand, quite magical, and on the other hand, totally depressing. Navneet Alang crystallizes the humanistic response:

https://twitter.com/navalang/status/479326894391693313

He’s on to something. With the exception of a few paintings, all of Amazon’s demo “items” were commercial products: things with ISBNs, bar codes, and/or spectral signatures. Things with price tags.

We did not see the Fire Phone recognize a eucalyptus tree.

CC-licensed photo from John Morgan: https://www.flickr.com/photos/aidanmorgan/2073661175

There is reason to suspect the Fire Phone cannot identify a goldfinch.

CC-licensed photo from nutmeg66: https://www.flickr.com/photos/rachel_s/3226596712

And I do not think the Fire Phone can tell me which of these “items” is kale.

This last one is the most troubling, because a system that greets a bag of frozen vegetables with a bar code like an old friend but draws a blank on a basket of fresh greens at the farmers market—that’s not just technical. That’s political.

But here’s the thing: The kale is coming.

There’s an iPhone app called Deep Belief, a tech demo from programmer Pete Warden. It’s free. Upon launching the app, you see this message:

The abstraction delights me: “the thing you want to recognize.” It could be a chair; it could be a person; it could be the Eiffel Tower. In this case, let’s address my criticism of Firefly. Let’s teach this app to recognize kale.

Deep Belief guides you through the process. First, you spend about a minute pointing the camera at some kale, sort of orbiting the greens, getting in close, pulling farther away. When the meter at the top fills all the way…

…you begin the next phase. You teach the app what kale isn’t.

This is the fun part of the process, because you just pan the phone wildly, trying to capture as much of the non-kale world as possible. Chairs, people, Eiffel Towers, anything, everything. For just a moment, you perceive the entire universe as composed of kale and non-kale. Is that the sky? No, it is non-kale. Are those the stars? No—they are non-kale.

Whoa.

After you’re done with all that, Deep Belief goes into recognition mode, and you can wave the phone around like a geiger counter, watching the app’s recognition meter rise and fall. Pointing the camera at the kale again, you get a full meter…

…and along with it, a bright pinging sound, like sonar. Ping! That’s the thing! I’m sure of it! Ping! Ping!

When you point the camera at a chair or a person or the Eiffel Tower, the recognition meter drops to zero. More impressively, when you point it at something that looks preeetty similar to kale, the app knows the difference.

The meter drops. No ping.

I should say that Deep Belief is far from perfect, still easily fooled. For example, the app is clearly on the fence about this broccoli.

But remember, this is just a tech demo, and it’s running on a phone, and it received two minutes of training. Remember, these techniques are improving all the time. I think it’s safe to say the broccoli will not be a challenge for long.

If Amazon’s Fire Phone could tell kale from Swiss chard, if it could recognize trees and birds, I think its polarity would flip entirely, and it would become a powerful ally of humanistic values. As it stands, Firefly adds itself to the forces expanding the commercial sphere, encroaching on public space, insisting that anything interesting must have a price tag. But of course, that’s Amazon: They’re in The Goldfinch detection business, not the goldfinch detection business.

If we ever do get a Firefly for all the things without price tags, we’ll probably get it from Google, a company that’s already working hard on computer vision optimized for public space. It’s lovely to imagine one of Google’s self-driving cars roaming around, looking everywhere at once, diligently noting street signs and stop lights… and noting also the trees standing alongside those streets and the birds perched alongside those lights.

Lovely, but not likely.

Maybe the National Park Service needs to get good at this.

At this point, the really deeply humanistic critics are thinking: “Give me a break. You need an app for this? Buy a bird book. Learn the names of trees.” Okay, fine. But, you know what? I have passed so much flora and fauna in my journeys around this fecund neighborhood of mine and wondered: What is that? If I had a humanistic Firefly to tell me, I’d know their names by now.

https://twitter.com/vitor_io/status/479392860756639744

Yep. That would be pretty great.

Here’s Pete Warden’s Deep Belief demo for iPhone. It is weird and powerful and flawed and fun and absolutely worth a few minutes of your day. Point it at something without a price tag.

Next Story — The Pickle: A Conversation About Making Digital Books
Currently Reading - The Pickle: A Conversation About Making Digital Books

The Pickle: A Conversation About Making Digital Books

1: Opening Salvo

Okay, Craig, I know you’re critical of the arrested state of ebooks today. For my part, I’m more… curious. It’s clear to me that, for all their commercial success, we don’t know what books on screens are supposed to look like; not yet. But that shouldn’t be surprising; the first Kindle came out a mere eight years ago, and most people have been reading books on screens for a few years at most.

It’s only now that we’re starting to see the really interesting work emerge.

I believe you’ve read The Pickle Index, and I think we can agree that it represents something interesting and new. The novel’s digital edition is much more than words on a screen; instead, it masquerades as a recipe app, complete with menus and lists and a wonderful little map. It’s quite slick; if it was a real recipe app, it would be a pretty solid one! Its creators, Eli Horowitz and Russell Quinn, use the idiom of the app to pull you more deeply into the story, to make you, as a reader, feel somehow like an accomplice.

It’s fabulous.

But the Pickle Index app (for reasons I don’t want to give away to people who haven’t read it yet) can only tell the Pickle Index story; the way it works is bound up with the tale it tells. Eli and Russell can’t reuse this machine for, say, The Istanbul Protocol or The Dragon Wizard. Those stories wouldn’t fit.

So, with your earlier criticism in mind: What do you think? Is this matching of content to container the road forward, or is it a lovely cul-de-sac? Is the non-naive, non-repackaged future of ebooks more of these unique apps, or is it some new, reusable “master format” that we have yet to invent?

Note to readers: This is (going to be) a long, loopy conversation. The Pickle Index is crisp and compact. Consider sampling its tangy delights.


The Indexed Pickle

  1. ➡ Opening Salvo [Robin Sloan]
  2. Blank Slates! [Craig Mod]
  3. Opportunity Cost [Robin Sloan]
  4. The Struggle (Sorry) [Russell Quinn]
  5. Self Selection & The Jaws of Venture Valley [Craig Mod]
  6. The Many Futures of Storytelling [Eli Horowitz]
  7. Recruitment [Robin Sloan]
  8. Blarp [Eli Horowitz]
  9. That Oracular Feeling [Robin Sloan]
  10. Benevolent Leaders of the Frothy Kingdom [Craig Mod]
  11. The Left-on-the-Floor Weirdness [Russell Quinn]
  12. Territory of the Book [Eli Horowitz]
  13. At This Moment [Robin Sloan]
  14. Make a Book, Even Just One [Craig Mod]
Next Story — Selling on the slider
Currently Reading - Selling on the slider

Selling on the slider

I was an economics major in college, and one of the things you do as an econ major is draw a lot of graphs, many of which purport to show the supply and demand for a particular good. You’re always looking for the point at which the curves cross. That point tells you, among other things, the price for that good.

The price.

I drew so many of these dumb graphs, and yet I never noticed the deep assumption built into the exercise: That of course you must choose a single price for a good.

So you can print it in the catalog.

So you can write it on the tags.

I learned all of this around the turn of the century, and prices have gotten a lot more interesting in the years since, thanks to the internet and, more specifically, to the fact that so many prices are now shown on screens rather than printed on paper. Today, the notion of the price is breaking down, being replaced by something more flexible.

Some of the forces behind that change are of course quite cold and rational, powered by new algorithms. But more of them are humane — the kind of forces we never learned about in econ 101 — and powered by a new interface.

Well. Not even that new.

Ta-da

The slider is a hot piece of technology that will play a central role in the future of prices.

The slider need not be cold, and it need not be rational.

It’s not economic efficiency that moves the slider, but emotion.

And for my money, this is the most interesting checkout screen on the whole internet.

That’s what you see when you’re about to buy a Humble Bundle. There’s a lot going on there, so let me just break it down quickly. The Humble Bundle is a popular product with roots in indie video games that’s expanded into music, movies, books, and more. The deal goes like this:

  1. You, the potential customer, are presented with a bundle of merchandise. Maybe it’s a dozen video games; maybe a truckload of digital comics.
  2. You decide what price you’d like to pay for all of it together. It can be as little as one penny! (This is section 1 in the screenshot above.)
  3. You also decide how that price should be apportioned, splitting it between the content makers, a set of related charities, and the company Humble Bundle, Inc. (This is section 2 above.)

There are some nuances. Most bundles include extra content that you only receive if you meet a certain minimum payment — $10, perhaps. More interestingly, bundles almost always offer extra content that you receive only if you exceed the current average payment — an incentive that, of course, has the effect of slowly raising that average over time.

Honestly, it feels less like a checkout screen and more like a video game.

Humble Bundle’s sliders are the most elaborate you’ll find anywhere, but the basic element is all over the place. The indie video distributor VHX has a nice one:

And while the sell-anything site Gumroad lacks a draggable slider…

…it delivers a clear message: There is no the price.

(Kickstarter offers the same ability to pay extra, but it’s often lost in the complexity of the project rewards. I think Kickstarter projects tend to feel more like small catalogs — listing many fixed prices — than sliding scales.)

The slider is a natural fit when you’re buying something from a specific individual, possibly someone known to you or whose work you’ve long admired. It acts as an affinity-meter through which you can convert surplus units of love and gratitude into cold, hard dollars.

But there’s more to the slider, particularly in Humble Bundle’s implementation. It has to do with communication.

The information content of most payments is one bit: either you agree to the asking price and make the purchase, or you don’t. When a payment goes through a slider, that information content increases. When it goes through an interface like Humble Bundle’s… well, I mean, look at this!

A payment can bloom into a rich little packet of information, much better than a survey because it’s weighted with dollars.

(It’s quite possible you feel exhausted just looking at Humble Bundle’s bank of sliders. In practice, they’re easy to manipulate and surprisingly fun to fiddle with. It’s really worth playing around with them for a moment.)

The slider is not appropriate in all, or even most, situations. It would, for example, be an affront to see one on Apple’s checkout screen. “Oh, really, hugely profitable megacorporation? I can elect to give you more money? Thanks.” Most products’ prices will stay fixed — catalogs and tags.

But the slider still has gains to make. There are a lot of products out there that would benefit from a more flexible, more emotional form of payment. Maybe most products created and sold by individuals should be sold on the slider. Maybe most writing should be! We don’t know yet.

That’s exciting. It means we’re still learning new things about prices and markets, and about people, too.

We’re learning that a price can be a bundle of information richer and more useful than simply “deal or no deal.”

We’re learning that it was perhaps never quite right to price indie movies the way we priced refrigerators.

Soon, we might have to start revising those economics textbooks.

Sign up to continue reading what matters most to you

Great stories deserve a great audience

Continue reading