Dennis Crowley’s Long Game
AR is part of Foursquare’s future — but not the way you might think
Foursquare and its check-in spinoff Swarm got major updates this month, with Foursquare’s home screen now highlighting its search utility and Swarm highlighting real deals (or perks) for the coins you get while checking in. Both upgrades are steps in a nonstop iteration process that began when founder Dennis Crowley created his earlier platform Dodgeball (“mobile social software”) while a grad student at NYU’s Interactive Telecommunciations Program (ITP) in 2001, as a “way for all the laid-off kids to meet up” in the wake of the dotcom crash.
When I interviewed Crowley for GQ Australia in 2010, Foursquare was still seen as a souped-up, smartphone version of friend-finder Dodgeball (which sent alerts by text message). But Crowley’s vision was bigger. “We’re at 10 percent of where I want us to be,” he said then. Looking ahead to what he termed “passive augmented reality,” he then said, “I wish Foursquare was better at telling me where to go and guiding me through the experience.” This became the goal behind splitting off Swarm’s check-ins in 2014, allowing Foursquare to rebrand as the “Best City Guide.”
So when Pokémon GO became a runaway hit this July — “gateway AR,” some called it — I noticed Crowley’s enthusiasm over social media and wondered if he saw it as a breakthrough for his own vision. Foursquare has long blended utility (both social and recommendations) with gamification (mayors, badges and the leaderboard, now all part of Swarm). I visited Foursquare’s Soho offices to find out.
There Is Only R: Has the development of AR in general and the mass adoption of Pokémon GO changed your thinking at all about your own goals and what you want Foursquare or Swarm to be?
Dennis Crowley: Pokémon GO stuff is super interesting, but I don’t think the AR part of it is what’s interesting. What Pokémon GO did for location-based games is a big deal. What Pokémon GO did for AR? I don’t think that is a big deal.
AR before Pokémon GO was still, “let me hold up my phone and I’ll take a look at what’s happening nearby through this lens.” And it’s basically what Pokémon is. You’re not moving around the city; you’re standing where this thing was placed at random.
So I’m still not super bullish on that. The type of AR that I think is the future is this idea of what Google tried to do with Google Glass: a little thing in front of your face — but that was a screen. It didn’t really change based on what you were looking at.
What Magic Leap is talking about doing, sending light signals right into your eye so you see things that aren’t there — that is it. I haven’t used that device, but heard from a wide variety of people that have, that’s the real deal — I’ve never seen it. I want to see it.
Until that happens, and I think even when they come out with something — it might be a huge helmet, who knows — there’s this other type of more passive AR. You’re experiencing the world as normal. You’re not walking around with your face in Twitter, you’re just walking like you normally do.
Then something is giving you visual cues of what’s in front of you. Most people think AR is what we see, but it can also be what you hear through the Apple AirPods — or any type of audio that you’re wearing all the time: “Hey, Dennis, look left, there’s an awesome piece of street art here. Turn right and go into this coffee shop.”
That, I believe, is a form of augmented reality that’s more interesting. I’m exploring the world normally, without a screen in front of my face, but there’s something telling me to go left or go right. We do that with push notifications. You’re in this neighborhood? Here’s how I want you to experience it. It’s not augmented reality creatures like Pokémon, but it is a message that’s supposed to change the way you’re experiencing the environment in front of you.
Now, because so much of Foursquare is focused on the real world, we’re not trying to get people to escape into a virtual world. When you look at demos from Magic Leap, it’s like we’re in this room and so are all the Star Wars characters. Pokémon is characters on this table and I have to catch it, and it’s putting fiction into the real world.
It’s entertainment.
Yeah, it’s someone’s story, and they’re bringing this story into the real world. It doesn’t have to be fiction. It could be utility. Augmented reality could be, I’m looking at you and seeing you have a score of 77 because you’re on Twitter. Now I’m seeing something that cues me off to how I know you or your level of influence.
When most people talk VR, they think about how do you bring fiction and entertainment into the real world. But it can also just be utility. And I think that’s interesting from a city guide perspective — exploring cities. In a “how do I make the most out of what’s happening” in front of me type of way.
Foursquare and Swarm always had a strand of utility and a big strand of gamification. You see that now with AR. Is there a way you can take advantage of this and advance how people use Foursquare, including the city guide function?
To be honest I don’t think a lot about what the AR or VR experience will be, not because they’re not interesting, but I have seen no one in the last 300 days walking around the city wearing one of these things.
So even though it’s interesting theoretically, it’s not interesting to me in practice. I’m interested — and I’ve always been — in not imagining this world of the future, but building a bit of the future and experiencing it today.
When we built Dodgeball, we were trying to do stuff that was impossible. Our thinking was: there are no apps, no GPS, so let’s make the best approximation that we can of a map of where all your friends are. We’ll just do it through text messaging.
I want to build something for the piece of hardware that comes out and people are still trying to figure out what to do with it.
With Foursquare, it was, let’s try to build this thing that tells you where to go because it understands where you’ve been. Technology didn’t exist — we did it with check-ins. Now we’re starting to fulfill it a little more aggressively.
So when I think about the stuff that I want to work on immediately here, it’s about, it’s imagining: will so many people buy those iPhone 7's, and get the wireless headphones, that you’ll see one out of every ten people wearing those wireless headphones all the time in the city?
If that’s something we see, then why don’t we build something just for those wireless headphones? I want to build something for the piece of hardware that comes out and people are still trying to figure out what to do with it.
You need the ecosystem to be in place, to have critical mass to make that worthwhile?
Yeah, there’s gotta be something that’s out there. Like when Google Glass came out, it’s like, “let’s build an app for this.” And we build one and then you see people really aren’t wearing it.
It’s like, okay, this is kind of what it was advertised to be, but people aren’t adopting it in a way that other people kind of expect it, so we kind of laid off that a little bit.
There’s not a room here where we have all of the virtual reality headsets and we work in silence trying to imagine this future. When those things start to become, not mainstream, but when real people start using them on the streets, that’s when we’ll say okay, this is how we see this technology is being used, let’s find a way to make Foursquare interesting and relevant.
I understand “building for now,” but I’m little surprised. I’ve always seen what you were doing as fairly future-focused.
The moment that someone comes out with something that we can build for, we’ll build for it. We’ll build the thing that as you’re walking down the street, you might see Swarm virtual coins, if you go in there, there’s 10 coins to be had. For perks, if you go in here, there’s virtual coupons to be had.
You don’t want something that’s so distracting that people can’t live their lives, but you want something that calls out this place is more special than the other place and that’s why you should go there.
Science fiction movies have given us a glimpse of what I think this stuff will look like. If you do have something that will project a virtual image in front of you, I don’t think Foursquare is ever going to do dragons that you have to slay. Instead we’ll do subtle reminders that hey, this place is interesting, you should go in there. And you go in, maybe there’s something we point out in the app.
You can imagine using a Foursquare version of this [AR], and you’re looking at a menu at a restaurant, and you see your face and Alex’s face next to one of the menu items. Because we understand: Oh, your friends have been here, your friends have had this dish before. We can do OCR [optical character recognition] on the menu, and we understand, this is the thing that you want — and we highlight it.
There’s cool ways to imagine it, and I like to imagine that once those things come out, if someone dropped a next-gen headset here and they’re like, “build something for us so we can demo it three months from now on stage,” I’d say, “alright, let’s get a team together to do that.”
But in the absence of that hardware being out, we focus on what is the next piece of universally accepted hardware that we can build for that will give people a little bit more of a glimpse into the future. And when I look at what that piece of hardware is, I’m thinking it’s more audio than visual.
I’ve demo’d the audio stuff, I’ve seen it. It’s not as sexy as the visual stuff, but at least it’s here, and so I’d rather build to the stuff that’s here than spend tons of cycles trying to imagine the stuff that doesn’t yet warrant building for.
Since you started getting more deeply into push notifications, what have you learned about what notifications or guidance people really want from an app like Foursquare?
I think we’re probably the best in the world at understanding where someone is, what building are they inside, what’s the proper name of the place they’re at. Then, understanding their relationship to that place, like did you just walk into Chipotle, and have you eaten here before, is this a category you go regularly, is this a new place for you? Are you in a familiar neighborhood or is this totally out of left field?
Did you just walk into La Colombe super hipster coffee shop, is this out of your element? Do you normally go to Starbucks, is this your regular place, is there something here you need to know about?
We’re really good at identifying the moment people walk in and out of places, then serving up a piece of content that can tell them, “Okay, you’re here, we think this place is interesting for you — now, this is the most interesting thing that we can tell you about this place.”
So we do that superpower, so to speak. We alone use our apps in this Foursquare format. We’re starting to build tools so that other developers can build those things. When you want to imagine what the future of AR — whether visual or audio form — will be, a core component has to be it has to know what you’re looking at.
It has to be able to identify this is Peter’s face or Dorothy’s face, like that’s hard to do. It’s got to be able to identify that you just walked into a bakery versus the butcher shop next door. That’s a hard thing to do.
Now, outside of just developing these apps, all the data that we have — we’re basically building the plumbing that enables this future world to work. By the plumbing I mean: GPS has been around forever, but it’s pretty basic, you just get coordinates. We’re building a better version of that, so as you walk into the bakery or whatever, the app can tell your earphones, your phone, your head mounted display: Dennis just walked into this bakery, has never been here before, has never been to this part of town before, is visiting from San Francisco, might be looking for this because his phone has not stopped at another place recently.
We can provide all this contextual information to whatever app wants it. Then the display or audio can start to say: all right, what decisions do we want to make about that? Does that mean we want to highlight certain things on the menu? Does that mean we want some character to sing and dance with a menu item on top of the counter?
People will dream up all sorts of crazy things that they’ll do with AR, VR, audiovisual whatever, and it will be our plumbing that is giving those apps instructions on: okay, this user just walked in and this is how this person is feeling in relation to the world right now, so these are the types of messages that you should display.
You’ve come out with findings about Chipotle and Trump properties that make people aware Foursquare has a lot of data. Has Foursquare been able to use this data to make your own engagement stronger and drive recommendations you think are going to be useful?
To go back to that superpowers analogy, we’re able to understand at this point where millions of devices are going every single day, which kind of gives us a unique look at traffic patterns in and out of different bars, restaurants, shops, parks, whatever those places are.
When you understand the way people are moving through the world, as you’ve seen, you can make predictions and say: listen, we can predict the movement of stocks. We know the sales data before it’s recorded. We know people are going in and out of a place.
There’s all these enterprise pieces for advertising data, analytics, urban planning, all interesting stuff. Then there’s use cases for it too: all right, Peter is walking through the Upper East Side for the first time in six months. We know about the places he’s downtown, what should we call your attention to as you walk into this neighborhood you haven’t visited recently?
There’s a difference between games and play. People look at gamification: okay, this is turning the real world into a game. But it can very much be making everyday things just seem more playful and delightful.
We might send you a push message that says okay, there’s a great coffee shop right around the corner, you should go there because I know you like coffee shops, and I can see you haven’t been to one within the last six hours, so maybe it’s something interesting. We could do that with a push message, or we could push it out to a visual interface, to an audio interface.
But the thing that’s fun to imagine is: we have all this data, we know a lot about what people like, about the types of places they go to and might want to go to. We know what’s popular, what’s not popular, what’s rising in popularity. We know what’s opened, what’s closed. We know all this stuff based on where phones have been going, and a way to link all that together.
So the creative and fun stuff for us is now, how do we present that to people so it changes the way they experience cities? As opposed to just wandering around, using a guidebook. As opposed to picking up your phone and asking it, where should I go? There’s an opportunity for the software to tap you on the shoulder and say I know where you should go, you don’t need to open an app, I can tell you what to do next.
Like: I recognize that you’re a little bit out of your element right now, and you might be wanting something, so I’ll get 30 seconds ahead of you before you think about taking your phone out of your pocket. I’ll push you some alert or notification that subtly suggests something to do.
Dodgeball was all social utility. Your friends are at this bar. When Foursquare started, it layered in a heavy element of gamification, with mayors, badges and everything else you’ve introduced — coins and now perks. You’ve been talking about the recommendation utility. Do you see people’s relationships with games helping, or does it get in the way of the utility?
There’s a difference between games and play. I love the Pokémon game, but people get tired of games after a while. There’s a reason why you don’t see everyone still running around playing that, because people have played it and they got as far as they need to get in that game. And then they go find something else to do.
If anything, we’ve erred on the side of trying to make our things playful. Playful utilities. This is Dodgeball from the early days: a utility that helps you find your friends, and does it in a playful way. Foursquare in the early days. Check in to places: it’s playful, it tells you where to go, tells you where people are. But it’s got this layer of flirtation and playfulness on top of it, which I think makes it a little bit more accessible.
On the other end of that spectrum, we have things like LinkedIn. LinkedIn is pretty serious and static, and kind of devoid of personality. I think what we try to do is make apps that have lots of personality and can get an emotional reaction out of people.
When people look at gamification, they’re like, okay, this is turning the real world into a game. Which isn’t always the case. It can very much be making everyday things just seem more playful and delightful. And I think the Swarm app is probably one of the best examples of that that you can find these days.
People anticipate that VR won’t really take off until it’s social. You were a social network before there was Friendster, you’ve been not only with location but with social since the very beginning. Do you have a vision for the future where the social graph people have been tending all these years through your app will play a new role?
Yeah, Facebook’s bet on Oculus is exactly this. Are people going to not want to meet in message or comment threads anymore, and instead they want to sit around over a social table and discuss the good times? That’s entirely a possibility and Facebook has made a big bid on that.
Will it affect your business?
It’s hard to imagine what that’s going to look like. The business of Foursquare is providing data and tools for people to build these amazing things. So they can build virtual reality or mobile applications that change based upon the places they walk in and out of, games that change based upon where you play them. As well as providing data to a wide variety of enterprise advertising and even consumer apps, so they can make great things.
Some of the most interesting Oculus demos are where three of you have headsets on, and all sitting next to each other, and I can see you moving as a virtual thing. That’s a really weird and awesome thing to experience.
Augmented reality is a thousand times harder because that thing has to know that’s Peter and that’s Dorothy. That’s a male, that’s a female. You’re inside of a shop, and that’s a piece of bread, that’s a sandwich, that’s cheese. That’s really, really hard to do.
There are going to be a whole bunch of companies that are probably similar to Foursquare. They’re going to provide little nugget of those data services. This is the service that you tap in to tell the difference between a piece of bread and cheese. You tap into Foursquare to understand the place that you’re inside of. You tap into Facebook to do facial recognition on people.
AI will supercharge that.
Yeah, if the stuff works, right? This stuff is really, really hard to build — we have a hard time just building walking in and out of places — like all the facial recognition and object recognitions. But that’s where it gets really interesting.
So you might see a lot of this stuff happen in virtual reality first because it’s a hell of a lot easier. Virtual reality is a video game in front of your face. Augmented reality is, I’ve got to make sense of the shit that’s in front of you, and that’s where it gets kind of weird and interesting.
