Democratizing Autonomous Vehicle Data

Introducing the real-time, street-level intelligence platform with a mission to automate cities and make them work better

TL;DR — CARMERA launches today in New York City. We operate a visual road sensor network built on top of safety monitoring services for professional fleets, to gather updated 3D scene, change detection and analytics data for city streets. Our machine vision pipelines extract rich texture and insight for autonomous vehicle mapping, as well as a broader set of built environment uses not served until now. Try it at carmera.com.

If you’ve lived in developing countries, you don’t take for granted the importance of roads. They are the central nervous system of the physical world. As soon as the government lays down a new one, you almost see overnight how life starts to spring up around it. People start moving in, businesses set up shop — eventually entirely new communities emerge.
 
Since early civilization — before the rise of the Internet or any communications infrastructure — roads, streets and highways made up the original network of real life, connecting your house to the blacksmith, allowing a message to be relayed to the next town, enabling armies to march. (There was a reason the Romans made sure all roads led back to them.)

Fast forward to 2017 — our roads are in the midst of a renaissance. Infrastructure investment is about the only issue that all political camps agree on, and driverless cars are poised to dramatically improve safety, congestion, emissions, land use, productivity, even worker happiness.

And you may now hear tech or auto industry people saying “data is the new oil,” as next-generation mapping data for roads has become critical in allowing autonomous vehicles to confirm what they think they see, and anticipate what they can’t yet see.

One of the first images of Google’s live, 3D map for autonomous vehicles

The problem is, autonomous vehicle mapping data tends to be non-existent, not good enough, or what little exists gets locked up in a few closed platforms.

We built CARMERA to fix this. We prefer a world in which the enabling technology for autonomy benefits many, and not years or decades into the future. Put another way, if achieving autonomous mobility everywhere is our generation’s getting to the moon…we are very excited for that giant leap…but in the meantime, we want to give people access to all that cool new space technology right now.

Built off our first-party and professionally crowdsourced sensor network, CARMERA is a real-time, street-level intelligence platform for autonomous vehicles and the built environment.

Our flagship offering, CARMERA Autonomous Map, provides real-time 3D maps and navigation-critical data for autonomous vehicles.

Our first product, CARMERA Site Intelligence, provides comprehensive 3D reconstruction and site analytics data on-demand, accessible to anyone, anywhere. Whether helping architects model construction projects, real estate brokers determine foot traffic trends, or urban planners track streetscape changes, having instant access to this data versus relying on repeated site visits or slow, costly surveys is game-changing. If you work in the built environment, or are interested for any other reason, you can try it yourself.

CARMERA Site Intelligence colorized, cm-accurate 3D point cloud — West Village, NYC

Roots
When we began our journey years ago, there was just a dull buzz — and a lot of skepticism — about the potential for autonomous vehicles. We heard from companies working on AV that 3D, machine-readable street maps were becoming increasingly important in solving the problem, yet everyone had to create their own. No independent platforms offered this data at the fidelity, scale and freshness required. Today, while we see encouraging signs, this is still largely the case as we talk to big OEMs, growing mobility platforms, emerging startups and others working on all parts of the AV stack. So we started CARMERA in 2015 to meet this problem head on, beginning in one of the most challenging environments of all — New York City.

CARMERA’s first panoramic LiDAR mapping vehicle, “CARMERA Force One”

Our collective experience in leading consumer product teams — from early video and gaming platforms like Blip.tv and Guitar Hero, to desktop 3D printing at MakerBot, to mobile-first maps at Apple and Amazon — gave us an appreciation for putting high-impact tech in as many hands as possible. 
 
And we’d long understood the seismic potential of autonomous mobility, from the late 1990s in Princeton’s Civil Engineering department, where we where we first modeled autonomous transportation concepts like Personal Rapid Transit, already decades old then. Little did we know it would take another 25 years for those concepts to finally take root in the form of shared, electric, autonomous systems we are seeing today (thank you, Moore’s Law, GNSS, DARPA, the Internet, mobile, GPUs, et al.).

A Personal Rapid Transit system proposal for Austin, TX, using a classic monorail approach. (Turns out the rail was there the whole time — it was just the road). [Credit: PRT USA]

[Data] is Eating the [Built] World
It’s a well-worn phrase now that ‘software is eating the world,’ but for those of us around for earlier phases of the S-curve we’re in, it’s surreal to see it all happening at such an accelerated pace. We’ve known for a while that atoms tend to become bits when possible, but what we’re witnessing everyday in the built environment — particularly in cities — is feeling more real than ever.

It makes sense based on what happened first on the Internet. Digital environments are based on network architectures, and the first breakthrough in making the net widely useful was crawling and indexing that network.

For most of us, the nodes of that network (media sites, web services, APIs, etc.) matter more than the connections (edges) between the nodes. But as noted above, roads are the edges of the original/IRL network. And in cities, information about what’s happening on those edges at street-level is critical, whether for physical infrastructure, commercial activity, population movements, or just about anything else.

There are, in fact, many questions about what’s happening on streets that have historically entailed highly manual processes to answer, or are just left unanswered altogether:

  • An architect designing: “Could this facade really work on that block?”
  • A retail broker assessing: “What’s foot traffic like in K-Town at lunch?”
  • An insurer verifying: “Has that building complied with ADA?”
  • A local POI firm tracking: “How many grand opening signs in Noho?
  • A zoning official quantifying: “Are Brownsville blocks becoming greener?”
  • An analyst predicting: “Will we see more luxury SUVs in Astoria?”
  • And, soon, a robot navigating: “How do I avoid that construction zone?

It is incredibly energizing to think we can now answer many of these questions instantly, almost like using a search engine or cloud API…

…and in cities, we will have to.

Urban centers already account for outsize portions of consumption. By mid-century, 2/3 of the world’s population and 4/5 of GDP will become concentrated in cities. To accommodate those shifts, we have no choice but to truly deliver on the promise of “smart cities” — and soon.

And just as we’ve seen with software and data permeating everything…the trends we’re seeing in the new generation of built environment technology (e.g. 3D reality capture, iterative design, collaborative project management, immersive visualization, etc.) are those that shaped computing over the past several decades — trends we wholly embrace at CARMERA:

  • On-Prem → Cloud
  • Custom → Platform
  • Closed → Accessible
  • Terrestrial → Mobile
  • Expensive → Affordable
  • Gatekeepers → Permissionless
  • Enterprise UX → Consumer UX …

Crawl & Index → Liberate & Democratize
So if digitization continues its march into the real world, and roads are the connective foundation for the ‘IRL web’, it begs the question: can we really crawl and index roads like we do the web?

We believe we are in a once-in-a-generation convergence of enabling technology to make this concept even thinkable. All those buzzwords you hear these days: IoT, Big Data, Computer Vision, Neural Networks…they’re real. And they’re only getting better, faster and cheaper everyday in allowing us to treat the real world like the digital world.

In CARMERA’s case, our symbiotic safety monitoring partnerships with high-coverage fleets that serve cities, make use of mobile sensor, 3D printing and vehicle telematics technology that would have been cost or size prohibitive, just a few years ago.

A CARMERA “Swarm” fleet partner vehicle with roof-mounted sensors, crawling city streets for visual change detection
CARMERA’s daily to monthly street-level coverage of NYC

In addition, CARMERA’s 2D and 3D data processing pipelines make use of data management, GPU and machine learning infrastructure that either didn’t exist or would have been too cumbersome at scale, until very recently.

2D feature extraction such as pedestrian, vehicle, street furniture detection
3D feature extraction such as roadbed, curb, sidewalk, facade detection

Thankfully, we do have the right tools today, or we create them ourselves when we need to. All of this lets us pay off an ethos that’s been central to CARMERA since the beginning:

<Liberate and democratize powerful information, responsibly and sustainably, instead of limiting to just the hands of a few.>

History has shown good things tend to happen when you do that, and we know that to be true for real-time, street-level intelligence in cities.

Join Us
If you’re still reading — :namaste emoji:

Sign up for our mailing list, and follow us here and on Twitter. We’ll give you first look at new things coming down the path, like upcoming city expansion announcements, product releases, job openings, events and more…and we’d love to hear your thoughts about how we can make CARMERA valuable for you.

Thank you, sincerely, for riding along with us on our journey. We’ll keep our eyes on the road, but feel free to backseat drive, while you still can.