Location Analysis Simplified In Cloud: How to analyze location-based data and produce publishing-ready geographic data using GeoBuffer

This is what most of us start with when we do any kind of location analysis. Lots of disconnected files, zero insight at the beginning.

The Problem

Regardless if you’re a policymaker, realtor, urban planner, journalist or a researcher, information-based decision making or reporting is an integral part of your job and all chances are that you deal with location analysis on a regular basis.

Even though location analysis comes in many forms and varies in inputs and outputs, one fact stays the same across all occupations and use-cases related to it: sooner or later you have to show the data on an actual map.

Thus, in order to do location analysis, you have to solve the following issues:

• find the public or proprietary data on which to base your analysis
• find the public or proprietary geographic shape data on which to base your visualization
• store the data somewhere — your own PC, cloud storage solutions such as DropBox, or FTP servers
• perform data operations necessary to assemble the desired output
• properly link or connect the resulting data to geographic shapes which you’ll eventually show on a map

Powerful specialized desktop-based applications are the de-facto standard for previewing and modifying geographies and their connected datasets. One considers themselves lucky if they find any kind of visual indicator of what geography looks like when searching the web for data sources. Data is then stored locally which makes teamwork unreasonably cumbersome.

At Social Explorer Inc. we have made it our mission to make location data analysis easy, using our unique expertise and years of experience in data science, data management, and data visualization design.

Our first product socialexplorer.com, was a web application that enabled you to explore our augmented and organized US Census datasets (including historical data) and design your custom report maps directly from the comfort of your browser with no IT or GIS-related expertise required.

Example of a static map image created in Social Explorer with only a couple of clicks: percentage of homes in Bay Area that cost more than $750k displayed on a block level. It took more steps to create a static image from this interactive visualization than to embed it to Medium directly.

Since we first launched that product, we have made hundreds of improvements, billions of data points, workspace capabilities etc.

That said, there has been one recurring need from our customers that have remained unmet: the desire to upload your own data and get a high quality, interactive map?

While we couldn’t immediately meet this demand for our users, we began developing improvements in our own internal workflow for US Census data management and processing.

For years, the data was stored on our FTP servers, processed by scripts and manually managed. It was easy to make mistakes, delete entire datasets and finding the exact dataset took a naming convention and minutes of file browsing. Creating base maps was hard as we had to release the entire application before seeing changes. If we made a mistake, we had to do the entire process all over again.

Ultimately, we decided to build our own GIS system that would power socialexplorer.com and simplify our own workflow enabling us to expand our data publishing operations. After we achieved our internal goals, we decided to start making its features more user-friendly and finally on December 17th, 2018 we released the beta version of geobuffer.com.

In this example scenario, we want to see how many noise-related 311-reported incidents occurred within each New York City block. To get this information, we will:

Upload our New York City blocks geography and NYPD’s 311 calls CSV dataset to the GeoBuffer’s cloud, making it instantly available for all our team members from any place, at any time. Upload analyzer will take care of the upload experience by immediately reporting any problems with our files before the upload even starts!

Uploading geospatial information has never been easier. Support for various storage formats with pre-flight upload analysis makes sure the process is easy and comprehensible regardless of your GIS proficiency level.

Preview uploaded data directly from the browser. Any malformations in the geography or incorrect data can be easily spotted this way.

Once the upload is complete, your files are immediately converted into datasets you can now manage through GeoBuffer. Geographies can immediately be previewed and each feature inspected for associated data. By using the built-in satellite overlay, you can make sure your geographies are in the correct format and compatible with the widely used geographic projection system.

Use the address data provided in the 311 calls dataset to convert it into geographic dataset compatible with the New York City blocks geography by using our proprietary, built-in and seamlessly integrated geocoder

The Geocoding feature will use values composed from selected address information columns to determine latitude and longitude for each dataset row. The output will also include normalized result addresses, which you should have a level of confidence hit the correct location.

The Geocode operation automatically converts the source dataset into geography consisting of one point for each geocoded address. You can use satellite view to check for any addresses that were not correctly recognized to be within NYC, or you can use the “confidence” column and sort or filter by it.

Create a new column in NYC blocks geography dataset that will contain the number of noise-related incidents by performing a spatial join, which is a GIS operation that will produce a count of incident address points contained within borders of each NYC block.

Spatial join is a technical term for an operation that is used for counting points from source dataset geographically contained within each polygon from target dataset.

Preview the results directly in the browser using label displays feature

After a successful spatial join operation, your original geography now contains an additional column with a number of noise-related complaints in each NYC block. Displaying label columns directly on the map preview enables you to quickly analyze results.

Once you are happy with the resulting data, you can download the modified shapefile for offline use if necessary. We support various formats for exporting geographies: SHP, KML, GeoJSON, or we can trim the geography and export the associated data as a plain CSV file.

Data export allows you to download specified datasets in any supported format. In this case, you want the output to be our NYC block geography with a newly added noise-complaint-count column.

Shape of things to come

At the moment, GeoBuffer can be used for cloud storage of your geographies, basic geospatial and data transformation/editing operations and geocoding. We are continuing our work on smoothing out the rough edges and we’re in the process of building additional tooling that will allow you to publish maps by using GeoBuffer as vector and raster tile source. We hope to deliver good news on this front within months.

If you want to help us shape the future of GeoBuffer, there’s a free 14-day trial with promotional early-bird subscription fees available at geobuffer.com. If you have any questions or would just like to chat with us, feel free to contact us at feedback@socialexplorer.com.

Thank you for reading!