Week 1 — Sorting Google Street View
Google Street View is an amazing archive of the city but has yet to be easily sortable. If we want to see all locations that are marked as historic in New York City, we would need to look up each location from a database of addresses copy the address into Google Maps, drop the pegman into each location, screenshot each street scene, and then repeat the steps for each location before being able to compare them all.
Artists like Josh Begley have found smarter ways to sample Google Street View. He uses Google’s API and custom scripts to automate the downloading of street view from various locations. In “Officer Involved”, he uses databases of police brutality (collected by non-governmental and news organizations) to sample Street View scenes at the location of each incident, thus immersing us in “the environment of someone’s last moment”.
We’re going to use a similar technique using a Python script to download Street View scenes from a dataset of your choice.
Data Selection
To get Street View images, we’ll first need to decide what locations we want to get Street View images from. All we need for our script is a list of latitude and longitude locations. There are a number of ways to get lat/long lists such as scraping websites or downloading data files from sources like NYC Open Data (which we’ll cover in another class). If you’re comfortable with Google Sheets and NYC Open Data, feel free to create a list of lat/long locations from that source. I’d recommend keeping the list under 100 rows.
For this exercise, we’ll make our own lat/long list.
1 Go to Google Maps and query something of interest such as: “arcade”, “church”, “family dollar stores”, “starbucks”, “historical landmarks”, “school”, “hardware”, or whatever else interests you. Your results will show up as text in the left bar. Select the text in the first page of results and copy and paste it into a new Google Spreadsheet. This technique is called “web scraping,” and in this case, we’re doing it manually.
Pre-Processing Data & Feature Selection
You’ll notice this list is pretty messy. There are names, ratings, addresses and open hours. We only want latitude and longitude. All the other information for our purposes is irrelevant. So lets clean up our list so that it’s just addresses. To do so, we’re going to perform some spreadsheet operations to combine the separate address lines into single lines.
2 In the top menu bar select “Data” > “Sort Sheet by column A-Z”. Delete every row without an address. (Cmd click or Ctrl click to select multiple rows). Right click to delete.
3 Highlight the column with your data, then go to menu “Data” > “Split text to columns…” A dropdown menu will appear on your screen. Change it from “Comma” to “Space”. Delete all non-address portions out of your data (highlight the cells you want to delete > right click > “delete” > “shift rows left”).
Now that we have address snippets, we need to string our data back together, and add in the city and state.
4 Copy and paste the text from below into the first empty cell at the top right of your data and hit enter. If you have more or less than columns A, B, C and D, you’ll need to change this formula accordingly.
=CONCATENATE(A1," ",B1," ",C1," ",D1, " New York, New York")5 Hover over the bottom right of this cell until a plus sign appears. Click and drag the “+” onto the cells below to use this formula on all of your addresses.
7 Next we’re going to turn these addresses into latitude and longitude values using a batch geocoder. Copy your addresses from Google Sheets then paste them into the batch geocode textbox labeled “Addresses”. On the righthand of the website, Select the following:
- “Addresses are in: United States”
- “Separate text output with: Tabs (for direct pasting into Excel)”
- “Include these columns in text output: ”
8 Scroll down and you will see the map populate as it turns each address back into marker points. When it is done it will say complete. Where the tab says “Map” select the “Text” tab. The points have been made into Latitude Longitude coordinates. Highlight and copy all of the Latitude Longitude text and paste them into a new Google Sheet.
Now lets download our files as a CSV file. A CSV (comma-separated values) is a simple file format used to store tabular data, such as from a spreadsheet or database.
9In your new Google Sheet go to menu “File” > “Download as” > “Comma-separated values”. Rename your CSV file to “locations.csv”.
Google’s Street View API
An API (application program interface) is a set of routines, protocols, and tools for building software applications that integrate with information from websites. Google’s API allows us to access a wealth of information such as getting Street View images. APIs use structured language so that we (the programmer and Google) both know we’re talking about the same thing. For example to get a street location, we would write code with specific names listed on the API such as:
size: 600x300
location: 46.414382,10.013988
heading: 151.78
pitch: -0.76
key: API_KEYHowever, in order to get information from an API, we’ll first need an API key. An API key is a code passed in by computer programs calling an API to identify the calling program, its developer, or its user to the website. API keys are used to track and control how the API is being used.
10 Go to Streetview API. Click the button “GET A KEY”. Sign in with your Google account; your Columbia email will work. Follow the prompts to enable Google Street View Image API. Copy your key and save it somewhere temporarily. Don’t give out your key!
Setting up Processing (Data Mining)
Processing is a flexible software sketchbook and a language for learning how to code within the context of the visual arts. We’ll use it with a programming language called Python. Python is commonly used in data science and can integrate with many other applications including 3D modeling applications like Rhino, which makes it a particularly useful language to learn for designers. It’s also a great language to learn if you intend to work in tech. Processing doesn’t use Python by default when you first install it, so you’ll need to follow these steps to enable Python Mode.
11 Download Processing. Place Processing in the Applications folder on your computer (or Program Files for Windows). When you double click Processing, it should install and launch the application with a new empty sketch. A sketch is where you will write your code.
12Once you’re in Processing, select the “Java” drop down in the upper right corner and select “Add Mode”. Select “Python Mode for Processing 3” and click “Install”. Once it has installed, close the window and change the “Java” drop down to “Python”. Now you’re in Python Mode, and ready to write Processing sketches using Python.
To get Street View images of your locations, we’ll use a Python script in our Processing sketch which sends your locations to Google’s API and automatically downloads each image. We’ll use a script developed by Allan William Martin.
13 Copy this code into an empty Processing sketch and SAVE the file in a permanent location. Don’t save in downloads. You must save the file before proceeding.
14 Next you will create a new tab and name it locationscsv.py . SAVE the file. It should automatically save inside the your google street view project folder. Notice .py means this is a Python file, whereas our main file is .pyde which is a Python-Processing file.
*This is a new tutorial version with locations saved as a Python file which is adapted to work with Windows.
15 Open your locations.csv file with all of your latitude and longitude locations with a Text Editor. (Right click > Open With > Text Edit). You can use any text editor on your computer but don’t open the file in Excel. Copy all of the lat long text. In Processing in the “locationscsv.py” file, paste your lat long text in between the triple quotes. Save the file again!
16 Copy your API key and paste it in your main Google Street View code between the quotes where it says api_key = "".
17 Make sure you’re connected to the Internet. From Processing hit the play button to run the script.
This will save north, south, east, and west Street View images from each of your latitude/longitude locations. This script will work on much longer lists of locations, but keep in mind that Google may block you if you try and call too many times. The images will be saved into your Processing folder as shown below.
Post to Medium
18 Go to Medium and sign up for an account (its free).
19 Click your profile image at the top right of the page and select “New Story”.
20 Change the title and write as little or as much text as you like.
21 Upload some of your Street View images by clicking the “+” and selecting the camera icon, then navigating to your sketch folder. Post as many or as few of your images as you like.
22 When you’re ready to publish, click the green “Publish” dropdown at the top of the page. Type in the tag “Research” and then hit “Publish”. This will ensure that your post is submitted.
During our next class we’ll go over how to submit to the class publication.
