Geo-GraphQL with ElasticSearch

Duane Bester
Dec 2 · 7 min read

A Tutorial

Photo by chuttersnap on Unsplash

We recently had our 2nd hackathon at RigUp, and my team took first place in the customer category! Our solution involved custom scrapers, queues, data ETL, and querying of the resulting data. We then wrote a React app to provide slick visualizations to display this data. 📈 As we only had 48 hours, our code was not production ready.

This tutorial explores my approach to improving the project in my free time. I decided to write a GraphQL server to access our ElasticSearch geo data. My language of choice is Scala, leveraging the Akka Http and Sangria libraries. It should be noted that RigUp isn’t a Scala house, just that Scala is a big part of my background.

For our hackathon, we had multiple ElasticSearch indexes for different types of documents, but each document type had a latitude and longitude. This required us to make multiple API requests to ElasticSearch, which means duplicating a lot of search logic. Since we were viewing data on a map, we leveraged Elastic’s Geo Bounding Box Queries.

// Pseudocode
fetch('/users', { geo_bounding_box, ...some_filter_here });
fetch('/coffee-shops', { geo_bounding_box, ...other_filter });

Ideally, we’d like to create a succinct GQL query where the geo bounding box would propagate to the nested queries. It should be easy to add additional models under geoSearch that would inherit the bounding box variable:

users and coffeeShops can access bbox variable

The $bbox reference will point to the “bbox” object that we define in the Query Variables pane of graphiql. We filter users that are outside the bounds of this box.

“bbox” variable as an object in the bottom Query Variables pane

The following takes heavy inspiration from the Sangria docs as well as the HowToGraphQL Scala tutorial. We will create GraphQL queries to filter users and coffee shops based on name and geolocation.

Project Setup

ElasticSearch

Running docker-compose up -d should get you started. Make sure you can get a response from ElasticSearch in your browser at localhost:9200

To load our test users and coffee shops, change directory into the scripts folder and run the load script:

$ cd src/main/resources/scripts/
$ ./load-test-data.sh

Navigating to http://localhost:9200/test-users/_search?pretty should give you a list of users.

Dependencies

Resources

Main Server

Models

Let’s add a models folder with nested directories and classes as follows:

    ├── Main.scala
└── models
├── common
│ └── Location.scala
├── responses
│ ├── CoffeeShop.scala
│ ├── SearchResponse.scala
│ └── User.scala
└── variables
├── BBox.scala
└── Filter.scala

Common

Variables

Responses

A User case class will have name, id, and location properties. The location property will be an object of type Location.

Our UsersResponse case class will contain a total property and a list of users.

The CoffeeShop and CoffeeShopResponse models are very similar to the User and User Response ones:

Core Code

├── Elastic.scala
├── GraphQLSchema.scala
├── GraphQLServer.scala
├── Main.scala
└── models

GraphQL Schema

Here we tie our models to a GraphQL Schema:

ElasticSearch

Next we will add a class that implements this trait and the methods we have defined:

Sangria has a concept of Context that flows with GraphQL Queries. It’s super important for our use case as it will hold an instance of our Elastic class and the BBox variable.

We can define our context as a simple case class in GraphQLSchema.scala:

case class MyContext(elastic: Elastic, bbox: Option[BBox] = None)

GraphQL Server

You’ll see that we make a call to executeGraphQLQuery. Let’s build that next:

Here is where we pass our Elastic instance as well as our previously defined GraphQLSchema.SchemaDefinition. Let’s not forget to update Main.scala to route requests to our GraphQLServer:

Testing our setup

$ sbt ~reStart

Navigate to localhost:8080 and enter our query and bbox variable:

When we search with the following bounding box coordinates, we will see 3 users returned from the query. All of these users are located in Austin, TX.

{
"data": {
"geoSearch": {
"users": {
"hits": [
{
"id": 3,
"name": "Ricky",
"location": ...
},
{
"id": 4,
"name": "Carter",
"location": ...
},
{
"id": 5,
"name": "Mitch",
"location": ...
}
]
}
}
}
}

Let’s adjust our bounding box to cover more area:

We see a 4th user, who happens to be in San Diego, CA:

{
"id": 1,
"name": "Duane",
"location": {
"lat": "32.715736",
"lon": "-117.161087"
}
}

We adjust our bounding geo box a last time:

And we see our 5th and final user in Mexico City 🇲🇽

{
"name": "Matt",
"id": 2,
"location": {
"lat": "19.42847",
"lon": "-99.12766"
}
}

Extending our setup

In order to filter by name, let’s update the buildQuery method in Elastic.scala:

Now if we run the new query from above, we will see only Matt and Mitch returned! Super easy to add in new functionality.

{
"data": {
"geoSearch": {
"users": {
"hits": [
{
"id": 2,
"name": "Matt",
"location": {
"lat": "19.42847",
"lon": "-99.12766"
}
},
{
"id": 5,
"name": "Mitch",
"location": {
"lat": "30.366666",
"lon": "-97.833330"
}
}
],
"total": 2
}
}
}
}

Finally, let’s say we want to search for users in our geo box with names that start with “M” as well as coffee shops in the area with the name “Starbucks”

{
"data": {
"geoSearch": {
"users": {
"hits": [
{
"id": 2,
"name": "Matt",
"location": {
"lat": "19.42847",
"lon": "-99.12766"
}
},
{
"id": 5,
"name": "Mitch",
"location": {
"lat": "30.366666",
"lon": "-97.833330"
}
}
],
"total": 2
},
"coffeeShops": {
"hits": [
{
"name": "Starbucks"
},
{
"name": "Starbucks"
}
],
"total": 2
}
}
}
}

We can quickly wire up a simple React app with Mapbox and Apollo to display some of our data (source code):

Display our users within the map’s bounding box
npx create-react-app gql-elastic-app
cd gql-elastic-app/
npm install apollo-boost @apollo/react-hooks graphql
npm install react-mapbox-gl mapbox-gl --save

Let’s start our UserFeatures component that will take our users and map them to Mapbox features:

Now we add the GraphQL query string:

Let’s setup Mapbox within our App component:

Now for Apollo:

Next we need to add some state for the Map’s bounding box:

A quick utility function to convert Mapbox’s Bounds object to our own BBox object:

Lastly, we add two handlers that take map events and update our local state:

Running the app will zoom you into Austin, TX just like the GIF above, and will display 3 users. If we move the map to San Diego, we see our 4th user:

You should see our 5th user in Mexico City as well! If you have been following along, you would have noticed a CORS error. This is an easy fix and has been addressed in Main.scala in the repo.

Hope you have enjoyed geo-searching with GraphQL & ElasticSearch! All of the code is open sourced. 😎 PRs, comments, etc are welcomed!

Scala backend code — https://github.com/duanebester/gql-elastic-scala

React frontend code — https://github.com/duanebester/gql-elastic-app

Building RigUp

The RigUp Product & Engineering Blog

Thanks to Candace Collins

Duane Bester

Written by

Adventurist // Engineer // Photographer

Building RigUp

The RigUp Product & Engineering Blog

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade