Geo-Fencing your Goats with a Cribl Pipeline

What

Based on an idea/question from a teammate (Thanks, Bojan) I implemented a way to “Geotag” events that contain GPS coordinates with Cribl Stream (or Edge). (NOT Geo IP Lookup, but might be combined with it.)

The Pipeline is able to identity if a position/location parsed from an event is within a defined area or not. (as an example).

Combined with other parts of Stream (just the Webhook Destination alone gives you millions of possibilities) you can realize a non/low-budget, real time Geo fence solution for your goat farm (just to give you the most common use case).

How

High Level Steps. Details below:

1. Create a Lookup with your Geo Fence Data and upload it to Stream
2. Parse the Lat/Lon coordinates from your events and check it against the Lookup.
3. Label/Tag the event based on the check (in area: yes/no)

How Exactly

Let’s go deeper.

Here is an example how you can create a csv Lookup based on an defined Area / Geo-Fence.

If the area is as simple as the example given, you can also skip the Earth part and just build your csv “manually” by hovering around in Google Maps and copy&paste the 4 locations around your Goat farm. (See last step how the CSV should look like)

To use Earth, follow these steps:

• Go to Google Earth
• Add -> Polygon
• Draw a Square around your desired to-be-monitored area. It can also be more complex, but each point will take away some CPU time later on…(*)

• Select the polygon and click “Save to project” (Blue button in screenshot) -> Local KML File
• It will show up in the left menu. From here you can actually download the KML file:

• Open the KML in a Text editor. The coordinates will be in the XML structure like this:

• Format the coordinates to CSV.
  For this simple example I would not create a script or Notepad macro. But with hundreds of coordinates, you will have to get creative. It’s even possible with Cribl, the KML is accepted by the XML Parse command.
  I let you deal with this on your own, but at the end you want something like this:

country,area,pos
nl,goatfarm,"4.82376446112908,52.31356709998632"
nl,goatfarm,"4.823791895483829,52.3126027225866"
(…)

• Store it as csv and create a new Lookup with it in Cribl. Or copy paste it directly in the UI Lookup editor. Check Cribl Docs for details if you are new to Lookups.
  (Country and Area is used in my examples as the key for the lookup command. You can use other fields based on the use case….)

Let’s look at the key element of the whole process, the logic to check if a point is within an area:

As with everything you do, somebody in the Internet already did it earlier:

https://www.geeksforgeeks.org/how-to-check-if-a-given-point-lies-inside-a-polygon/

Key take aways from the linked website:

Definition of the states (just to make things clear :P ):

Point f crosses 1 time (odd number) = Inside Area
Point b crosses 2 time (even number) = Outside Area
[I tried very complex polygons with pen&paper => it works :) ]

The website even has JS Code (that does not need any library) which implements this logic.

I basically just changed the input variables to be taken from the event and that the result is given back to the event. The other parts of the JS was more or less put as-is in a Code Function to the Pipeline:

(...)
let p = new Point( __e['lat'], __e['lon'] ); // input from event
(...)
if (checkInside(polygon, n, p))
    __e['IsInArea'] = true; // result back to event
else
  __e['IsInArea'] = false; // result back to event

In my PoC I used the Android App “Own Tracks” to send the location of my mobile on a regular basis. It produces a simple JSON with the current lat and lon coordinates.

Due to the JS Code, the same License as described here applies to the Pipeline/Pack

https://www.geeksforgeeks.org/legal/copyright-information/

Visualization in Grafana.

Red o: Goats outside of fence
Green x: Goats inside of fence
Purple — : Fence

The colors are based on field(s) provided by Cribl/the Pipeline.

It does not make use of any “geo logic”/calculations of Grafana (besides the background layers taken from OSM).

(*) A Pipeline referencing a Polygon with ~2500 Points took around ~1500ms per Event to be processed. => Bad for high volume streams.

Why

Advantages:

• Geo Fencing/Tagging with just Stream.
• No need to call (and pay) an third party Map API.
• Also no need to talk to Legal/Sec if you may send the data to third party APIs.
• For offline environments.
• Calculate once on (before) Ingest…No need to run the same Lookups 1000s of times in your analytics platform.

Use cases Brainstorm:

• Is the robot driving around where it should be?
• Related: How many “robots” (or whatever) are in each area? Is the count balanced? (Service Robots, Robots in storages…)
• Are all of our Fleets Car in Countries that are insuranced?
• Geo Fence (notify yourself if object leaves/enters area):
• Bike
• Cat
• Kids, Wife, Husband :)
• Car
• Goats

Limitations

• Can currently only check 1 area at a time. The setup needs a Lookup and a separate Lookup/Code Function Group for each area.
• Goal is to have one CSV with several areas and have Cribl check against all of them. (Return on/out for each of the areas. Will be added to a future version of the Pack.
• Can be slow if Polygon is complex. (*)

Summary

• It works, tested with up to ~2500 Point Polygons…
• Has Limitations, or let’s call it Potential for Optimization

Pack

A Pack with Sample- Files and Lookups, and the Pipeline has been submitted to the Cribl Packs Dispensary.

Should be available soon:
https://packs.cribl.io/?query=geofencing