Laravel Horizon & Earthquake Stream
When Taylor Otwell teased Horizon what feels like a year ago at Laracon Online I envisaged something unlikely to be something I needed. How wrong I was. Horizon is a queue management solution for Laravel; and ️I ❤ it!
https://earthquake.stream
I tweeted about it and got 20 thousand views just because of the quick screenshot; a timely post!
Stats
The key info taken from Earthquake Stream over the last few weeks are:
- Max 28,000 jobs
- Max ~2,000 jobs a minute process (22 total processes)
- 3 queues — poll, process & message
- About 2,500 unique earthquakes reported each day
- Average 400 jobs per hour
About Earthquake Stream
This app came from a hackathon at the start of 2017 hosted by Geovation Hub — CrisisHack. I built a MVP using full blown Laravel allowing spatial filtering and alerting in realtime of an earthquake occurring — a true first. This won the competition and I was approached by Geovation, IBM, and DSTL to invest in the idea and make a start up.
Since then I have been in talks and presented the idea to investors and the community; however I have yet to find the opportunity that allows me to jump feet first on this opportunity.
So when Horizon was released I endeavoured to provide at least something beyond the MVP and a half built Spark solution and create a web socket based earthquake data stream.
Why for free?
Two simple reasons:
- No one wants to pay for it yet…
- I can cover the costs right now so why charge for something so useful!?
- Did I say two?? This may lead me onto an opportunity to build something more
Horizon is so cool!
It really is cool; from the interface, the “it just works” feeling and the information you can gather immediately about the health of your app.
I use Laravel Forge for deployment assistance and would highly recommend it. Digital Ocean provide the hosting ($5 a month), NameCheap the domain name for 61 cents, and Pusher for free web socket messaging.
All this put together with 10 minutes of setup and trawling the documentation and I had Horizon working locally managing queues of earthquake events, processing the incoming data, and broadcasting to Pusher.
Queues, Redis & Documentation
The system is basically thee stages, each with a queue that passes something onto the next.
A Poll job is fired onto the Poll queue every 5–10 seconds to check the USGS Earthquake service (a Java app that spits out events into a SQLite DB) for any new events of interest and for every one it finds dispatches a Process job on the Process queue.
Processing effectively takes the earthquake ID, interrogates the USGS database for its info (magnitude, web link, significance, etc.). Once complete this class is setup to fire an event which has ShouldBroadcast configured.
The event is broadcast to Pusher, and depending on the severity and size of the earthquake gets pushed to several channels.
After the fact I PR’d some changes to the Horizon docs; much of which was based from reading Matt Stauffer’s blog post on the subject.
The future
Hopefully Earthquake Stream gets some traction and afford me the opportunity to go full time on it. We shall see! (PS. If you are interested do message me).
As for Horizon — any MVP / app I make that relies on queues will have Horizon included no doubt.
TL;DR
Horizon is great — check it out. It is a awesome solution to monitoring queues in a Laravel app.
I do need to thank Taylor Otwell, Adam Wathan, Matt Stauffer and Laracasts for all their help they provide to me and the community as a whole. Also special thanks to LaravelUK a Slack channel that is the ‘Brit’ home to Laravel.
I will post an update with the extra traffic usage in a follow up post.
If you liked it please do show me some love 👏
