Apache NiFi, Apache Kafka, Postgresql, Apache Flink, Apache Iceberg
— — Adding MTA SIRI inference, MTA Routes, RSS
— — Pennsylvania SEPTA
Philadelphia SEPTA Bus
https://www3.septa.org/api/pbtojson/Bus/Services/index.php
Output — SEPTA Bus Alerts to Kafka
{
"routeid":"2",
"descriptionlang":"","effect":"DETOUR",
"descriptiontext":"There is a detour at 17th & Pine starting on 3/4/2024 10:54 AM.","activeperiodend":"","cause":"CONSTRUCTION","stopid":"","alertlang":"","alerttext":"",
"activeperiodstart":"","maxlat":"",
"locationcountrycode":"US",
"maxlong":"",
"locationmunicipality":"Philadelphia","minlong":"","minlat":"",
"locationsubdivisionname":"SEPTA",
"ts":"1709587631661",
"uuid":"4d5cd6d6-cf3f-4175-aa37-7d8fde8db3d2",
"rundate":"Mon, 04 Mar 2024 21:09:48 GMT",
"providername":"SEPTA",
"transitname":"SEPTAPABUS"
}
SEPTA Train
https://www3.septa.org/api/TrainView/index.php
In this one we retrieve Train information.
OUTPUT — SEPTA Train to Kafka
{
"lat":"40.1891135",
"lon":"-74.892222166667",
"trainno":"5304",
"service":"LOCAL",
"dest":"West Trenton",
"currentstop":"Langhorne",
"nextstop":"Woodbourne",
"line":"West Trenton",
"consist":"157,158,453,454,409",
"heading":"22.191216920965",
"late":11,"SOURCE":"Overbrook",
"TRACK":"2N",
"TRACK_CHANGE":"",
"ts":"1709563210694",
"uuid":"7c334b00-6114-4729-ae41-696e16cd3322"
}
SEPTA Transit View All
https://www3.septa.org/api/TransitViewAll/index.php
OUTPUT SEPTA TRANSIT — To Kafka
[
{
"lat":"40.045227",
"lng":"-75.125603",
"label":"3354","VehicleID":"3354","route_id":"K",
"BlockID":"2616","Direction":"EastBound",
"destination":"Arrott Transportation Center",
"Offset":"19","heading":100,"late":19,"original_late":19,
"Offset_sec":"1122","trip":"775127","next_stop_id":"31349",
"next_stop_name":"Godfrey Av & 2nd St","next_stop_sequence":63,
"estimated_seat_availability":"FEW_SEATS_AVAILABLE",
"timestamp":1709590702,"ts":"1709590723846",
"uuid":"b4f951e6-d526-4a2c-95ec-d5bc03aa800f"
}
]
RESOURCES
SOURCE CODE