How to get statistics from OpenStreetMap data

A guide to parsing and analysing OSM data using Python and Osmium

Introduction

OpenStreetMap (OSM) is a collaborative project to create a free and editable map of the world. OSM data is available in various formats, such as XML, JSON, and PBF. PBF is a binary format that is more compact and faster to process than XML or JSON. In this blog post, we will show you how to get statistics from OSM data files using Python and Osmium, a library for working with OSM data in various languages.

The main goal of the project is to extract meaningful statistics from OSM data files, such as the number and types of nodes, ways, and relations, the distribution of tags, the length and area of ways, and so on. These statistics can help us understand the characteristics and quality of OSM data, as well as identify potential issues or errors.

Process & Code Walkthrough

The first step is to obtain the PBF file, which can be downloaded from various sources such as Geofabrik or BBBike or acquired using the following steps.

1. Get the GeoJSON first from geojson.io | powered by Mapbox
2. Next generate the PBF file from any service available or open-source library like osmium or Tippecanoe. Gaussian Solutions — GTFS, OSM, Maps, Cloud also has utilities that provide for pbf cutting.
3. Save the PBF and run it through the below analyser code.

We will use the PBF file for the city of Seattle, Washington State USA, as an example. The file contains over 8 million elements.

Pre-requisites

1. Python Environment
2. Pip Packages —

Pyosmium

networkx

FastAPI & others — if you want to serve the data via API.

MathPlotLib for graph rendering if required.

We first count the total number of elements in the OSM file using a customized Osmium Simple Handler class called ElementCounter. The osmium.SimpleHandler is a class that defines methods for handling different types of OSM elements, such as nodes, ways, and relations. We can override these methods to implement our own logic for processing the elements. In this case, we simply increment a counter for each element type.

We then initialize another customized Osmium Simple Handler class called StatsAnalyzer with the total number of elements from the above and run it on the same PBF file again. (More efficient to run twice than calculating in one go) This class is responsible for parsing the PBF file for gathering statistics about the data. We also declare a couple of Enum classes to define the different types of ways and nodes that are present in the OSM file for which we need the statistics. For example, we define a WayType Enum class with values such as HIGHWAY, FOOTWAY, SIDEWALK, etc. We use these Enums to classify the ways and nodes based on their tags and attributes.

Code Walkthrough

We override the way from osmium.SimpleHandler explained below

1. Calculate the distance of the way using the osmium.geom.haversine_distance function and adds it to the total distance of ways.
2. Iterates over the tags of the way. If a tag key is HIGHWAY, it calculates the distance of the way and adds it to the total distance of highways.
3. Depending on the type of the way (sidewalk, footway, or pedestrian), it adds the nodes of the way to either the sidewalk graph or the highway graph.
4. We use the networkx python library for adding the nodes to the graph.

To calculate the Intersections, we use the graph from the above

1. From this graph, we calculate the intersections for Highways and Sidewalks/ Footways.
2. We use the degree of the node in the graph and check if the degree is greater than 2 we add it to our stats library.

We also need to keep a tab on new tag values encountered and not in the Enum, it prints to a console a message indicating that the Way Types need to be updated. Based on need we can then just update the Enum to get the new tag statistics.

Apart from the above we also maintain counters for nodes, ways, relations, kerbs & crossings.

Once the processing is complete, we return a JSON file containing the following statistics:

1. The number of nodes in the OSM file.
2. The number of ways in the OSM file.
3. The distances of each Way type available in the OSM file. For example, the total distances of highways in the map area, or the total sidewalks / footways in the area in meters.
4. The total kerbs count in the file.
5. The total crossings count in the file.
6. The intersections with highways. The intersections are classified based on the number of highways at the intersection.

Below is the sample JSON file for the Albany County NY area — geojson.io | powered by Mapbox and the Analytics response for this area follows.

GeoJSON used for Albany County

{"type":"FeatureCollection","features":[{"type":"Feature",
"geometry":{"coordinates":[[[-74.106499,42.739235],[-74.106452,42.739241],[-74.106399,42.739248],[-74.101712,42.739808],[-74.08957,42.741234],[-74.067655,42.74388],[-74.060957,42.744689],[-74.057836,42.74506],[-74.054586,42.745449],[-74.0544,42.745471],[-74.051305,42.745749],[-74.051279,42.745744],[-74.050986,42.745789],[-74.05038,42.745859],[-74.049781,42.745944],[-74.047589,42.746174],[-74.047365,42.746204],[-74.047194,42.746234],[-74.047153,42.746239],[-74.046619,42.746299],[-74.044289,42.74668],[-74.042741,42.746862],[-74.038517,42.747358],[-74.038374,42.747375],[-74.029904,42.748447],[-74.029667,42.748478],[-74.0249,42.749065],[-74.024554,42.749108],[-74.024209,42.74915],[-74.019692,42.749706],[-74.015412,42.750233],[-74.013884,42.750431],[-74.011414,42.750755],[-74.001903,42.751979],[-73.994026,42.752992],[-73.993898,42.753008],[-73.993862,42.753013],[-73.992929,42.75314],[-73.985361,42.754166],[-73.982416,42.754566],[-73.978269,42.755127],[-73.977962,42.755168],[-73.977655,42.75521],[-73.976255,42.7554],[-73.976212,42.755406],[-73.976145,42.755416],[-73.976079,42.755424],[-73.975846,42.755456],[-73.975612,42.755487],[-73.971295,42.756073],[-73.967018,42.756654],[-73.960278,42.757566],[-73.960228,42.757572],[-73.955186,42.75812],[-73.95073,42.758606],[-73.944785,42.759423],[-73.944668,42.759435],[-73.94447,42.759451],[-73.943545,42.759541],[-73.942965,42.759576],[-73.942683,42.759648],[-73.942363,42.759678],[-73.94179,42.759677],[-73.941682,42.759677],[-73.94037,42.759774],[-73.940205,42.759789],[-73.940108,42.759798],[-73.938082,42.759988],[-73.933967,42.760676],[-73.933755,42.7607],[-73.93235,42.760855],[-73.931912,42.760906],[-73.931131,42.761003],[-73.930745,42.761047],[-73.929994,42.761135],[-73.916949,42.762368],[-73.911747,42.763096],[-73.911688,42.763104],[-73.90928,42.763116],[-73.906785,42.763432],[-73.899621,42.764249],[-73.898451,42.764429],[-73.898205,42.764467],[-73.897854,42.76451],[-73.8947,42.764898],[-73.892862,42.765124],[-73.891497,42.76526],[-73.889825,42.765461],[-73.889356,42.76555],[-73.888887,42.76564],[-73.888297,42.765752],[-73.887723,42.765858],[-73.88421,42.766179],[-73.883951,42.766203],[-73.883101,42.76628],[-73.882841,42.766304],[-73.881196,42.766455],[-73.879877,42.766616],[-73.87929,42.766687],[-73.875713,42.767127],[-73.874715,42.76727],[-73.874688,42.767274],[-73.873692,42.767403],[-73.873233,42.767463],[-73.872614,42.767544],[-73.871834,42.767573],[-73.871294,42.767494],[-73.868295,42.767973],[-73.862365,42.768706],[-73.859157,42.769089],[-73.858234,42.769198],[-73.858162,42.76921],[-73.852189,42.76992],[-73.842284,42.771107],[-73.840189,42.771355],[-73.838058,42.771613],[-73.834413,42.772047],[-73.832618,42.772263],[-73.82815,42.772798],[-73.827085,42.772926],[-73.825127,42.773163],[-73.824797,42.773203],[-73.824516,42.773237],[-73.819308,42.773855],[-73.816529,42.774188],[-73.816424,42.774201],[-73.813281,42.774576],[-73.811946,42.774736],[-73.811626,42.774774],[-73.809603,42.775016],[-73.809559,42.775784],[-73.809517,42.776509],[-73.809503,42.776703],[-73.809489,42.776897],[-73.809484,42.776981],[-73.809479,42.777064],[-73.809462,42.777344],[-73.809441,42.777685],[-73.809369,42.778869],[-73.799464,42.778878],[-73.799399,42.778903],[-73.799297,42.778944],[-73.797965,42.779477],[-73.793565,42.782577],[-73.790563,42.785942],[-73.790265,42.786277],[-73.787065,42.789677],[-73.782464,42.791777],[-73.779964,42.792177],[-73.777364,42.791777],[-73.769564,42.786277],[-73.768764,42.785877],[-73.767764,42.785977],[-73.766664,42.786077],[-73.764865,42.787277],[-73.763465,42.788576],[-73.761496,42.791579],[-73.761433,42.791675],[-73.76137,42.791771],[-73.758466,42.795276],[-73.752465,42.799676],[-73.750165,42.801376],[-73.748365,42.805276],[-73.737663,42.818877],[-73.736963,42.819477],[-73.731669,42.821305],[-73.728363,42.822277],[-73.726663,42.822577],[-73.725463,42.822277],[-73.723663,42.821477],[-73.722663,42.820677],[-73.722163,42.819677],[-73.721363,42.813077],[-73.720163,42.811477],[-73.718863,42.810277],[-73.717421,42.809429],[-73.717163,42.809277],[-73.716663,42.808177],[-73.716963,42.807477],[-73.717091,42.807284],[-73.717763,42.806277],[-73.718663,42.805677],[-73.719863,42.804877],[-73.720363,42.804177],[-73.720563,42.803077],[-73.720363,42.802277],[-73.719863,42.801277],[-73.718263,42.800167],[-73.71461,42.797633],[-73.713663,42.796977],[-73.712363,42.795977],[-73.711363,42.795177],[-73.710935,42.794504],[-73.710663,42.794077],[-73.709963,42.793377],[-73.708963,42.787677],[-73.708699,42.78744],[-73.70359,42.78286],[-73.703163,42.782477],[-73.697862,42.778777],[-73.697761,42.778777],[-73.697668,42.778735],[-73.696862,42.778377],[-73.696795,42.778347],[-73.696763,42.778332],[-73.693762,42.776977],[-73.693525,42.776858],[-73.691362,42.775777],[-73.690262,42.775477],[-73.690065,42.775459],[-73.689162,42.775377],[-73.688362,42.775477],[-73.687762,42.776377],[-73.687262,42.777077],[-73.685928,42.777743],[-73.684862,42.778277],[-73.683952,42.778736],[-73.683457,42.779112],[-73.682704,42.779931],[-73.681869,42.780962],[-73.681403,42.7819],[-73.681162,42.782977],[-73.680262,42.783377],[-73.680202,42.783437],[-73.680188,42.783447],[-73.680064,42.783576],[-73.678262,42.783477],[-73.676762,42.783277],[-73.677508,42.781345],[-73.678462,42.778877],[-73.681311,42.772759],[-73.681807,42.771695],[-73.681862,42.771577],[-73.682378,42.770247],[-73.683269,42.767949],[-73.683313,42.767834],[-73.683762,42.766677],[-73.683774,42.766621],[-73.683896,42.766051],[-73.684362,42.763877],[-73.684362,42.760778],[-73.684362,42.757878],[-73.68483,42.75701],[-73.684997,42.756699],[-73.686433,42.754022],[-73.686466,42.75396],[-73.687262,42.752478],[-73.687063,42.751679],[-73.687148,42.751069],[-73.687416,42.749103],[-73.687445,42.748894],[-73.688007,42.744902],[-73.688328,42.742602],[-73.688486,42.741471],[-73.688643,42.74034],[-73.688656,42.740251],[-73.688668,42.740162],[-73.688707,42.73988],[-73.688763,42.739479],[-73.688962,42.738278],[-73.689945,42.73582],[-73.689953,42.7358],[-73.690011,42.735661],[-73.690026,42.735626],[-73.690162,42.735277],[-73.69049,42.735045],[-73.690524,42.735021],[-73.690547,42.735005],[-73.69231,42.73376],[-73.692721,42.733465],[-73.693698,42.732764],[-73.694662,42.732078],[-73.695781,42.730658],[-73.69648,42.729771],[-73.6972,42.728858],[-73.697244,42.728818],[-73.697263,42.728778],[-73.697346,42.728365],[-73.697484,42.727675],[-73.697563,42.727278],[-73.698212,42.725121],[-73.698612,42.723794],[-73.699363,42.721478],[-73.69948,42.721169],[-73.699609,42.72083],[-73.700037,42.7197],[-73.700691,42.717982],[-73.700736,42.71785],[-73.700761,42.717777],[-73.701428,42.715808],[-73.70212,42.713757],[-73.703295,42.710273],[-73.703462,42.709776],[-73.703563,42.709478],[-73.703463,42.705578],[-73.702963,42.704378],[-73.702375,42.70101],[-73.702363,42.70088],[-73.702463,42.699978],[-73.702921,42.6993],[-73.706463,42.694078],[-73.716363,42.682378],[-73.719763,42.677779],[-73.722063,42.674779],[-73.723263,42.672879],[-73.724964,42.670179],[-73.72718,42.667448],[-73.728202,42.666188],[-73.729127,42.665052],[-73.729229,42.664937],[-73.730227,42.663916],[-73.730864,42.663279],[-73.736364,42.659179],[-73.739429,42.656803],[-73.740364,42.656079],[-73.741764,42.654379],[-73.741776,42.654358],[-73.741798,42.654318],[-73.744864,42.648879],[-73.746564,42.646179],[-73.747644,42.643233],[-73.747664,42.643179],[-73.747706,42.643084],[-73.747874,42.64271],[-73.748068,42.642276],[-73.748396,42.641548],[-73.748432,42.641467],[-73.748964,42.640279],[-73.749986,42.638328],[-73.75067,42.637024],[-73.751165,42.63608],[-73.752665,42.630279],[-73.752965,42.628679],[-73.759565,42.614179],[-73.759739,42.61379],[-73.761265,42.610379],[-73.761365,42.607679],[-73.761385,42.607274],[-73.761565,42.603579],[-73.761265,42.600779],[-73.759571,42.594625],[-73.758265,42.58988],[-73.757265,42.58718],[-73.754365,42.58258],[-73.752365,42.57668],[-73.752185,42.567657],[-73.75218,42.567391],[-73.752168,42.56672],[-73.752165,42.56658],[-73.754065,42.55678],[-73.755665,42.55158],[-73.756465,42.54868],[-73.756765,42.54378],[-73.756666,42.543681],[-73.756789,42.542972],[-73.75696,42.54199],[-73.758313,42.534215],[-73.759847,42.529433],[-73.76113,42.526322],[-73.761332,42.525832],[-73.763091,42.522026],[-73.76377,42.521135],[-73.764085,42.520721],[-73.765101,42.519649],[-73.766079,42.518782],[-73.766683,42.518048],[-73.768534,42.51557],[-73.773161,42.509377],[-73.773178,42.509354],[-73.773235,42.50928],[-73.773872,42.508442],[-73.773903,42.508401],[-73.77588,42.505817],[-73.777085,42.503999],[-73.779397,42.500081],[-73.78046,42.498023],[-73.780883,42.497306],[-73.781382,42.495921],[-73.784594,42.489947],[-73.784973,42.489013],[-73.785244,42.488135],[-73.785548,42.487209],[-73.785754,42.486114],[-73.785927,42.484761],[-73.786047,42.483835],[-73.786112,42.482804],[-73.786165,42.480065],[-73.786128,42.478017],[-73.786124,42.477814],[-73.786078,42.475249],[-73.785839,42.472881],[-73.785709,42.471632],[-73.785459,42.470859],[-73.785036,42.469852],[-73.784537,42.468394],[-73.784298,42.467749],[-73.784136,42.467508],[-73.78406,42.466968],[-73.784049,42.466517],[-73.783721,42.464231],[-73.787484,42.463831],[-73.791862,42.463367],[-73.800081,42.462493],[-73.800277,42.462473],[-73.807895,42.461663],[-73.808021,42.46165],[-73.80833,42.461617],[-73.809251,42.461519],[-73.809414,42.461501],[-73.812442,42.461179],[-73.817455,42.460647],[-73.817485,42.460644],[-73.818097,42.460578],[-73.825173,42.459825],[-73.836757,42.458592],[-73.837787,42.458439],[-73.838056,42.458399],[-73.839491,42.458187],[-73.843715,42.457565],[-73.85273,42.456286],[-73.854061,42.456097],[-73.854076,42.456095],[-73.86559,42.454372],[-73.865704,42.454355],[-73.874571,42.453078],[-73.876035,42.452881],[-73.876612,42.452803],[-73.878505,42.452549],[-73.878651,42.452528],[-73.879131,42.452458],[-73.884536,42.451726],[-73.885986,42.451526],[-73.886126,42.451507],[-73.889765,42.451008],[-73.892719,42.450558],[-73.892771,42.45055],[-73.8949,42.450216],[-73.901037,42.449362],[-73.903599,42.448999],[-73.903827,42.448967],[-73.905478,42.448756],[-73.90558,42.44874],[-73.907085,42.448507],[-73.907754,42.448413],[-73.907919,42.44839],[-73.913536,42.447608],[-73.915632,42.447295],[-73.920097,42.446628],[-73.927147,42.445718],[-73.929697,42.445341],[-73.930699,42.445192],[-73.931368,42.445093],[-73.931491,42.445075],[-73.931701,42.445044],[-73.93326,42.444814],[-73.937867,42.444281],[-73.939837,42.444063],[-73.939973,42.44405],[-73.94231,42.443794],[-73.947612,42.443243],[-73.948003,42.443204],[-73.951564,42.442856],[-73.95876,42.442147],[-73.962712,42.441735],[-73.963792,42.441622],[-73.966664,42.441323],[-73.969339,42.441044],[-73.969502,42.441027],[-73.971117,42.440859],[-73.975808,42.44039],[-73.979518,42.440066],[-73.980293,42.439999],[-73.984156,42.439658],[-73.987746,42.439241],[-73.988185,42.43919],[-73.992017,42.438644],[-73.992641,42.438556],[-73.995684,42.438316],[-73.995803,42.43829],[-73.995873,42.438273],[-73.998115,42.438061],[-73.999575,42.437918],[-74.01728,42.435827],[-74.017553,42.435795],[-74.018465,42.435689],[-74.01855,42.435679],[-74.018636,42.435669],[-74.02369,42.435086],[-74.028744,42.434502],[-74.036435,42.43361],[-74.043579,42.432703],[-74.057055,42.430985],[-74.061539,42.430428],[-74.071831,42.429155],[-74.071903,42.429146],[-74.076377,42.428592],[-74.084058,42.427643],[-74.087099,42.427268],[-74.087348,42.427237],[-74.090671,42.426834],[-74.096366,42.426145],[-74.096495,42.426129],[-74.107934,42.424744],[-74.108057,42.424729],[-74.116845,42.423664],[-74.117311,42.423608],[-74.124578,42.423243],[-74.126442,42.422984],[-74.126722,42.422938],[-74.129645,42.422569],[-74.130067,42.422515],[-74.137404,42.421581],[-74.138266,42.421471],[-74.139101,42.421363],[-74.147402,42.420294],[-74.147496,42.420275],[-74.149656,42.419995],[-74.156361,42.419128],[-74.157858,42.418932],[-74.160253,42.418624],[-74.164963,42.418106],[-74.174014,42.417109],[-74.175424,42.416955],[-74.190568,42.415289],[-74.190614,42.415284],[-74.194646,42.414841],[-74.194807,42.414824],[-74.194844,42.41482],[-74.198186,42.414453],[-74.198521,42.414417],[-74.20147,42.414093],[-74.201584,42.414081],[-74.201993,42.414036],[-74.21704,42.412365],[-74.217558,42.412307],[-74.225652,42.411408],[-74.232356,42.410663],[-74.240915,42.409712],[-74.244809,42.40928],[-74.249587,42.408762],[-74.253302,42.408325],[-74.254303,42.408207],[-74.263469,42.407127],[-74.263749,42.409695],[-74.264271,42.414459],[-74.264285,42.414589],[-74.264741,42.418747],[-74.264755,42.418876],[-74.264863,42.419863],[-74.264731,42.420127],[-74.263185,42.423213],[-74.261614,42.426476],[-74.261547,42.426615],[-74.260537,42.428717],[-74.260426,42.428949],[-74.259471,42.430937],[-74.256505,42.436905],[-74.256018,42.437915],[-74.25586,42.438243],[-74.255686,42.438606],[-74.255613,42.438757],[-74.255333,42.439338],[-74.254413,42.441252],[-74.25422,42.441653],[-74.253797,42.442534],[-74.252603,42.444945],[-74.252437,42.445334],[-74.252431,42.445347],[-74.252424,42.445359],[-74.252424,42.445369],[-74.252414,42.445401],[-74.252408,42.445422],[-74.251815,42.447366],[-74.250733,42.450773],[-74.250201,42.452482],[-74.250072,42.452899],[-74.249585,42.454598],[-74.244754,42.469353],[-74.244614,42.469782],[-74.244509,42.470103],[-74.244484,42.470182],[-74.244457,42.470265],[-74.241629,42.472036],[-74.236447,42.485459],[-74.234037,42.48774],[-74.233176,42.488716],[-74.233163,42.488731],[-74.233155,42.48874],[-74.230217,42.492071],[-74.229328,42.493267],[-74.229311,42.493289],[-74.22928,42.493331],[-74.229255,42.493365],[-74.229081,42.493599],[-74.22899,42.493722],[-74.228976,42.493741],[-74.228924,42.493811],[-74.228501,42.494381],[-74.227294,42.497231],[-74.227213,42.497422],[-74.226447,42.499233],[-74.226494,42.499762],[-74.226522,42.500076],[-74.22655,42.501642],[-74.226551,42.501681],[-74.226551,42.501694],[-74.226568,42.502633],[-74.225772,42.506956],[-74.224985,42.50855],[-74.224986,42.50856],[-74.225137,42.510562],[-74.225139,42.510585],[-74.225186,42.51118],[-74.225431,42.514962],[-74.225436,42.515046],[-74.225502,42.515413],[-74.225602,42.515716],[-74.225633,42.515948],[-74.225637,42.516446],[-74.225585,42.516578],[-74.225574,42.51676],[-74.225609,42.517165],[-74.225603,42.517351],[-74.225574,42.517491],[-74.225568,42.51761],[-74.225603,42.518166],[-74.225597,42.518318],[-74.225557,42.518513],[-74.225557,42.51873],[-74.225517,42.518814],[-74.225436,42.518924],[-74.225419,42.519026],[-74.225453,42.519225],[-74.22554,42.51953],[-74.225626,42.519776],[-74.225632,42.519937],[-74.225557,42.52023],[-74.225564,42.520467],[-74.225639,42.526403],[-74.225655,42.526415],[-74.229065,42.52903],[-74.229186,42.529284],[-74.230887,42.532035],[-74.233045,42.535624],[-74.23358,42.536486],[-74.233602,42.536522],[-74.234419,42.53784],[-74.238151,42.543857],[-74.237977,42.544593],[-74.241572,42.550802],[-74.241019,42.55182],[-74.240993,42.551868],[-74.240428,42.552911],[-74.240517,42.553105],[-74.236959,42.56012],[-74.237621,42.563124],[-74.237081,42.564629],[-74.237018,42.564811],[-74.236954,42.564971],[-74.236917,42.565063],[-74.236167,42.567795],[-74.234505,42.573854],[-74.23442,42.574162],[-74.23412,42.575256],[-74.234114,42.575276],[-74.233995,42.575688],[-74.227489,42.58028],[-74.226392,42.581054],[-74.226334,42.581094],[-74.225303,42.581801],[-74.224323,42.58248],[-74.224586,42.582238],[-74.220274,42.585412],[-74.220204,42.586283],[-74.220012,42.588021],[-74.219639,42.591386],[-74.219614,42.591406],[-74.219502,42.591545],[-74.219141,42.592066],[-74.219115,42.592107],[-74.21903,42.592241],[-74.218756,42.592646],[-74.218155,42.593644],[-74.217487,42.59428],[-74.215166,42.597435],[-74.210509,42.60099],[-74.209492,42.602295],[-74.209383,42.602435],[-74.209035,42.602881],[-74.206763,42.605796],[-74.206697,42.605993],[-74.205223,42.607431],[-74.204656,42.607985],[-74.202263,42.61032],[-74.201316,42.61124],[-74.2012,42.611354],[-74.198504,42.613981],[-74.198439,42.614044],[-74.197698,42.614768],[-74.197585,42.614882],[-74.193495,42.618999],[-74.193468,42.619183],[-74.192798,42.625076],[-74.192737,42.625608],[-74.192735,42.625628],[-74.192731,42.625658],[-74.192665,42.626233],[-74.192569,42.627069],[-74.192024,42.627849],[-74.191731,42.628271],[-74.19164,42.628402],[-74.188557,42.632825],[-74.18674,42.635514],[-74.18642,42.635988],[-74.186407,42.636007],[-74.186391,42.636031],[-74.186155,42.636379],[-74.18596,42.636717],[-74.18593,42.636769],[-74.185572,42.637363],[-74.185385,42.637678],[-74.184258,42.639582],[-74.184235,42.639622],[-74.184085,42.639611],[-74.184018,42.63963],[-74.183938,42.639666],[-74.183834,42.639733],[-74.183788,42.639765],[-74.183099,42.64036],[-74.182728,42.640805],[-74.182059,42.641788],[-74.181427,42.642842],[-74.180929,42.643806],[-74.18092,42.643824],[-74.180875,42.643911],[-74.180367,42.644871],[-74.180325,42.644931],[-74.17998,42.645436],[-74.179933,42.64549],[-74.179883,42.645537],[-74.179686,42.645766],[-74.179627,42.645852],[-74.179324,42.646384],[-74.178923,42.647284],[-74.17806,42.649321],[-74.177603,42.650135],[-74.177437,42.6505],[-74.177285,42.650971],[-74.174934,42.655661],[-74.174875,42.655758],[-74.174775,42.65587],[-74.173553,42.656963],[-74.17348,42.657065],[-74.173428,42.657206],[-74.173428,42.657318],[-74.17356,42.65794],[-74.173639,42.658689],[-74.173613,42.658878],[-74.173546,42.659247],[-74.173011,42.660429],[-74.172648,42.661298],[-74.172152,42.662693],[-74.171941,42.663126],[-74.171395,42.664127],[-74.171166,42.664544],[-74.171012,42.664804],[-74.17076,42.665227],[-74.170562,42.66556],[-74.170192,42.666342],[-74.170088,42.666635],[-74.169922,42.667133],[-74.169878,42.667226],[-74.169848,42.667278],[-74.169725,42.667426],[-74.169703,42.667588],[-74.16947,42.66929],[-74.169436,42.669462],[-74.16857,42.673868],[-74.168525,42.67431],[-74.168705,42.67866],[-74.168743,42.679248],[-74.168756,42.679452],[-74.168867,42.681187],[-74.168869,42.681216],[-74.168891,42.681549],[-74.168895,42.681609],[-74.169111,42.684953],[-74.169498,42.690953],[-74.169525,42.691365],[-74.169496,42.691501],[-74.169046,42.694107],[-74.169034,42.694177],[-74.169018,42.69427],[-74.169015,42.694288],[-74.169011,42.694312],[-74.169009,42.694321],[-74.169007,42.694331],[-74.169004,42.694349],[-74.168984,42.694466],[-74.168193,42.699051],[-74.168148,42.699314],[-74.168045,42.69991],[-74.167181,42.704931],[-74.166253,42.709479],[-74.166241,42.709536],[-74.165155,42.714867],[-74.165066,42.715304],[-74.164769,42.71676],[-74.164678,42.717207],[-74.16616,42.71842],[-74.166985,42.719096],[-74.167084,42.719177],[-74.16967,42.721295],[-74.169716,42.721333],[-74.171317,42.722645],[-74.171558,42.722842],[-74.172518,42.723628],[-74.172928,42.723964],[-74.180274,42.729979],[-74.168808,42.73145],[-74.155935,42.7331],[-74.155283,42.733184],[-74.154631,42.733268],[-74.153716,42.733385],[-74.146111,42.734361],[-74.140172,42.735125],[-74.139736,42.735179],[-74.139492,42.735209],[-74.139199,42.735248],[-74.139082,42.735263],[-74.138131,42.73535],[-74.137765,42.735384],[-74.137409,42.73543],[-74.136448,42.735555],[-74.135511,42.735676],[-74.127553,42.736699],[-74.124566,42.737083],[-74.120326,42.737573],[-74.113753,42.73833],[-74.113732,42.738333],[-74.112839,42.738473],[-74.1128,42.738478],[-74.112599,42.738502],[-74.112398,42.738526],[-74.111089,42.738682],[-74.110646,42.738735],[-74.110203,42.738788],[-74.107648,42.739095],[-74.107074,42.739165],[-74.106499,42.739235]]],"type":"Polygon"}}]}

{"overall_stats":{"title":"Overall PBF Stats","data":[{"label":"Total Elements","value":1341306},{"label":"Node Count","value":1216247},{"label":"Relations Count","value":1444},{"label":"Kerb Count","value":22},{"label":"Crossings Count","value":1804}]},"way_distances":{"title":"Way Distances","data":[{"label":"overall","value":19449212.58},{"label":"residential","value":1699453.75},{"label":"highway","value":7165006.08},{"label":"tertiary","value":755785.85},{"label":"service","value":2300961.02},{"label":"motorway_link","value":119421.08},{"label":"unclassified","value":462637.61},{"label":"track","value":134733.04},{"label":"primary","value":526335.3},{"label":"primary_link","value":21366.13},{"label":"tertiary_link","value":4135.55},{"label":"secondary_link","value":333.58},{"label":"motorway","value":203469.63},{"label":"footway","value":323957.4},{"label":"road","value":539.32},{"label":"busway","value":468.03},{"label":"path","value":485078.91},{"label":"living_street","value":2499.19},{"label":"pedestrian","value":12744.29},{"label":"secondary","value":34881.57},{"label":"cycleway","value":60966.37},{"label":"construction","value":963.53},{"label":"steps","value":1344.45},{"label":"sidewalk","value":87058.51},{"label":"trunk","value":8845.26},{"label":"raceway","value":988.03},{"label":"services","value":1744.92},{"label":"rest_area","value":836.27},{"label":"corridor","value":515.99}]},"way_counts":{"title":"Way Count","data":[{"label":"overall","value":123615},{"label":"residential","value":5650},{"label":"highway","value":35457},{"label":"tertiary","value":990},{"label":"service","value":20222},{"label":"motorway_link","value":440},{"label":"unclassified","value":664},{"label":"track","value":466},{"label":"primary","value":1064},{"label":"primary_link","value":136},{"label":"tertiary_link","value":51},{"label":"secondary_link","value":4},{"label":"motorway","value":452},{"label":"footway","value":3542},{"label":"road","value":2},{"label":"busway","value":11},{"label":"path","value":1303},{"label":"living_street","value":11},{"label":"pedestrian","value":88},{"label":"secondary","value":95},{"label":"cycleway","value":131},{"label":"construction","value":3},{"label":"steps","value":102},{"label":"sidewalk","value":656},{"label":"trunk","value":9},{"label":"raceway","value":5},{"label":"services","value":2},{"label":"rest_area","value":1},{"label":"corridor","value":13}]},"highway_intersections":{"title":"Highway Intersections","data":[{"label":"4","value":5572},{"label":"5","value":111},{"label":"6","value":6},{"label":"7","value":2}]}}

The performance of the code for the above is approx. 50k elements per second. And it takes around 197 seconds or approximately 3 min and ~30 seconds.

Analysis of Albany County NY

Using the above data we now have the details for the Albany County NY on Road and Sidewalk network. As you can see below visualization the Highway network is around ~4452 miles (or 71,65,006 metres)

whereas the Sidewalk network is only ~54 miles (or 87,058 metres).

When we represent it on the map we can clearly see the contrast and the sea of blue (which represents the highway network, and the red that represents the sidewalk network).

Future Upgrades

1. Time taken to process the pbf file.
2. Optimizing memory usage.
3. Improving the performance using multi-threading or parallel processing.

Conclusion

This blog post has demonstrated how to use Python and Osmium libraries to extract statistics from OSM data files. We have created a custom osmium.SimpleHandler class to process and examine the data and produced a JSON file with the results. We further are able to visualize it into charts and extrapolate that information into a map as demonstrated above. We hope that this tutorial has been useful and informative for you. If you have any questions or feedback, please feel free to leave a comment below. This project is a part of the complete suite of solutions that Gaussian Solutions offers for Modernizing Geospatial Analysis & Transit Data Pipelines.