The Definite (& ONLY) Guide to FastKML.

I recently completed a 3 Month Software Development Internship with Mission Support System, an Open Source Flight Planning Software under the aegis of the umbrella organisation Python Software Foundation under Google Summer of Code 2020.

My project involved Enhancing KML Support as a feature, and I used FastKML library to do that. Over the course of these 3 months, I found that there isn’t a lot of documentation for FastKML, and lots of its features are in the grey area.

I faced a lot of difficulties starting out with FastKML. But, through experimentation, and reading the code, it became easier to understand. I did a lot of work and I am writing it all down, so that you don’t have to. Hopefully, your FastKML implementation will be truly, fast :))

I’ll start with the basic foundation, and then build up to using FastKML.

Starting out with KML

XML, HTML , XHTML … sound familiar? They all have ‘ML’ in common which stands for Markup Language.

A Markup Language is a computer language that uses tags to define elements within a document. It is human-readable, meaning markup files contain standard words, rather than typical programming syntax. Its NOT a Coding Language!

Basically, if you have some data within certain categories, you can write that data within defined tags, and that data can be easily understood by humans and machines alike.

Sometimes the data can be generic (Grocery Inventory) or quite specific. The tags are usually defined and specifically cater to the particular extension. (eg. .kml, .xml)

Its time we talk about KML.

KML is a file format used to display geographic data in an Earth browser such as Google Earth.

KML stands for Keyhole Markup Language and is a notation for expressing geographic annotation and visualisation within 2D maps and 3D Earth browsers.

Extra Reading : Check out this link to know why KML came into existence and being accepted as an International Standard in the Open Geospatial Consortium (OGC).

Major Tags of KML

Here’s a simple example of a KML File. The tags and data inside those tags are very specific to geographic annotations.

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
  <Placemark>
    <name>Simple placemark</name>
    <description>Attached to the ground. Intelligently places itself 
       at the height of the underlying terrain.</description>
    <Point>           <coordinates>-122.0822035425683,37.42228990140251,0</coordinates>
    </Point>
  </Placemark>
</kml>

I’ll be explaining KML in short, since Google already has a vast documentation on it.

The major tags of KML can be divided into 3 categories :

• Features
• Geometry
• Styles

(Its hard to define feature in terms other than hierarchy. It might not be clear at first, but as you read on, things will make sense.)

Feature

We’ll start by defining Placemarks. Placemarks are the specific plots for which the data is available, in the form of coordinates, geometries etc.

For example : Placemarks in a KML File can be the 7 Wonders of the World. So, these 7 wonders will be plotted in the form of various geometries having different styles.

These Placemarks can be contained inside Folders and Documents. The natural order is :

<Document>
     <Folder>
        <Placemark>
         .....
        </Placemark>
      </Folder>
</Document>

However, it can be in other orders as well.

• Features or Placemarks can be a point, an area, or a big region.. anything than can be plotted.
• Placemarks can have attributes such as <name>, <description> and more.
• With the advancement of 3D browsers , it is now possible to plot the terrain of a Placemark region as well, and that data is stored using <Overlay> tags. There are 3 Overlay tags : Photo, Screen and Ground Overlay.
• There are other tags such as <Network Link> which allow you to connect to a site, and get realtime updated KML Data every few seconds. This is useful when you want realtime earthquake data and so on.

You can find a table below, explaining the major KML Tags with examples!

KML Reference | Keyhole Markup Language | Google Developers

Geometry

Geometry refers to the actual geometry of the plot that the KML file contains. These are all the Geometrical categories supported:

• Point
• LineString
• Polygon (Inner & Outer Boundaries)
• Linear Ring
• MultiGeometry (Geometry consisting of combos of the above)

They have various attributes which help define these geometries more clearly.

Styles

We have different drawing Styles for different geometries :

• LineStyle for drawing Lines through attributes such as color and linewidth
• PolyStyle for drawing Polygons through attributes such as fill and outline
• IconStyle for drawing icons for Point Placemarks
• LabelStyle drawing the name of Placemark in 3D viewer
• BalloonStyle for drawing description balloons for Placemarks
• ListStyle for listing Features in Google Earth

You can find out about KML in detail below.

KML Tutorial | Keyhole Markup Language | Google Developers

As per OGC, KML Files cannot be created by anyone, i.e. KML follows a specification standard which has to be met to release such files in the public domain. You can find the conformance levels here.

Python Libraries dealing with KML

Here is a list of commonly used KML related Python Libraries.

• lxml : Main Library to deal with all kinds of XML
• SimpleKML : For generating KML Files easily
• FastKML : For creating, parsing KML Files
• PyKML : For creating, parsing KML Files
• Signxml : Provide payload security for verifying signatures
• Defusedxml : Provides security from xml vulnerability attacks

Getting Started with FastKML

We know now what KML is, and what are the constituents of a KML File. But how can we extract useful data from the file?

Either we can use the general library dealing with xml-ish file extensions — the lxml library. Its very useful, and you can find its usage to no end. The only problem is.. its so generic that you have to develop your code on this library for making the KML data useful.

Usually, parsing KML is a smaller part of a bigger project. For such reasons, its better to use a library specifically catering to KML.

FastKML is such a library. Like it advertises, it fast in its usage. i.e. its built on the lxml library focused on KML related functions. This forms a safe level of abstraction, and you only need to call relevant KML functions, and keep working on the bigger end of your project.

You can check out the FastKML project right below.

cleder/fastkml

Here’s a link to its documentation.

The only issue I had while using FastKML was the lack of documentation. There’s literally none.

In terms of documentation, there are 3 Python scripts; to create a KML File, read a KML File and one to run a KML File.

But, our use case isn’t running a KML File , is it? If that was the case, we would simply have ran the KML Files on Google Earth and save us all this trouble.

We need to parse the file, use that data for maybe visualizing on our maps or something.

Fastkml consists of a number of python scripts/modules. I’ll be focusing specifically on kml.py, geometry.py and styles.py as I worked with these modules extensively.

The only help you can find is through reading the test cases present in “test_main.py”.

Since it is missing documentation, I feel I can bridge some of the gap here by building on my own code.

I’ll take a moment here to explain my project : Creating a GUI based docking widget to access, parse and display multiple KML Files simultaneously on a given map.

Here’s my demo for the MSS Application :

In this article, I’ll focus more on using FastKML. For those interested to view the codebase of my work, I’ll add a link at the bottom of the article, so that its not lost in the long work that I’m gonna be writing.

I think its time to really get started.

Getting the examples to work.

Open fastkml/examples. In there, you will find 5 files : a Google KML Sample File, a README and 3 Python files namely CreateKml.py, ReadKml.py and UsageExamples.py.

Starting with CreateKml.py — It creates a KML File.

(Please note that the files haven’t been updated for Python 3. Therefore small syntax changes will throw errors. I will be pointing them out in this article, so you don’t have any problem).

To run CreateKml.py , change the line beneath Print out the KML Object as a string.

From

print k.to_string(prettyprint=True)

to

print(k.to_string(prettyprint=True))

Python3 changes.

Now you will be able to run the file without any problem.

The code and output of CreateKml is self explanatory. Its basically creating objects of different classes and calling them.

ReadKml.py : A KML String is provided, and then fastKML attempts to read it.

Make sure to make changes to the Python syntax for the print statement.

Try to run the code again. You will get the following error :

Traceback (most recent call last):
  File "ReadKml.py", line 35, in <module>
    k.from_string(doc)
  File "/home/aryan/anaconda3/lib/python3.7/site-packages/fastkml/kml.py", line 91, in from_string
    parser=etree.XMLParser(huge_tree=True)
  File "src/lxml/etree.pyx", line 3222, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1872, in lxml.etree._parseMemoryDocument
ValueError: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.

Something to do with Unicode Declaration. No issues. I’m really thankful to the developers contributing selflessly their knowledge on StackOverflow, StackGISExchange, StackAbuse and so on.. From there only did I learn how to solve this error.

The problem is with this line :

k.from_string(doc)

Its not utf- 8 encoded. Therefore, lets try to do that.

k.from_string(doc.encode('utf-8'))

The code works now! The KML String is read and printed out.

Lets try to get UsageExamples.py to work.

After removing the print syntax issues.. running the code gives the same error as before with the encoding problem being in :

k.from_string(kmlFile.read())

Simply change it to :

k.from_string(kmlFile.read().encode('utf-8'))

Running the KML File now prints out all the feature(which we discussed earlier) names present in the KML File! Also go through the KML Samples file to understand the KML Tags and what has been printed in the Terminal.

So, now we know how to create as well as read KML Files!

I got the code working, right? So you can trust me to know what I’m talking about here :))

Parsing Features

Now I’ll try to explain the code.

This is the relevant code.

from fastkml import kml
# Create the KML object to store the parsed result
k = kml.KML()
# Read in the KML string
k.from_string(doc.encode('utf-8'))
# k.from_string(kmlFile.read().encode('utf-8')) # when its a file

Till here the string/ file is read.

Now we have a method named ‘features’. Its a generator object, and generates all the features present inside a KML File. i.e. Document, Folder and Placemark. (Nothing else).

Mostly the files that you use have a root tag.

<kml>
</kml>

Inside we have usually a sub — root in which everything else is contained. This might be a <Document>, <Folder> or <Placemark>. But everything must be contained inside it.

<kml>
   <Document>
         .......
    </Document>
</kml>

should be of this form.

To get the features inside , we use the method ‘features’.

features = list(k.features()) # gives all the features inside

However, you must note these are only the features present on the outside. The tree structure, it doesn’t go inside the branches.

# ReadKml.py
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
<Document>
  <name>Document.kml</name>
  <open>1</open>
  <Style id="exampleStyleDocument">
    <LabelStyle>
      <color>ff0000cc</color>
    </LabelStyle>
  </Style>
  <Placemark>
    <name>Document Feature 1</name>
    <styleUrl>#exampleStyleDocument</styleUrl>
    <Point>
      <coordinates>-122.371,37.816,0</coordinates>
    </Point>
  </Placemark>
  <Placemark>
    <name>Document Feature 2</name>
    <styleUrl>#exampleStyleDocument</styleUrl>
    <Point>
      <coordinates>-122.370,37.817,0</coordinates>
    </Point>
  </Placemark>
</Document>
</kml>

Here, we see that the outer tag is <Document>. Therefore, when we run ReadKml.py, it gives len(features) = 1.

Inside Document, we have 2 Placemarks. Therefore when we run len(list(features[0].features())) it gives 2.

So, we need a recursive function to parse throughout the file, and you can find it in UsageExamples.py.

def print_child_features(element):
    """ Prints the name of every child node of the given element,   recursively """
    if not getattr(element, 'features', None):
        return
    for feature in element.features():
        print(feature.name)
        print_child_features(feature)

This will print out all the features (Documents, Folders and Placemarks). Of course, our main resource is Placemarks.

Here’s a function that I had written to find all Placemarks inside a KML File.

k = kml.KML()
document = list(k.features())

def parse_placemarks(self, document):
    for feature in document:
        if isinstance(feature, kml.Placemark):  
           placemark = feature
           self.parse_geometries(placemark)
    for feature in document:
        if isinstance(feature, kml.Folder):
            self.parse_placemarks(list(feature.features()))
        if isinstance(feature, kml.Document):
           self.parse_placemarks(list(feature.features()))

The above function parses , finds Placemarks throughout the file, and then parses the individual geometry of the Placemark as well (through ‘parse_geometries’ method).

Note: After getting the placemark, you can use placemark.name, placemark.description to get more attribute information :))

Parsing Geometries

The codebase for parsing geometries is present in the geometry.py file in fastKML. This file has been derived heavily from shapely, and currently pygeoif. So, you might want to have a look into it to solve your issues. Luckily for you, i already went code diving, and here is all the gold I found.

Just like placemark.name, placemark.description, you can call placemark.geometry.

All different geometries are classes inside the file. So i can test if the placemark geometry object is an instance of any of the geometry classes present in geometry.py.

Now that we can identify the geometry of a Placemark, mostly the focus is on acquiring its coordinates. These coordinates can later be used to plot on map using Matplotlib or other plotting means.

I was fairly shocked seeing this question on StackExchange :

Extracting coordinates of placemark from KML using FastKML?

This shows that there is a major lack of documentation for developers to use FastKML.

Based on my work, I identified all geometries and how to extract their coordinates through effortful experimentation. Hope the following gist is helpful for you all!

Now you can identify the geometry and extract relevant coordinates! These coordinates can then be used to plot the coordinates. You can check out my plotting of the coordinates at the link at the bottom :))

Parsing Styles

The relevant file for parsing styles in fastKML is : styles.py. You can find all the methods for different types of Styles in it.

Unlike Geometries, Styles can be global i.e. defined outside of a Placemark. So, in coding terms we can say that we have a global Style and local Style.

Most of the functions in FastKML are related to the KML Tags. So, we can easily call classes (such as styles.LineStyle and styles.PolyStyle) for our work.

Styles is a bit difficult to explain; its better to try it or see its implementation.

I would encourage you to try out using Styles on your own, based on geometry related gists above.

If you face any issues, check out usage of styles in here.

For a real world usage, you can check out the functions “draw(), parse_styles(), parse_local_styles() and get_style_params()” in my project link at the end of the article.

Bonus : Treasure trove of KML Examples!

googlearchive/kml-samples

Find examples of all types of KML here. Amazing collection!

My Project

Here’s a link to my project where I worked with Features, Styles and Geometries. This might be the single best use case of FastKML and KML in Open Source. So, for anyone dealing with KML, do check out the link below.

Bitbucket — KML_UseCase_MSS

I will be spending time over the following weeks answering stackoverflow, gis.stackexchange questions to help impart my KML knowledge. Giving back to the community is one of my main aims in life, and this will be a small attempt towards fulfilling it :))

I am really grateful to be a part of the Open Source Community!

Thanks for Reading! If this helped you out, do let me know! If you would like to discuss my work, or have doubts, you can connect with me on LinkedIn. I might be slow, but I do reply back :P