It’s Been a Ride 🎢

A Story about Contributing SAP HANA Support to QGIS

Mathias Kemeter
Jan 21 · 10 min read

Tl;dr QGIS 3.18 comes with support for SAP HANA. Contributing took a little while. Get a free trial instance of SAP HANA Cloud and download the QGIS nightly build.

Image for post
Image for post
Photo by Christian Perner on Unsplash

About the author: Mathias works as a development manager for SAP and is responsible for SAP HANA’s Spatial engine. This article may be biased and reflects personal views, but be assured that it’s based on a true story 😊

the field of GIS (Geographic Information Systems) the open-source software packages are rather feature rich and mature, so that for many companies these are a valid enhancement or even alternative to proprietary software packages. Thus, at our customer’s site we often find hybrid landscapes using proprietary GIS software, open-source GIS software and SAP systems for covering their non-GIS business processes.

SAP is committed to enabling The Intelligent Enterprise. For us, as a development team within the Business Technology Platform, this means that one of our goals is to provide a platform, which is capable of transparently incorporating our customer’s business data to fuel their business processes. Moreover, as the development team behind SAP HANA’s Spatial Engine, we care about the location-side of things and want to enable our customers to seamlessly integrate their GIS and ERP to harvest the value and the sometimes-untouched potential of their business data.

Image for post
Image for post

rom a GIS perspective SAP HANA is already well-integrated with Esri software, the open-source middleware GeoServer as well as DBeaver as a tooling for the spatial database developer. Any GIS professional will have already spotted one or two gaps on that list — one of them for sure being the open-source GIS client QGIS.

While spatial data in SAP HANA could already be displayed in QGIS by leveraging the WMS/WFS services provided by GeoServer, we wanted to investigate the possibility of having a more native integration to leverage QGIS extensions like the TimeManager. Back in the days — somewhen around 2018/2019 — we took the decision to contribute native SAP HANA support to the QGIS project.

aid and Done. Well… uhm… almost. Let’s have a look at how our concise and clear path towards contributing to the QGIS project turned out.

Image for post
Image for post

May 2019

On May 21st, 2019 and after being clear about what we would like to achieve, Max from our team forked the QGIS repository and started to technically plan the contribution. We started developing and testing the plugin internally before we placed our first (we didn’t know yet, that it wouldn’t be the last) pull request.

July 2019

That was on July 16th, 2019. We were done with the implementation and just had to wait for the pull request to get merged. We thought. Then we got some feedback from the community:

“[…] we might need a bit more context on what this is and why it needs to be a core provider?”

“HANA database, that’s new to me :)”

“[…] a [HANA] provider would not be of use to a vast majority of its user”

“Historically when we’ve done this is becomes a huge maintainance burden on the project”

“I’d like to point out process of QGIS Enhancement Proposal […] before going deeply in implementation like in Pull requests”

Image for post
Image for post

There were some valid questions, some advice and a couple of comments that hurt our developer egos (Come on! If there is no HANA support, how could it possibly have a large HANA user base?). There were also kind comments and, of course, there was the pointer to the QGIS Enhancement Proposal (QEP) process, which we have — regrettably — not considered before.

After having a couple of discussions with the community and clarifying some of the questions, including license concerns, we sorted ourselves and made the official enhancement proposal according to the process.

That was July 24th, 2019. Sticking to the process helped (Hey, we are SAP. We should’ve known before). The discussion got considerably more objective. Well, almost.

A: „Adding a new provider isn’t just a free gift to the project. Rather it’s more akin to giving someone an unexpected puppy for Christmas. Sure, it’s all fun at first, but then there’s always someone who inevitably has to feed it, clean up after it, pay the vet bills,…“

B: „It seems unfair to me that QGIS has core providers for commercial databases, but does not allow to contribute support for our database in the same way it has been done for other commercial databases. Even worse, it feels like we are being punished for a competitor neglecting the maintenance of its core provider, while the competitor was actually rewarded by the community taking over the maintenance of that provider.“

A: „Unfair? […] Maybe in the same way that a first child might get different standards for parenting and have different rules from a second child, simply because the parents have learnt more about parenting due to their previous mistakes experiences!“

Image for post
Image for post
Photo by Karsten Winegeart on Unsplash

When looking at the history of communication the above dialogue really nails down the core suspicions between both parties — the QGIS community and our SAP team.

The QGIS community has learnt from past experiences and was concerned about the additional maintenance burden, while we felt misunderstood and unfairly treated.

August 2019

The discussion went on for a couple of weeks. On August 15th, 2019 our original pull request got closed due to inactivity. In the beginning, it felt that discussing with an open community (rather than with a single representative) is like fighting windmills. Once you agreed on one point, another person stepped out of the dark and brought a totally new aspect, so that you could never be sure, what the state of the discussion actually is.

September 2019

Finally, on September 21st, 2019 we concluded on the major pillars of our contribution (e.g. testing platform, CI integration and maintenance).

For us this was a major break-through! We finally had a reliable agreement we could build upon. It was clear, that we had some additional work in our backlog — but it all seemed manageable and clear.

December 2019

Last day before Christmas break on December 20th, 2019, I had the wishful thinking, that we could issue our second pull request and could possibly get into the next version of QGIS. I even had a celebration blocker in my calendar — but of course, this remained a wishful thinking. We have learnt that we would require more community expertise to efficiently flesh things out.

Image for post
Image for post
We missed the Bossche bollen 😢

January 2020

There was a QGIS Contributor Meeting (also known as Hackfest) planned to take place in Den Bosch in March 2020. That would be the best opportunity to get in touch with the community and discuss technical topics! (…as a 2021-reader you will already anticipate that something ‘unprecedented’ came in the way). We gathered a travel group of 3 team members and — since you do not go to a house party without having a present — SAP also sponsored the event.

In the meantime, we adapted our 1-year-old codebase to some major API changes, which affected our contribution.

March 2020

On March 10th, 2020 the Contributor Meeting finally got cancelled due to COVID-19.

As it was clear, that in near future there was no opportunity to liaise with the community, we finally issued our second pull request on March 11th, 2020. Almost certain that almost everything has been done and we are almost good to go. Due to my devotion for the number Pi, I remember that a realistic feature release was QGIS 3.14 at that time. (I have learnt that I seem to be an optimist rather than a realist)

April 2020

We received a couple of review comments, which we considered until April 22nd, 2020. Afterwards, Stefan started our internal peer review process — as it is part of our engineering standards.

July 2020

We finished the full peer review process of the meanwhile quite large codebase. On July 3rd, 2020 I took another stab in declaring production-readiness. However, there was still this one nasty issue: Due to memory consumption, we did not manage to run our HANA Express docker image on the Travis CI of the QGIS project next to the images of the other databases. This ultimately led to the fact, that we could not run our test suite on the CI.

Image for post
Image for post

September 2020

Meanwhile Max gained expert knowledge of Travis, which is why today he is unwillingly considered to be the CI expert in our team. Finally, we gave up to run the database on the CI.

On September 17th, 2020 we could run our tests using an SAP HANA Cloud instance on the QGIS Travis CI. Thus, we took advantage of the fact that SAP HANA Cloud has been released in the meantime.

December 2020

We have been undergoing further community reviews until December, which we have thankfully considered for our contribution. On December 18th, 2020 — last day before Christmas break — I had a nostalgic look back and thought of the foolish feeling last year, that we could get our contribution merged in 2019. This time, I felt confident to see the merge happening in 2020 still.

January 2021

Of course, based on prior experience my confidence was clouded. On January 6th, 2021 Matthias — one of the main contributors of the QGIS project — announced that he plans to merge our contribution by the end of the week if there are no further concerns.

Image for post
Image for post

After not sleeping for 4 full days (actually, I slept well… but this would not add to the dramaturgy), our almost 2-year-old pull request finally got merged into the QGIS master branch on January 10th, 2021.

Still, our tests ran for almost 10 minutes as the CI infrastructure was in the US and our HANA Cloud instance in Europe. So, we spun up an instance on the US east coast and came out with an acceptable latency.

Today

Here I am writing this article and waiting for the last changes to the QGIS Mac Packager to be merged. Once this is done, I will install the nightly build and publish this article not one single second before I see the SAP HANA data source. As mentioned before, I have learnt that I tend to be too optimistic. So I’d rather play it safe this time.

The heck with it! I’ve seen it working with the nightly build on my Windows VM — what could possibly go wrong?

Image for post
Image for post

fter all, it seems that we have delivered our unexpected puppy present. We were a bit late for Christmas, but we will take it out and clean up after it. And of course, it’ll be fun! No one ever doubted that!

And they lived happily ever after.

Are we all friends now? Yeah, kind of. There is still a bit of suspiciousness. But I personally do understand the rational behind way better than I did 2 years ago.

We, as a team, are looking forward to becoming a good citizen of the QGIS project. After all, we are developers and spatial enthusiasts. The bumpy ride we had, just got us more engaged with the project.

If in doubt, this article should be interpreted as being self-critical and self-deprecated. Once we left the current pandemic behind, we are very much looking forward joining an upcoming QGIS Contributor Meeting!

Learn more about SAP’s commitment to open-source software

Follow me on Twitter and watch out for further communication and tutorials on how to use SAP HANA Cloud with QGIS 3.18. If you cannot wait, just start using the SAP HANA Cloud Trial and the QGIS nightly build (both free of charge) today!

Image for post
Image for post
It’s been a ride.

The Startup

Medium's largest active publication, followed by +771K people. Follow to join our community.

Mathias Kemeter

Written by

Spatial, Data Science and other math stuff @SAP @math1ask https://about.me/mkemeter

The Startup

Medium's largest active publication, followed by +771K people. Follow to join our community.

Mathias Kemeter

Written by

Spatial, Data Science and other math stuff @SAP @math1ask https://about.me/mkemeter

The Startup

Medium's largest active publication, followed by +771K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store