Why did I write keyserver-fs?
It started with me wanting to remove my personal details, and I didn’t understand why this was impossible. It frustrated me, so I wanted to learn more about how keyservers work and why they have been designed like this.
The Keyservers are originally designed in 1990 to resist government interference hence the need for immortal keys. This was during a time when encryption was still in a hot debate by governments.
A lot has changed since then.
Most people who are now aware of the inability to remove keys or don’t want them exposed to the public, use private servers or supply them on request or use their websites.
I quickly came to realize reading through forums and mailing lists there where much bigger problems that made my need to remove my public key unimportant and insignificant.
I discovered there is a hostile environment to change in the forums and mailing lists. People over the past few decades have repeatedly brought up concerns about the design of the SKS key servers and possible abuse scenarios that could happen in the future, unfortunately, this fell on deaf ears. One of the main excuses has been “it breaks the fundamental design of the keyservers”.
This is where I started wondering how these types of abuse would be implemented. I looked at things like the UID and wondered if I could do anything with it. I tried on the command line adding random things, and it turned out I didn’t have to use an email address.
Then I wondered how many characters can I add to the UID. To my surprise over 2k of characters could be added, more than this and the key becomes unusable, I’m not sure if it can still be uploaded, never tried. This is when the alarm bells went off, and I decided to write a PoC of how the SKS keyservers could be used as file storage and abused illegally.
I was not the first person to come up with this as I found out later while searching through a dump from a keyserver.
I found multiple keys with the phrase “PGP file system” next to the encoded text. There was also evidence of someone else testing something similar and using a reference “<test”> next to the encoded data.
I even found a dead magnet link, to what that originally linked to who knows. There was also over 1.5k of entry’s of address’s that linked to ssh servers in this format “ssh://xxx.xxx.xxx.xxx” and some with domains instead of IPs. The list of abuse and junk that already exists is hard to get through because there is so much of it. It’s clear this is only going to get worse.
I felt approaching the community was pointless as more than a decade of complaints in forums and mailing lists where met with hostility or disinterest. I needed to write an article using my proof of concept to highlight the issue and try and get media attention to maybe force changes in the keyservers. I also found out I am not the only one discovering problems with SKS as you will read later in this article.
Unfortunately, the responses have been hostile from some of the devs and some of the keyserver owners, some of the more polite response is below.
My approach was seen as “a deliberate attack” on the SKS key servers:
Phil being surprised that this has happened doesn't make much sense, he had warned people of a similar problem in an earlier article:
We’re one illicit-material-in-photo-uid incedent away from global shutdown.
Yet nothing was done, this worries me as this has been warned against before in the forums multiple times so this form of attack being exposed was just a matter of time and the Devs and keyserver owners new this, if this was a large software company they would fix this issue, instead of hoping nothing is going to happen.
I was disappointed in Robert J. Hansen’s response:
I am sure this was written in a moment of anger and he is a nice guy in irl, regardless Robert said he has been warning people over the past decade of these issues:
After a decade of warnings, it sounds like an article was needed to shake up the devs and admins.
People have also been shutting down their servers citing attacks causing their drives to fill up. All though there has been no unusual increase in key upload activity in recent days or weeks. Current dumps of the 5million plus keys only come to a little over 11GB. Which isn’t much for over 2 decades of keys, although a large amount of it is actually just junk. This could be cleaned up with new mechanisms to support key removal.
[Update + Correction]
Phil Pennock has corrected me on the topic of why some admins are having issues with Disks filling up, it is down to broken keys causing issues with the database and increasing disk usage by almost 5 times. It appears these keys where designed to do this. This is why some servers are being repeatedly knocked offline.
I would like to make it clear there are people in the mailing list and forums who are interested in making changes for the better and have already multiple times given useful input on these issues, but they are met with disinterest, excuses, and hostility, which is a shame as they could be the ones to save the SKS keyservers.
One ex-keyserver admin was nice enough share his experiences with SKS and its devs:
I first interacted with Kristian while setting up my keyserver a few years back. I wanted to join the HKPS pool, as I had configured HKPS to allow users secure HKP access over TLS, but couldn’t find a way to do so. I reached out to Kristian (who runs all of the pools) and he basically told me he didn’t recognize me and to come back when more people signed my personal key. He seemed very blunt, and I couldn’t understand why he would turn away free resources until I got random people to say they verified my identity. I believe in GPG, but any idiot can blindly participate in a key signing party. The strong set feels more like security theater than a valid vetting process.
Seeing the GDPR concerns and SKS vulnerabilities on the mailing list has resulted in a very polarizing environment. I’ve been running a keyserver for a few years and the mailing list has always seemed a a bit rough around the edges. People are friendly enough when you request peers for a proper setup, but as a new user I’d be scared of being reprimanded for something I miss-configured rather than be offered help. When dealing with these recent concerns, half of the people on the list seem to be angry that any of this is happening while the other half want to make everything compliant and protect against the discovered exploits. It’s a shame, considering the reasonable people pushing to have the flaws dealt with are faced with so much opposition. The developers have been silent so far, not even a “we’re looking into it” as far as I have seen. It doesn’t say a lot for the future of SKS, or the longevity of the current pools while more admins drop off amidst concerns.
The Design of the SKS servers are deeply flawed, and the public I don’t think fully understand this and that’s a problem. This is something that the devs are fully aware of and have done nothing to help protect its users from the abuse of “their” servers.
If you care about the SKS project and want to help people securely communicate and you see something that hinders this, it seems obvious to fix it and evolve with the changing landscape of threats and tech.
Apps like Matrix or Signal believe it is up to them to make secure communications are easily accessible by all and make it their responsibility to make sure its safe through auditing and listening to user opinions and bug fixing.
On to some of the issues that SKS keyservers are currently or have previously experienced.
I found this discussion between the main Dev Kristian and Micah Lee from the Intercept, where Lee tries to make suggestions for the removal of a serious bug. The discussion didn’t go anywhere unfortunately regardless of Micah Lee’s very clear point, it just ended with “wontfix it”.
Here [Yegor Timoshenko] points out faked keys belonging to Kristian the main dev:
The tool Yegor Timoshenko created is the source of the issue Micah Lee tried to discuss earlier with Kristian.
Yegor once again finds more problems where you can take down a server with a simple command:
or make keys unusable:
Please take time to visit Yegors repo on Gitlab.
This is just one person looking at the keyservers, what happens when it starts attracting people who have no interest in seeing these things being fixed but have malicious intent.
I am impressed by Yegor and his research, and I believe he will find plenty more bugs, I just hope his work isn’t wasted by a lack of fixes.
Even people in the Freebsd forums suggest to set up a private keyserver and not use the public ones due to unwanted behavior. The immutable feature is well known among professionals and isn’t seen as necessary anymore in order to use PGP.
So are SKS public keyservers safe when you can’t rely on your keys to not be interfered with by individuals or 3 letter agencies? When was the last time someone audited the SKS code base? What system do they have in place to deal with logs?
As suggested in an article by an ex-keyserver Admin Phil:
“You don’t know who is running the keyservers. You don’t know what’s happening to the logs. You don’t know that the keyservers are trustworthy. They are, at most, a useful swamp for collecting the data from so that clients can do WoT calculations without caring about fishing in a contaminated swamp, as the WoT if done right takes care of filtering out the sludge.”
WoT has also been highlighted as a privacy issue as well since government agencies can without much effort see who knows who.
These people are meant to “care” about user privacy, but they have been mostly hostile and negligent.
SKS keyservers are doing more damage to the reputation of PGP than actually helping it!
There is an effort to move to HockeyPuck another PGP keyserver software, it looks nicer but doesn’t really resolve any issues other than adding a privacy policy that still doesn’t comply with GDPR for the exact same reasons the SKS servers don’t, this is like trying to stop a fire by building another one next to it and then blowing on it.
The name of the server is Flanga I think the privacy policy for upload is a nice touch on the web interface and a move in the right direction, although no one will see this when using PGP from their client. Until you add the option of key removal your still in direct violation of the GDPR which I’m pretty sure will approach this subject in the near future.
Some kind of email verification is needed as well, much like the Global PGP directory which allows adding keys via email verification and removal the same way. Solutions are there, someone just needs to do something about it and stop holding on to the past.
Summary:
SKS keyservers are old and may be part of the bygone era in a world where encryption comes as default on a lot of recent applications such as Wire/Signal/matrix and even Whatsapp, there are tor and Yggdrasil for encrypted networking. Finally, there’s Keybase.io which caters to the general public and not just terminal junkies, even though its weaker in a sense because of the centralization of data it’s still a better option.
A̶n̶y̶o̶n̶e̶ ̶i̶n̶t̶e̶r̶e̶s̶t̶e̶d̶ ̶i̶n̶ ̶f̶u̶r̶t̶h̶e̶r̶ ̶d̶i̶s̶c̶u̶s̶s̶i̶o̶n̶s̶ ̶y̶o̶u̶ ̶c̶a̶n̶ ̶f̶i̶n̶d̶ ̶m̶e̶ ̶o̶n̶ ̶M̶a̶t̶r̶i̶x̶ ̶a̶t̶ #̶k̶e̶y̶s̶e̶r̶v̶e̶r̶s̶:̶h̶-̶i̶c̶.̶e̶u̶,̶ ̶p̶l̶e̶a̶s̶e̶ ̶b̶e̶ ̶p̶o̶l̶i̶t̶e̶ ̶a̶n̶d̶ ̶c̶o̶n̶s̶i̶d̶e̶r̶a̶t̶e̶ ̶t̶o̶ ̶o̶t̶h̶e̶r̶s̶ ̶i̶n̶ ̶t̶h̶e̶ ̶c̶h̶a̶n̶n̶e̶l̶.̶
Thanks to the people who proofread this article and made contributions to the discussion.
[Edit: 18-jul-2018]
Andrew Gallagher on the mailing list has submitted a thread called “SKS apocalypse mitigation” in which he highlights some very good solutions to illegal content and to unlawful processing of other peoples data. I hope he manages to get this implemented and its not just another suggestion lost in the wind.