We recently made some major improvements to the quality of search results on figure.nz. This is a technical guide to how Elasticsearch can be creatively used for searching open data.


On a site like figure.nz, giving people the search results they expect is hard. Improving that experience for our visitors matters a lot to us, and it’s a challenge we’re constantly working on. We’ve recently dedicated a significant piece of work to doing exactly that, and it’s showing promising results.

The figure.nz website is a treasure trove of New Zealand’s open data. We take data published by organisations, clean it up, make pretty charts and maps, then publish it all on our site, where it’s free for anyone to use.

We believe data is for everyone, not just experts, and a key part of that is making sure everyone can find what they’re looking for when they come to our site. However, data is collected by experts who use jargon, which for accuracy purposes we often can’t simplify. Whether it’s that flights to the Gold Coast are recorded as being to​ Coolangatta​ (the indigenous name for the area), that chicken pox is technically called​ varicella, or that reported crimes are classified as victimisations​, there are endless examples of this divide in worldview. …

@InternetNZ is currently running a consultation on what their position on Online Voting should be. This post is inspired by a discussion (login required) around the threats that apply independent of any particular online voting system.

For context, New Zealand is considering trialling online voting as an alternative option to run alongside postal voting for the 2019 local authority elections.

What follows are two things:

  • The beginning of a catalogue of the security issues relevant to online voting, excluding any that apply to the online voting system itself
  • A collection of scenarios in which those security issues could be used to attack an election in various ways. …

This article was first published at The Spinoff.

News of Auckland Council’s IT cost blowout, and the tediously predictable way in which it happened, had eyes rolling up and down New Zealand yesterday. With $1.2 billion dollars spent and counting, this is shaping up as one of the biggest overruns ever for a country all too familiar with them. As a software developer building complex, bespoke IT systems for years, here is some free advice that the public service could find useful.

#1. Half of IT projects costing more than $15M MASSIVELY blow their budgets

Source: McKinsey-Oxford, 2012. “Massively” for software projects means an average 66% blow out. These project also overrun their time by 33% and deliver 17% less benefit than they were predicted to. …

This article was first published by The Spinoff as “The Bullshit and Myths of Online Voting”.

“A little knowledge is a dangerous thing”. This is a refrain we should keep in mind wherever we can, because a little knowledge is not enough to know just how dangerous we can be. It’s for this reason that I find the subject of Online Voting so troubling.

As someone who has been writing web applications for years, who has been on the sharp end of penetration tests, and who writes software holding medical records, I know too much about the limitations of the internet to be happy with the notion that we could vote online. It’s a real shame — like many I do so much online — but like a large number of engineers who build and maintain the various parts of the internet, I know the limits of my craft. It’s time more people did. …

On the 30th of July, 2015, I was able to make an in-person submission to the Justice & Elections Select Committee, which was holding a (routine) inquiry into the 2014 New Zealand general election. What follows is a rough transcript.

So, online voting!

We have people so excited about the potential online voting has for voter participation, and reducing cost, and I really do understand why that is. In the past five years especially, there has been so many pieces of software, and websites, like Xero and Trade Me, that have dramatically changed people’s lives, and shown us all a shiny future. Looking at this chart (below), the willingness to vote online seems to be building, and is especially high among people with tertiary-level qualifications. …

This data has come from Statistics New Zealand. What does it show?

Image for post
Image for post
Screenshot of an Excel file from Statistics New Zealand.

The obvious answer is that it shows the estimated population by sex from 1926 to 2013. Since it comes from Statistics NZ, you may assume it is the estimated population of New Zealanders, and you’d be correct.

If you’re like most people, getting any more meaning out of this data looks like effort. …

New South Wales is currently holding its state elections. Voters can either use a paper ballot, or an online voting platform called iVote, which is expected to collect 200,000–250,000 votes. And guess what…

Thousands of NSW Election votes open to online tampering!

For six days, the website was vulnerable to a malicious script injection attack which researchers have demonstrated could have been used to read and change votes. In that six day period, 66,000 people voted using the system. And the researchers don’t mean “could have, but wasn’t”. They mean “could have, nobody knows”.

There are some points we should be aware of…

I went to Kiwicon 8 last week. Superficially, it’s a conference where people talk about hacking things — whether it’s your apartment intercom, your DVD player, a Cisco firewall appliance, the law, electric skateboards, home detention bracelets, you name it. However, the attendees also discuss how the systems around us are/can be broken, and how to prevent, prepare for and defend against their failures.

Given that New Zealand is moving ahead with plans to roll out Online Voting to some councils as a so-called “trial”, Kiwicon was an interesting place to be. Not only was there ample evidence that apparently drunken tinkerers in their spare time could hack almost anything, but any systems left standing are at the mercy of criminals who can buy exploits or rent time on a botnet for peanuts. …

I see that the government is taking another step down the Online Voting path by enabling a small number of local government authorities to “trial” online voting.

The quotes are mine, because they’re not talking about a trial. It’s not like people will do their postal vote, then also use an online system as a test. The systems rolled out will be production systems with a politically binding outcome. It only makes sense to call this a “trial” if the end goal is much larger. I am afraid the end goal of this project is Online Voting for national elections.

The Online Voting project in New Zealand is being driven by a combination of some policy people, some local government mayors and the government itself. Behind that is some feedback from voters who say they’d like to vote online, but based on the last election they care more about, well, almost anything else. That’s it. The government wouldn’t even engage in an issue with this level of support if it was, say, feeding hungry schoolkids. …

In New Zealand, we have the right to vote. I want to unpack what this means, to demonstrate one of the biggest problems with Online Voting and how it’s marketed today: it ignores, and risks subverting, the democratic principles on which our elections are based.

To begin, what does the right to vote mean? Article 21 of the Universal Declaration of Human Rights says:

“Everyone has the right to take part in the government of his country, directly or through freely chosen representatives… The will of the people shall be the basis of the authority of government; this shall be expressed in periodic and genuine elections which shall be by universal and equal suffrage and shall be held by secret vote or by equivalent free voting procedures.” …

Nigel McNie

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store