So, some background about Figure.NZ and why we exist might help to answer some of the questions! We’re a charity, rather than a commercial enterprise, and our aim to to try and get New Zealanders to use numbers in their thinking so we’re all more informed.
To achieve that, we looked at the barriers to using public data currently — there are a lot, but one of them is that it’s in unusable forms and it’s hard to find and understand. That’s the problem we’re tackling with Grace. We realised that it’s a long, slow road for organisations to change their internal processes to make usable data in standardised formats, so we bridged the gap until they’re ready. We’ll be delighted when they release more standardised formats via APIs, because it will make part of what we do much easier! At the moment, one of our team processes and checks every dataset we publish. They need to tell Grace what the person who made the spreadsheet intended with their colours and indents and footnotes, and create rules to handle all those strange and wonderful things we find.
Outside of the technology, our team work to improve data literacy, including working with schools and media — we’re trying to understand what social and personal factors make people scared to use numbers, and figure out how to solve that.
So with that in mind:
- Why limit it to NZ? Our charitable purpose is to help New Zealanders use numbers, so it makes sense to us. We’d like to focus on serving that goal, and our team isn’t large enough for us to process global data and still do that. We’ve also learned that it’s incredibly beneficial to have an understanding of the context the data is created in. Long-term, we’d love to make the software available to organisations with similar goals globally, but that’s not a short-term thing. As a non-commercial, internal-use product, we’re able to build and experiment fast without worrying about impacts on other teams or organisations.
- How is it different to something like BigQuery? Solutions like BigQuery are great once data is in a usable form, but not much help getting it there. They also don’t have the specialist typing features we’ve created in Grace (the benefit of building something to solve a particular problem!) Grace is designed specifically to get data out of unusable formats, build relationships via typing, and then visualise. It’s not designed for general-purpose query-driven analytics work — it’s not a database in that sense.
- Do you think that it’s important for collecting institutions to move toward interoperable data standards, or will it become a moot point because of the rise of data mining and tidying services like Grace? I can’t speak for our team, but personally, yes I think it’s important. We only tidy up the data because we have to in order to fulfil our mission. If we didn’t have to tidy it, our job would be much, much easier. We could take tidy data, and type and relate and visualise in a much more efficient fashion. We could also dedicate more resources to our education and outreach. Our goal isn’t to replace standardised formats, but to provide something usable now until organisations are ready to release their own. We see the tidying piece as the slightly annoying thing we have to do before we can start the fun stuff!
If you have other questions, feel free to drop me an email at nat @ figure.nz