Data standard for democratic organizations — Popolo
New to data standards? Please read these introductions.
About Popolo
Popolo is a data standard used to monitor democratic organizations and their decision-making processes. It models organizations’ assemblies, elected representatives as well as other employees, debates and processes such as voting or filing of motions. It does not model legislation documents though, a field where the Akoma Ntoso specification and its successors are strong. It features tools dedicated to national parliaments, local councils as well as organizations like co-ops, associations or political parties.
Popolo was created to support the development of Poplus open source components and since then has been used in other tools, mainly devoted to monitoring legislative and executive bodies at a national level.
There is no standard similar to Popolo, though there are many standards that intersect with it. For example, a person that is modelled by Popolo, is also modelled by Friend of a Friend, Schema.org, IETF’s vCard specifications, OpenSocial or Facebook data specifications to name just a few. Popolo aggregates all those specifications into one that is dedicated to democratic organizations.
Popolo’s closest “ally” (meaning both specifications complement each other) among standards would be the Akoma Ntoso standard for modeling executive, legislative and judiciary documents. The standards overlap and both allow the modelling of speeches, but Popolo is a lot simpler to use.
Popolo’s official website is PopoloProject.com.
What kind of data is covered?
You can use Popolo to model MPs voting for a piece of legislation, a government hierarchy with posts and even the debates of your local co-op.
Popolo models real-life objects / scenarios into data classes (ie. Member of Parliament -> Person class instance) at a level of generalisation which can fit a variety of legal and social contexts. For example, while the Person class is quite specific (modelling real person, alive or dead), the Organization class can be used to model a chamber of parliament, a committee, a political party, a club, an assembly, a department, etc. It is the responsibility of the person modelling a given organization to categorise it to be closely compliant with other deployments of the Popolo standard.
As for the classes, I’ll cite the Popolo documentation:
- Person is a real person, alive or dead (e.g. Donald Tusk).
- Organization is a collection of people with a common purpose or reason for existence that goes beyond the set of people belonging to it (e.g. IX convocation of The Verkhovna Rada of Ukraine).
- Membership is a relationship between a member and an organization (e.g. member of the XYZ Party).
- Post is a position in an organization that exists independently of the person holding it (e.g. Prime Minister).
- Contact detail is the means of contacting a person, organization or post (e.g. a telephone number, email address, postal address, contact form, etc.).
- Motion is a formal step to introduce a matter for consideration by an organization (e.g. “I move that the group donate $5 to Wikipedia,” then comes the debate and voting on the introduced motion).
- Vote event is an event at which people’s votes are recorded (e.g. a vote to pass a bill).
- Vote is a voter’s vote in a voting event (e.g. John Q. Public voted in favor of passage).
- Area is a geographic area whose geometry may change over time (e.g. a country, city, ward, etc.).
- Event is an occurrence that people may attend (e.g. a legislative session, a meeting, a hearing, etc.).
- Speech is a speech by a Person, a scene (e.g. applause), a narrative (e.g. “The House rose at 3:20pm”), or another non-speech part of a discussion (e.g. a list of bills).
The Popolo specification was build by reusing parts of several other standards to make it as understandable and familiar as possible for its users. For example: Person class has birthDate, deathDate, image, email terms taken from the widely-known Schema.org standard and name, nick, gender taken from the Friend of a Friend standard.
Popolo is a dynamic standard, there is an ongoing discussion on several new classes and properties including standardisation of contact types, candidacy and sponsorships.
Side note: Sunlight has expanded Popolo with Jurisdictions, Divisions and Bills classes creating the Open Civic Data data standard, that also standardizes read-only API serving the extended Popolo standard. Work on this project has slowed down, but it may become the basis for new classes in Popolo.
Who uses Popolo?
The power of the standard comes from the power of all the stakeholders that are using it. So who uses Popolo?
Worldwide Popolo is mainly used in deployments of Poplus tools — SayIt, WriteIt and PopIt — by a number of organizations behind these projects, including MySociety from UK, OpenNorth from Canada, Ciudadano Inteligente from Chile, as well as organizations in South Africa, West Europe, and CEE that has done deployments of these tools.
In CEE region Popolo is being used to gather in one API parliamentary data from over 10 countries, as part of ParlData project done as a collaboration of Kohovolit, ePanstwo Foundation and Open Data Kosovo.
What tools can process Popolo data?
Having open-source tools that can immediately visualize and analyze the data you have just opened can be helpful. So, what’s available for Popolo?
SayIt visualizes debates and allows to search them.
WriteIt, a Poplus component allows you to write and send messages to public figures, and share their responses publicly in effort to collaboratively build a knowledge base on public matters. You can import public figures’ data into WriteIt.
Visualize organizations and people in embeddable widgets gathering data from the ParlData API.
SayIt
You can use Poplus’ SayIt component to visualize and search public debates. See it presenting real-time speeches from a number of CEE parliaments or the works of Shakspeare.
SayIt supports Popolo allowing it to import Speakers. Speeches themselves can be imported through a custom importer developed as part of the ParlData project.
ParlData
Data on elected officials, chambers, committees and other parliamentary assemblies can be viewed through interactive widgets that can be embedded on any website. These widgets are fueled by Popolo-compatible ParlData API.
Where to find Popolo compatible data?
If you are looking for parliamentary data to analyze there are two (at least) databases to check for the existence of data structured for the Popolo format: EveryPolitician by MySociety and ParlData API by Kohovolit and Fundament.
Every Politician
EveryPolitician is a MySociety initiative to provide basic data about every politician in the world. They focus on consistency and usefulness, which specifically means that “EveryPolitician will only provide data that we [MySociety] are confident can be expressed in the same manner across the majority of national legislatures.”
Data for all CEE country parliaments can be exported in a bulk (single file containing all the data) for eachcountry.
ParlData API
The ParlData project aggregates data from a number of CEE parliamentary chambers. The focus was put on comprehensiveness, so it generally offers data on every Popolo datatype: MPs, assemblies, speeches, votings, etc, though the availability of specific datasets differs by country.
Data can be accessed by API at http://api.parldata.eu.
Warning for developers: Event class offered by API is slightly different than the standard, because it had yet to be standardised within Popolo while this project was developed. If you’re planning to create a new tool based on this API and process events please contact the team behind it to let them know that it is time to fix it.
How to open up the data?
This section is here to help organizations that have started opening up parliamentary data. We assume that you know the basics (data scraping and cleaning), so let’s jump to challenges with modelling cleaned data into the Popolo standard and the next step: sharing it with the world.
Challenges in data modelling
Modelling is inseparably connected with generalising and losing details. With a standard as general as Popolo questions such as “How should I model a particular case?” will come up quite often. In this section, we will gather tips for common case scenarios? Want to add something from your experience? Please share it with us!
The safe strategy for modelling, focusing on easy re-use is the one developed by the Every Politician project. The strategy is to model only the fields that are unambiguous:
“There is lots of other very valuable data available on politicians worldwide, and eventually we hope to include much more. But for now we don’t include anything other than the above [basic info about a politician] — no asset declarations, no memberships of corporate boards, or parliamentary committees, or ministerial positions, etc. Of course, we’re not saying that this data isn’t important — far from it! — just that we haven’t yet worked out how to provide it in a way that is consistent and comparable across all national jurisdictions.” — Every Politician docs
However, if you’re trying to gather as much data as possible this safe strategy is not the ideal option. The tips below come from ParlData project development and should help you do the modelling in a similar fashion.
Please check Scraper development guidelines for Visegrad+ project
Which public database to contribute to?
If you are opening up parliamentary data, please remember that sharing is caring. Please contact the project teams mentioned in the previous section and ask how you can contribute data there. This way,, others won’t have to repeat your work in the future.
While opening only the dataset of members of a national parliament we suggest that you contribute data to the EveryPolitician project.
When you plan to open more datasets we suggest using ParlData project’s API. You can host its own instance or you can use api.parldata.eu as a hosted solution. Write-access to API can be granted for partners contributing the data and this has the advantage that the provided data will be immediately available to all data consumers that base on this API.
How to contribute to the standard?
Popolo is developed publicly on Github in an open process. At PopoloProject.com you will find clear guidelines on how to contribute to the standard (the focus is on providing use cases). Decisions are made through rough consensus. Development is lead by James McKinney, founder of OpenNorth. To start, join existing issues or create new ones describing your use cases and propositions to extend the standard.
Are you ready to release your organization’s data in the Popolo standard?
Originally published at transparencee.org.
