Knowledge4Policy is Out of Beta. What have we learnt so far?
Getting Out of Beta marks a beginning, not an end.
The long-term goal of the EU Commission’s Knowledge4Policy platform (K4P) is to help organise and present policy-relevant scientific knowledge from across Europe, for policymakers across Europe. That’s a vast undertaking, so when we launched the Beta version last May it was little more than:
a search engine sitting atop a database of knowledge, migrated into the platform from a few pilot ‘Knowledge Services’ managed by the EC’s in-house science service
- Building Knowledge4Policy (May 2018)
At the time, to be frank, it wasn’t even much of a database: we had duplicate records from the five or so pre-existing databases we were consolidating; we hadn’t figured out how to organise their conflicting taxonomies; and we presented content inconsistently. Unsurprisingly, while the Knowledge Services involved were happy to have a presence on K4P back in May, they kept their existing sites online. I’d have done the same thing in their place.
Since then, however, we have refined K4P to the point that it offers them more than they had, even with the inevitable compromises involved in migrating 9 existing sites, each with its own identity, look and feel, into one shared platform. This week K4P became the principal home for all bar one of these Knowledge Service, and therefore officially exited its Beta phase.
K4P officially exited its Beta phase. So what have we learnt since May, and what’s next?
So what have we learnt since May, and what’s next? To answer that I thought we could share selected slides from some internal presentations to set out where we are, and explore some of the innovations we’re exploring.
These slides are from internal presentations, so don’t expect (many) marketing visuals — just the facts. Which is probably appropriate, given our focus on evidence-based policy. Notes follow after the slides:
Slides 1–8: Joining up the Science
Knowledge4Policy brings together ‘evidence4policy’ content which had previously been distributed across over a dozen different websites by nine different Knowledge Services.
Those seeking this content can now find everything in one place, using a shared taxonomy to improve content discovery.
we avoided creating an ‘archipelago platform’ of stand-alone knowledge silos
Of course, this is just a start: while the Beta phase was all about ‘joining up the science’, K4P must become much more than a database to fulfil its mission. Post-Beta, our focus will be on adding value to that science to help policymakers make sense of it.
Structurally (slide 8, left), K4P is composed of a Platform layer, which allows users to seamlessly navigate all knowledge on the platform regardless of origin, and a “subsite” for each Knowledge Service.
The existence of subsites was controversial to some, who thought such organisational constructs terribly 20th Century, but the plain truth is that each Knowledge Service needs an identity if it is to engage policymakers, develop an online community, or exploit a local, more granular taxonomy (below).
Nevertheless, despite the subsites, we avoided creating an ‘archipelago platform’, where each subsite is a stand-alone knowledge silo. All the knowledge is in one single database, using a global taxonomy, so users need no longer search multiple sites to find the knowledge they need.
Slides 9–13: Knowledge Hates Siloes
Usability is also massively improved when all Knowledge Services use the same content types, presented with the same presentation templates.
These slides provide a tour through a few of the many different content types and menus, and how they are connected together.
We chose these slides because they emphasise how knowledge is presented globally, and closely integrated across the platform.
You will find, for example, only one profile on K4P of Eurostat, the EU’s statistical office, instead of one profile for each Knowledge Service using Eurostat data. That’s because we consolidate the data imported from the various Knowledge Services’ systems (CKAN repositories, Sharepoint databases, etc.), and in some cases have built two-way links between them.
you will find one profile of Eurostat, not one for each Knowledge Service using Eurostat data
And while each Knowledge Service may have its own local taxonomies, all content is also categorised by a global Thesaurus, allowing content to be linked together across the Platform. More on that below.
The economies of scale, finally, created by the Platform are also non-negligible, with developments for one Knowledge Service immediately available to all others. When we delivered a different Knowledge Service Home Page design to the Foresight Competence Centre, for example, that design became immediately available to the eight others (Slide 13).
Slides 14–18: Faceted Search
Behind the above simple presentation of a few content types lies additional complexity.
There are actually three levels of content type. Some are grouped together: ‘Resources’, for example, encompasses content types ranging from Datasets and Visualisations to Publications and Online Resources. Some content types also have subtypes: Publications, for example, come in four varieties (study, briefing, leaflet/brochure and audiovisual).
This lets us exploit faceted search, allowing users to successively refine their search results to find the content of greatest interest to them. For example:
- Slide 15: From the Platform home page, users can do a freetext search, and then refine their results by knowledge service, content type, related organisation and the global thesaurus. This thesaurus (based on the Commission’s upcoming ~1500-term Digital EUROPA Thesaurus) is itself searchable, allowing the users to search for terms with which they can refine their results.
- Slide 16: But before hitting Search, however, users could select a radio button in the Search Bar to limit the search to one or a ‘bundle’ of content types, such as Resources (6 different content types).
- Slide 17: Limiting search to a particular content type ‘unlocks’ facets unique to that content type: searching only Publications, for example, allows users to refine search results by both Publication subtype and Year of Publication.
- Slide 18: Searching within a Knowledge Service limits the search to content linked to the Knowledge Service, and ‘unlocks’ its Local Taxonomies, providing the greater granularity scientists and other specialists need.
This relies completely on our taxonomic architecture, set out later.
Slide 19–22: Under the Hood
Technology Approach: scalable system ecology
We are not reinventing what would be a very large, very complex wheel.
Instead, we are developing an API-based ecology of multiple existing systems, each focused on doing one thing well.
Rather than reinvent a very complex wheel, we are developing a system ecology
While the Content Management System (CMS) is the Commission’s Drupal 7 Multisite, for example, most of the content is imported from other systems, currently:
- three CKAN repositories,
- the JRC’s internal publication management tool (‘Pubsy’),
- the cross-EC Newsroom, which will also manage the many KS’ enewsletters.
This means publishers don’t need to enter their content into multiple systems. It also is a highly scalable architecture: we aim to link K4P to CORDIS next.
Taxonomic Architecture
As the above slides show, the site is effectively one big database. Search is the principle Content Discovery method, with ‘recommended reading’ linking related content from different teams across the platform. Getting the taxonomic architecture right is crucial for both.
Each pre-existing Knowledge Service, of course, had its own taxonomy: flat lists of keywords. Being scientific in nature, these are very granular: one Knowledge Centre, for example, had over 1500 keywords.
With another eight, highly diverse Knowledge Services on the platform at launch, not even the EU Institutions’ enormous EuroVoc thesaurus (over 7000 terms) is fit for purpose. Complicating things further is the Digital EUROPA Thesaurus (DET), a new thesaurus which every site on EUROPA will have to use. It’s a great idea, as it improves content discovery across the different EU Commission silos, but with only 1500 terms it’s clearly not granular enough for scientists managing knowledge.
So what to do? Fortunately for us, the EU’s Publications Office does a lot more than manage EuroVoc. With their help, we aim to develop a ‘3-layer’ taxonomic architecture (Slide 21):
- Level 1 is EUROVOC, with over 7000 terms, updated twice a year.
- Level 2 is the Digital EUROPA Thesaurus (DET): most of its 1500 terms are from Eurovoc, but it has ~180 DET-only terms, and is faster to update
- Level 3 contains the Knowledge Service’s “local taxonomies”.
The DET (Level 2) is therefore K4P’s global thesaurus: it links knowledge developed by different Knowledge Services, piercing organisational siloes.
The Knowledge Services’ local taxonomies (Level 3), on the other hand, provide greater granularity to users searching within one specific Knowledge Service. This, our audience research showed, was essential for the scientists we need to bring to K4P if it is to succeed.
Crucially, we can use the Publications Office’s “VocBench” tool to not only host and manage these taxonomies, but also link terms in different taxonomies together via relationships (e.g., ‘similar to’), improving cross-silo content discovery even further.
Today, however, we have only levels 2 & 3, and no connection between our CMS and VocBench. But while we have still plenty of work to do, we already offer users global and local taxonomies, with all the advantages set out above.
Slides 23–26: Connecting internal silos
Although each Knowledge Service had been tackling similar problems of knowledge management and online communications, the first time they actually met to compare notes was at the first K4P workshop.
Building the Platform has now become one of the primary ways these different teams exchange ideas and develop shared approaches to similar problems at a detailed, operational level.
Building K4P has now become one of the primary ways different teams develop shared approaches
And that’s only been possible thanks to Connected, the EC internal collaboration platform. We put a lot of effort into Connected in the lead-up to the Beta Launch in May, because creating K4P meant finding a consensus across these different teams.
On their old sites, of course, they had autonomy: if they wanted to add a content type, or change the way content was displayed, they simply called their front-end developer. On K4P, however, there is only one set of content types, and only one way to present them.
For all teams to agree, the way K4P works must not only be good — it must have their buy-in. We needed an internal community of practice.
Central to K4P’s community on Connected, therefore, is an Interactive Development Pipeline that involves the Knowledge Services every step of the way (Slide 25).
This worked better than hoped, which says a lot of good things about the scientists’ working with us on this project. Today, it’s not uncommon to see scientists from different teams discussing quite technical issues on Connected, and even adopting each others’ taxonomies and terms (Slide 26).
What’s next?
Of course, the above slides only touches on some of the lessons learnt, and not in any great depth. There was also no room to address the way K4P is designed, so a post on that is coming soon.
On our desks right now is a much bigger challenge: to set out a workprogramme to guide the post-Beta development phase. We already have a ‘draft wishlist’, reflecting the above lessons and our audience research, carried out as we launched K4P and got it out of Beta. Our next step is to compare that wishlist with the way the Commission’s corporate web publishing is evolving, notably its move from Drupal7 to Drupal8.
Interesting times ahead. Stay tuned, and join in, below.
Join in
As (I hope) the above post shows, transparency is one of our bywords, so we’ll continue to publish as much as possible here. Follow us to stay informed, and contribute by either Responding to this post, submitting your own or by Tweeting to @EUScienceHub.
