How research and factchecking work at Kurzgesagt
Kurzgesagt ist proud to be one of the biggest science channels on YouTube. But what does the most sciency part of our work — research — actually look like? My name is Lizzy Steib, I joined Kurzgesagt in 2016 as a researcher and am now leading the editorial team as Head of Research. Let’s take a tour of our editorial department and look at what we do here, how we do it — but especially why!
Science provides the foundation for all our content, so almost every project, no matter if video, print or interactive, includes an early research stage and passes through the editorial and research team. Currently this team consists of six full-time employees: a head of research, two fact checkers, two researchers and scriptwriters and one social media editor. Our specific tasks vary, but in principle we all work together to make sure what Kurzgesagt puts out is scientifically reliable and of high quality.
YouTube is the core of our brand, so I will mainly focus on this example here. Similar steps apply to our product and interactive process and in a more condensed way to TikTok and social media in general though.
For our YouTube videos, research usually happens in one of two ways: either as an in-depth research or as an intense vetting process.
Research
In the first case, our inhouse research serves as a jumping-off point for script-writing. An in-house researcher is asked to find out everything interesting there is about a topic and put it in a document. This assignment is the gate way for a deep dive and helps to decide if a topic will be turned into a video. We often start out with simply reading the Wikipedia article just to get a very basic understanding and continue with reading primary sources like scientific papers, meta analyses and sometimes whole books. We extract all the prime information, the juicy facts, links, tables, graphs, interesting and relevant quotes from the papers, as well as comments and assessments from the researcher and put them into one document. Sometimes the key findings are also put into an essay that already suggests a storyline. The document is usually around 10 pages long, but can also grow to a couple dozen pages. Depending on the complexity of the topic and the current deadlines this process might take between a couple days and a couple weeks.
The scope of this task is a bit of a wormhole: you rarely know beforehand where it will take you and how much interesting stuff you will be able to find. Also, what do you consider interesting and what is trite and has been said a million times before? This assessment and the hunt for new angles needs an experienced instinct and is what makes research so time-consuming.
This is where script topics are born, but also where some die. If we don’t think the facts we find are surprising or fresh enough, the topic will be scrapped, however painful this might be. In the end we only want to publish topics we are absolutely happy with and we are the only ones to make that decision, whether it is for a sponsored video or one where we point to our products in the end card. If you want to learn how we work with sponsors in general, Philipp wrote an article about that specific aspect of our work a while back: Kurzgesagt & Sponsorships on Youtube
If the topic is deemed fit for a video, a writer takes over to shape the story and angle of the video and works in tandem with the researcher to finish the script, add information to certain talking points or do calculations for scenarios we want to discuss in that video. (Not a lot of papers around that tell you how many nukes you need to pulverize the moon. We did look first though, rest assured.)
The research process for our products like our posters looks similar: after developing an initial concept idea with the product team we create a research document as a scientific base. The finished product might look like “just” a poster to the unsuspecting onlooker, but collecting the information for it often requires weeks of research into the most obscure corners of the internet — or creating things that were unseen before, like mapping out the life cycles of stars after talking to an expert or creating and assessing a list of over a hundred species and their specific relevance for the development of life for the Map of Evolution.
Factchecking
The other way we do research starts with us coming across a fascinating paper or a conversation with a scientist that sparks our curiosity and sets off a rabbit hole of intense research and writing, sometime in collaboration with an expert, which results in a YouTube script. This is when the time of the fact checkers has come. A new script has been born — and their goal is basically to dismantle it. They are the bad cops here, the tight-lipped accountants, the nitpickers who weigh every word. We like to call this our internal peer review: the fact checkers get the plain script and try to look at it from an outside perspective, going through the script statement by statement with a fine-toothed comb. They compare multiple sources to make sure the statistic or fact is the most recent or commonly accepted. They recalculate numbers that initially made it into the script as back-of-the-envelope estimates. They assess if concepts have been simplified or summarized so much as to be misleading.
Based on what the fact checkers find in their review, they then suggest edits: numbers might change, wordings might be tweaked to be more accurate and additions can be made where more context is necessary.
Sources
Whenever you feel like it, you can trace this factchecking process back yourself: we document our research and fact checking meticulously in the form of a source sheet that is published in the infobox of every video we release. There you find the links and quotes of the papers we used, statement by statement, sorted chronologically as they appear in the video. For people who aren’t physicists or biologists themselves those formulas and terms might still not make a lot of sense, so we often also provide some context in the form of commentary. Plus we break down calculations we made or give some context why we might have chosen one number floating around the internet over another. For especially contentious or controversial topics we also put the sources directly into the video as onscreen text, so viewers are aware they don’t just have to believe us blindly. These documents are usually around 15 to 30 pages long, but some count up to 50 pages. We even have an ongoing source sheet with sources for our TikTok videos — to us that is a way to maintain our quality standards on this other platform as well.
The factchecking process also presents one the biggest challenges of being a fact checker, but is also one of the reasons why this job is so much fun: We work on many different topics from many different fields, so you get to have a new job every week! One time you are a paleontologist, the next time you transform into an astrophysicist. Of course the people on the team have different professional backgrounds and bring a range of experience in design, biology, physics, geology and maths. This might make someone a perfect fit for some topics, but because of the size of our team, the amount of projects we work on in parallel and also because we all love to learn new stuff, any topic might land on anyones desk. A typical quote from the editorial office might be: „I will do a little cancer until lunch and then move on to tardigrades.“
Working with experts
At this point, after all the reading and documenting and evaluating, we have already become rather close with our script storyline — you could say we have met its parents and are playfully talking about moving in together. Still we want to make sure before we ultimately commit, so we involve experts as outside council in every one of our videos. There are a number of reasons for this: first of all, we are aware of the limits of our knowledge. There are topics you just can’t pierce through unless you have expert insights and years or even decades of experience. Looking at a topic for weeks a time and discussing it with a specific goal in mind — making an entertaining and inspiring video — can also make you snowblind about controversies and logical faults in the story. The best reason to talk to experts when you ask the editorial team themselves though is just that — being able to discuss scientific topics with people who are working at the heart of that research, who are passionately letting you in on the secrets of their field and who often are able to point you to findings or points of view that you wouldn’t be able to get from any other source.
By now we are lucky enough to have built a network of scientists from various fields who we work with regularly and who we consider friends of the company. One example is Our World in Data, who create complex infographics about various topics and who we have been collaborating with for years now. They are a globally renowned source for accurate information about the world.
Whenever we work on a completely new topic we also might reach out to new experts who specialize in a niche field. This was the case for example when we got to talk to one of the scientists who co-discovered the Laniakea cluster about our poster on the topic.
When we work on inherently more controversial topics like the ones we did on nutrition a while back we make sure to deliberately get several opinions from experts who have different takes on the matter to create a more nuanced overall representation of the topic.
Sometimes we also get contacted by scientists offering their help for future projects — if you are a scientist reading this, you are welcome to do so as well! We will put your name in our expert pool to reach out to later if the right topic and opportunity come up. We usually reach out to two experts per video — sometimes one of them is already involved in the earlier stage of research and creating a research essay. When working in expert feedback we try to find a balance between being scientifically accurate but still using metaphors that make sense to people without extensive background knowledge. We make short internet videos, so we need to simplify, but our aim is to make sure every simplification is a deliberate choice based on a discussion.
The comments, additions or cuts from that expert review are then collected, carefully discussed and worked into the text — now the script is ready.
Quality assurance
Did you think the editorial team was done as well at this point? Nope, our work has just begun. When the script is finished, it goes into illustration and later animation. During those stages, the editorial team keeps accompanying the video and checks in at various stages to make sure no errors creep in. These quality assurance or QA rounds were first established when we noticed a typo in a final render in 2016 (looking at you, bacteria/o/um!) By now, they are based on a detailed checklist that looks at visual, typographical and content aspects — we are making sure that planets spin the right way, infographics are labeled correctly, numbers are consistent throughout the video and also match sources we have used before — the list literally goes on and on.
We apply different versions of the list after specific steps in the video production process like after the sketch phase, finalization of the storyboards and after the first render with final timing. One QA round for a YouTube video takes at least two hours, often more, and at least two people are involved into each QA round. Things that are unclear are discussed internally before our comments are handed over to the illustration and animation department.
Here again, we try to balance scientific accuracy with our medium: we create vivid animation videos, so our cells and molecules and planets will have funny faces and take on character traits and everybody will understand that this is creative liberty for entertainment’s sake. On the other hand, the locomotion of an amoeba that isn’t physically able to do what we are showing might be taken at face value without any context — so we will tell the animation team sorry not sorry, but this needs to change. (True story.)
This example brings us to our last question: Why do we go through all that trouble? Let’s be honest here: for an internet company that makes videos, being so granular with our research isn’t really playing in our favor. It takes a lot of time, it fried more than a handful of braincells and it still doesn’t mean that nobody will critize us for talking about certain topics, using certain metaphors or the way we approached a calculation. We still think it’s worth it.
For one, we are still around because millions of people watch our videos. But they don’t just enjoy our content, they also trust it. We are aware of this responsibility, so we feel it is our duty to do our best, continuously question our processes and improve them. The other reason is tied to why Philipp started Kurzgesagt and why every one of the editorial team does the job they do: we genuinely enjoy science. This initial spark of curiosity is what lead to the founding of this channel and describes the feeling we want to install in our viewers. To be curious to us is to go deep, to the bottom of things and even beyond that sometimes. In a nutshell, we all just love to nerd out — and because you all like to watch us do it, we actually can. This is a huge privilege, because many traditional media outlets can’t afford to spend as much time and resources on research as we do. Or they need to navigate the interest of sponsors and are forced to avoid certain topics or statements. Kurzgesagt was set up in a way that allowed us to stay clear of these pitfalls so far and we believe our love and excitement for science is what made this possible, so we will continue to try to do it justice. — Lizzy
