The PIFT method: 4 necessary conditions for expertise
This post series presents basic ways of understanding your own expertise or that of others.
The puzzle of expertise assessment
A physicist-turned sociologist friend of mine told me a story about a Chinese emperor named Qin Shi Huang. As the story goes, Qin Shi Huang sought to live forever. So, he recruited a multitude of advisors to find the hidden secret to immortality. Unfortunately, the men he recruited had no ability to assess which magicians or alchemists might be experts in immortality elixirs. Furthermore, the emperor had no ability to assess which advisors might be experts in assessing experts in immortality elixirs. (Note even further that there are no true experts in the relevant domains here. This may remind you of fields such as technological forecasting.) In the end, as these things tended to go, a lot of advisors got executed.
This has a lot in common with Meno’s Paradox. Here is Meno’s paradox rephrased as a paradox of expertise assessment:
- If you are not an expert in domain D, it is impossible to independently assess whether someone is an expert in D.
- You are not an expert in D.
- Therefore, it’s impossible for you to independently assess whether someone is an expert in D.
This argument is probably false. Particularly, premise 1:
- If there exist domain-general markers that tell you whether someone may be an expert — then it is possible to independently assess whether someone is an expert in D, even if you are not an expert in D.
- There exist domain-general markers (that tell you whether someone may be an expert).
- Therefore it is possible to independently assess whether someone is an expert in D, even if you are not an expert in D.
To follow I will describe four domain general markers for assessing expertise across domains. I’ve combined them into a “back-pocket” method — one you can use to understand the expertise of people you meet at conferences or dinners, or for people who you deliberately seek out. The four posts are definitely not exhaustive, but it should cover most cases.
The thoughts in these posts come from our research in the Human Advancement Project, but also personally putting many many hours into things like designing hiring processes, interviewing lots of job candidates, reviewing 1000+ applications for a conference I used to co-direct (Effective Altruism Global), and creating the Pareto Fellowship evaluation pipeline.
The claim: each of the four markers are necessary (but, importantly, not sufficient) conditions for expertise.
How to read these posts
You may find the four conditions to follow to be pretty obvious, but ask yourself:
- Do I explicitly assess for these four conditions while engaging experts? If not, you may run the risk of, e.g., hiring the wrong people or acting on faulty expert information.
- For areas in which I am attempting to gain expertise, am I accounting for the four conditions in my training? If not, you will find skill gain difficult or impossible.
To gain fluency in assessing each of the following conditions, I recommend the following process:
- Write down a list of examples where it is important that you trust either your own knowledge or skills or someone else’s. These are examples where either you might be the expert or you are counting on an expert.
- For each example, think of ways in which the four conditions apply.
To help with memorization, I call this the PIFT method. Click below for the relevant posts.
