From Confusion to Expertise

How to ramp up your expertise quickly

“Your strength as a rationalist is your ability to be more confused by fiction than by reality. If you are equally good at explaining any outcome, you have zero knowledge.” — Eliezer Yudkowsky

When you work in libraries, learning how to learn is a vital skill. Every new area that emerges means a new struggle to study, understand, experiment and eventually integrate the knowledge into oneself.

I have done with various degrees of success for diverse areas such as Social Media, Bibliometrics, mobile, web analytics, chat reference &Web Scale Discovery Services.

When done right, the knowledge is so internalised in you that you can give an impromptu lecture or at the least summarise briefly the state of art of the area. Things we know definitely about the area, and the things we are still wondering about. You can also judge if a certain new development or talk is advancing new knowledge and hence worth amplifying. You probably can’t be the leading expert in every area, so knowing who are the go to go experts or innovators in that area would be something you should try to achieve as well.

The goal is not always to be in the group of leading experts or go around giving keynotes but be good enough to converse with such people and make points that make sense.

Having been around the block a few times on this roller coaster many times, this is what I have learnt about learning new areas.

As an example, one of the areas I am currently struggling with is in the area of Open Access as well as Loan/RFID issues.

The pre-foundation stage

Depending on the time you have before you need to demonstrate expertise, you can choose to plunge into the depths of knowledge starting from zero or adopt a slow bite-sized osmosis approach.

If you are good at anticipating when a certain issue will start becoming a “must-know” or are simply reading widely, you can slowly build up a base of knowledge over a period of time which is far less stressful, then suddenly realising you need to master something new.

For me, spending at least two hrs every day reading Twitter helps reduces the learning curve in most cases. For example though my institution was relatively slow to Web scale discovery and Open Access compared to some, by the time I decided it was an area I wanted to learn systematically I had read or glanced at countless blog posts, editorials and even a peer reviewed article or two on the topic.

Don’t get me wrong even if you do that your “knowledge” of the area is going to be superficial at best. You are going to have many misconceptions, or vague impressions based on what you read or heard people say or tweet.

At this point, if you spend even a small amount of timing trying to sort out your thoughts on the subject you will realise you don’t actually understand anything at all.

At this stage, you probably shouldn't offer your opinions on this topic unless you have a gun pointed at your head.

The foundation stage

Eventually, the mist will start to thin and you starting seeing the shape of things.

How you get there can vary.

It can involve very intense systematic thinking of things you have read coupled with real world experiences. This can take anything from 6 months to 2 years depending on the complexity of the issues.

Easiest though is if there is a very good “foundation text“ that systematically lays out the lay of the land. For example in the area of Open Access, Peter Suber and Walt Crawford have published exactly the type of “foundation texts” I am talking about.

Peter Suber and Walt Crawford’s — Open Access

The “foundation text” need not be an actual book or text even. It can be a bibliography or blog by an expert in the field commenting on studies or a series of interviews with experts.

However realistically speaking such texts may not exist if you are studying a really new tech area. This was the case when I wanted to study web scale discovery in 2009–2010.

Regardless of whether such texts exist, you are going to have spend lots of time thinking and reflecting on what you think you know or you don’t.

You should seek out confusion

It is a design flaw in human cognition that this sensation manifests as a quiet strain in the back of your mind, instead of a wailing alarm siren and a glowing neon sign reading “EITHER YOUR MODEL IS FALSE OR THIS STORY IS WRONG.” — Eliezer Yudkowsky on confusion

So you read and absorbed the “foundational text” on the subject and you think you are now an expert yes?


If only expertise was so easily gained. I’ve found that whatever area I study there will inevitably come a point where I am pretty sure I “figured it out” and then I read something or hear some arguments, and I feel confused.

Don’t fight it! Don’t delude yourself you “understand” when you feel confusion. This is a important warning sign that you don’t actually understand as much as you do.

I noted in my review on Peter Suber’s and Walt Crawford’s book that I found the final chapters on Economics of Open Access, strategies to reach Open Access extremely confusing.

You could say confusion is expected because these involve Open Access Controversies that have plagued the field and divide the open access advocates into many camps.

Still, it would be nice to be able to say something like…. “the differences between open access advocates can come down to different end goals or in some cases differences in local tactics employed.” Or perhaps the “arguments are a result of uncertainty about researcher behavior and can be resolved by empirical data”.

I would add that issues can be very deep, I have gone through multiple cycles of thinking I got it , and then some time later, a question or a thought passes my mind and I realise it’s not that simple!

Even as I write this, I am not 100% convinced I understand all the nuances of the open access debates. I wish I could say I did, but as of 2014, I don’t.

Why You Should Teach What You Know, Even If You Aren’t an Expert

So I already admitted I don’t have the level of expertise with Open Access, I have with say Web Scale Discovery, so does it mean I can’t blog about it?

Belle writes

I read a story recently about two developers. Both had roughly the same expertise and learned at about the same rate. As they improved their skills, one of the developers shared everything he learned on a blog. The blog soon became so popular that he grew a huge audience and raised thousands of dollars in a related Kickstarter campaign.
The other developer, having shared nothing he learned, had barely an audience to speak of.

It isn't an accident that I blogged my short book review on the two open access books, mentioning what I learnt from them.

I finding writing/blogging what you know or figured out is the best way to internalise what you mastered.

Many of us may feel we shouldn't blog or teach unless we are expert and trust me I understand the feeling. But I have found there is no better way to show you understand something by trying to explain to others.

It is also a way to seek out confusion, because when you explain things to others, you may start to notice you actually have gaps in your knowledge, or something you are saying is not quite convincing to yourself even if the one you are teaching keeps totally silent.

Seen from a certain angle you can see my five year old blog on librarianship has nothing more than a learning log. I started blogging about social media/library 2.0/chat etc (2009–2011) , shifted to web scale discovery (2010–2013) and started making moves towards Open Access (2012-). You can even predict with some amount of accuracy what new area I am or will be working on by looking at my blog posts from a year to two ago.

It can be scary to post about things you are well aware you haven’t mastered but even non-experts can say interesting things as you can bring to bear a different angle on things. I had interesting things to say about Open access, Google Scholar, even though I was not an expert in those areas by making a connection to other areas.

It also goes without saying that blogging what you know (say my series of blog posts on web scale discovery from 2010–2013), can also help improve your reputation and can lead to invites for talks, interviews or at the very least show evidence of expertise.

Final tips

Some things good to do on your road to expertise

  1. Identify online communities of expertise and hang out there. It could be a mailing list, facebook group etc. Lurk first if you must, but if you don’t eventually ask questions and interact your learning will be limited
  2. Identify experts — after a while you can’t help but notice the same names will tend to popup, a lot of them will be on Twitter, follow them and engage with them if you have interesting things to say.
  3. Attend conferences — Meet people in real life! Don’t just attend the talks, stay back after the talk to ask questions. Often speakers can tell you the most fascinating things in private they can’t or won’t say in public.
  4. Once you reached a certain level of expertise, you should give back by blogging, writing, speaking!

Show your support

Clapping shows how much you appreciated Aaron Tay’s story.