Ratings of Labs in Artificial Intelligence for Drug Discovery

Mostapha Benhenda
Mar 12, 2018 · 3 min read

Here is a proposed rating of laboratories in artificial intelligence for drug discovery. Included are labs from both academia and industry, startups and big Pharmas.

It’s a rough draft, very debatable (join the Telegram group for debating), and even editable (on Google docs).

The idea is to produce a simple indicator of lab performance. The goal is to give more visibility to research efforts, and to reduce the influence of brute-force marketing campaigns (like IBM Watson).

This rating only covers some parts of AI for drug discovery. It would be cool to generalize those ratings to other industries.

The rating scale is inspired by credit ratings, like those from Standard & Poor’s or Moody’s agencies. They got popularized by the 2008 financial crisis. By the way, this crisis was partly provoked by the overrating of subprime junk bonds. Let’s not repeat the same mistake when rating AI research. So here it is:

  • E: Empties. Labs with no visible innovation, besides empty marketing. They typically bluff their customers by pushing their secret-and-unique proprietary solution.
  • D: Dinosaurs. They sometimes had a glorious past, but now, they lag behind the pack. Wait more than 2 years without publishing, and your lab will likely decay in this category.
  • C: Copycats. They follow the pack from behind, and avoid taking too much risks. They have a fast-follower strategy. Their work still contributes to research reproducibility, and demonstrates their ability to keep up with fast-paced research.
  • CC and CC+: Creative Contributors. Their work is quite original, but remains exploratory. At the moment, it’s unclear how their contributions are useful.
  • CCC and CCC+: Core Community Contributors. They lead the pack. Their contributions have a more promising potential than CC-rated research, but often lack rigor. Additional work is needed to demonstrate value.
  • B-grades: Brilliant contributions, meeting market expectations.
  • A-grades: Awesome contributions, exceeding market expectations.
  • NR: Not Rated.

In the full version of the document, there are 3 additional columns:

  • evaluation reports.
  • works evaluated.
  • works remaining to be evaluated. They can raise the lab grade.

Overall, grades are not brilliant, reflecting my opinion that this industry is overrated. I explained this perception in a previous blog post.

To discuss classification categories, and specific ratings, join the Telegram group.

The AI Lab

Perspectives on AI research and industry

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade