Hello, world

Caliper is a platform to evaluate practical machine learning (ML), artificial intelligence (AI), and data science (DS) skills when recruiting ML/AI/DS engineers and scientists.

Why?

We believe the process of hiring in ML/AI and data science is broken.

What these positions typically require are practical data science skills — formalizing a problem, deciding what data to collect and annotate, munging the data into an appropriate format, visualizing for outliers and biases, picking an appropriate ML model, training it, defining and computing an appropriate evaluation metric, analyzing and visualizing results, understanding the biases exhibited in the results, searching over reasonable hyperparameters, iterating over model architecture, iterating over data curation/annotation, iterating over problem formulation.

And what do interviews for these positions look like?

A bipolar mix of CS puzzles (“write a function to serialize a balanced binary tree”) and ML theory (“How would you prove logistic regression cross-entropy loss is convex?”).

It’s like holding tryouts for a football team entirely based around a player’s understanding of ball aerodynamics. Sure, there’s likely a (weak) correlation between what we care about and we are measuring, but that’s not the best way.

With the growing number of ML/AI engineer and data science positions in companies as well as relevant candidates — we believe it’s useful to be able to evaluate a candidate’s practical ML/AI/DS skills in a scalable way, at a screening stage, even before they come for an onsite interview.

So what does Caliper offer?

Caliper offers recruiters and hiring managers the ability to

• pick from our collection of prepared challenges (in computer vision, natural language processing, autonomous driving and other domains) or create their own challenge around data from their domain,
• invite candidates (from a specific list or the general public) to participate in this challenge, including preparing a model for this task, submitting their code and predictions,
• compare various candidates, inspect their code (if needed), and then decide to interview (and eventually hire!) them.

Guiding Principles

• Portfolio not prestige: We believe AI hiring (and hiring in general) should be based on a judgement of skills relevant to the job and not on signaling or gatekeeping mechanisms, or measures susceptible to subjective biases.
• Low barrier to entry: We believe candidates should be able to showcase performance on a small number of AI/DS tasks that are neither burdensome (in terms of time commitment) nor exclusionary (in terms of computational or other resources required to participate).
• Gauge not rank: Our goal is not to create yet another test to be optimized. Hence, our challenges are binary ‘completed or not’ (e.g., model performance above a reasonable baseline or not) and meant only to create a scalable pre-on-site pipeline. We leave fine-grained judgement to the phone or on-site interviews.

If you are a recruiter or hiring manager looking for people in ML/AI/DS, check out caliper.ai and get in touch!