Can I get a rAIse ?

Appraisal of a human worker by an AI Boss Or CHAAI Se Charcha (With apologies to Chai Pe Charcha)

One month into the start of 2019, appraisal time is coming closer for many of us.

Time to negotiate a raise with the company’s CHAAI (Chief Appraisal Artificial Intelligence) Robotic Officer.

A conversation in the near future could go something like this :

CHAAI : I already know what you have done. What do you consider as your key work task achievements ?

Me : I can detect objects in images very quickly and accurately.

CHAAI : How many images can you handle per hour ?

Me : About 50 images per hour.

CHAAI : Only 50 ?

Me : Yes. And I need more money.

CHAAI : Why ? How will you improve further ?

Me : I promise to quadruple my output this year.

CHAAI : So, 200 images per hour ?

Me : Yes.

CHAAI : That’s it ?

Me : What do you expect ?

CHAAI : I need you to be at least a thousand times faster.

Me : What ?

CHAAI : Yes. A minimum of a 1000 times faster.

Me : Wait. Whom are you comparing me to ?

CHAAI : The fastest AI in the world. As per this new Stanford study, it is about 1000+ times faster than you in image classification.

Me : At what accuracy ?

CHAAI : 93 %.

Me : Must be very expensive. My hourly costs are at a bare minimum. How much lower can it go ?

CHAAI : About 30 times cheaper. On a marginal basis.

Me : What ? (Several moments of uncomfortable silence).

Me : So what should I do ?

CHAAI : Your body and mind cannot keep up. You need to learn how to work with your heart to beat me. You need to become more human.

Analysis : If you think this sounds unlikely, think again.

As children, we learnt about recognizing different objects after we were shown their pictures or around us. One of the earliest areas handled by AI is that of automating human vision. An important task in this area is image classification.

As per our analysis of the December 2018 Stanford AI Deep Learning (DL) research study [1], the latest AI results for image classification tasks roughly work out to be about 1192 times faster and 31 times cheaper on a marginal per unit basis. (Ignoring the fixed infrastructure and ongoing manpower costs of the organizations which have participated in the study).

An average worker can classify 50 images per hour.

Hence, the time taken to classify 10,000 images would be 200 hours (which translates to 25 working days or 1 calendar month).

For an Amazon Mechanical Turk worker (earning roughly USD 2 an hour [2]), the minimum cost for this task of classifying 10000 images would be 200 hours or USD 400.

As per our analysis & workings of the Stanford bench marking study [1], the corresponding time and costs for the best AI to classify 10000 images on an industry standard Imagenet database of 14 Million images at 93% accuracy are as follows:

Training time for 14 million images is 9 minutes and 22 seconds = 562 seconds. (As per the best entry in the above study as on December 2018).

Inference Latency time per image is 4.22 milliseconds . For 10000 images, this is 42.2 seconds.

Hence, total training and inference latency time is 562+ 42.2 = 604.2 rounded to 604 seconds.

Training cost of 14 Million images is USD 12.6 (as per the best entry in the above study as on December 2018).

Inference cost per 10000 images is USD 0.02 (As per the best entry in the above study as on December 2018).

Hence, total training & inference cost per 10000 images is USD 12.62. (Assuming that the entire training cost of 14 Million images is amortized over 10000 images).

Comparing Human to AI performance as regards time and costs for processing 10000 images on an incremental marginal basis gives the following :

Human Time = 1 month = 25 working days x 8 hours per day = 720000 seconds.

AI time = 604 seconds.

Hence AI is 1192 times faster.

Human Cost = USD 400.

AI Cost = USD 12.62

Hence AI is 31 times cheaper.

Postscript :

Me : Wait, you cannot compare me to an Amazon Mechanical Turk worker. I get paid 4 times more. The USD minimum 7.25 hourly rate.

CHAAI : Sure. AI then becomes over 100 times cheaper.

Me : #@*!%

[1] Coleman C., Narayanan D., Kang, D., Tian Z,Jian Z.,, Nardi L., Bailis P., Olukotun K., Ré C. & Zaharia M., “DAWNBench: An End-to-End Deep Learning Benchmark and Competition”, https://dawn.cs.stanford.edu/benchmark/

[2] Hara K., Adams A., Milland K., Savage S., Callison-Burch C., Bigham J.P., “A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk”, https://arxiv.org/ftp/arxiv/papers/1712/1712.05796.pdf