Quantifying the Invisible Labor in Crowd Work

by Carlos Toxtli, Siddhartha Suri, Saiph Savage

Key takeaways:

• We found that workers spent 33% of their daily time on invisible labor (i.e., labor for which workers were NOT paid).
• If we account for the amount of time that crowd workers invest in invisible labor, which is the overhead workers endure to get crowd work done, their hourly wages go down to $2.83 from $3.76.
• We identified that the two most time-consuming categories of invisible labor revolved around managing payments and looking for good work.
• To ensure a fair and equitable future for crowd work, we need to be certain that workers are being paid fairly for all of the work they do. As a result, this research earned an impact award at CSCW’21.

Context.

The A.I. industry has created new jobs essential to the real-world deployment of intelligent systems. Such jobs typically are posted on crowdsourcing platforms and focus on having human workers complete tasks that A.I. alone cannot do (e.g., labeling data to power the machine learning models of Tesla’s autonomous vehicles).

However, it can be expensive to integrate human workers into the A.I. pipeline, which can result in A.I. companies not wanting to bother with it…