Great post. I’ve been arguing about the importance of label quality for ML applications. The lifecycle of labels is crucial as they are used for training sets, modeling and evaluation. In my humble opinion, I see a tendency to rush the labeling step in most of the ML pipelines. Getting high quality labels is hard. And with new emerging data sets, this problem is going to get even more difficult. This is, obviously, independent of any crowdsourcing platform.