Last Fall, the Australian census website crashed within hours of launching because of heavy traffic. The following weekend, two university students at a hackathon built an unofficial census website that was able to handle 4x the traffic that had crashed the official site. The kicker here? The unofficial site was built for a grand total of ~$400, about $10 million less than what the Australian government had spent on the official site!
Think of the crowd in three worker types. First are the spammers, low quality workers trying to maximize profit and minimize effort. Next are the triers, workers who answer faithfully, but don’t always deliver the best work. Last are the producers, workers who provide quality ratings. Our challenge on the Human Computation team is to develop techniques that remove and ban spammers, mitigate the adverse effect of triers and extract signal from the producers.