Hey Ian,

Really nice post, thanks for sharing. Yeah, it turns out sorting by rating can be kind of complicated. :-)

When we first implemented polls on 99designs we looked into a bunch of more sophisticated ranking schemes—one of the most promising approaches we tried used wilson confidence bounds: http://www.evanmiller.org/how-not-to-sort-by-average-rating.html .. (reddit’s “best” comment ranking uses this for example)

Most polls on 99designs only have a handful of participants which can lead to surprising/unexpected ranking that’s difficult to explain, especially with more complex ranking algorithms.

That’s why we opted for a simple average rating. It has some problems, but can be explained and understood by customers the easiest. To account for the problems, we made extra sure to show the distribution and counts of votes as clearly as possible.