To your main points:
- Kaggle competitions measure objectively a very subjective metric: model performance. The whole point of my post is that this metric is incomplete. Of course, I agree that valuable skills can be learnt by maximizing this metric, but also bad habits can be taken: you forget the reflex to look at the big picture, and so on. If all data scientists are trained on Kaggle, the global economy will suffer. A more comprehensive education is necessary, as a ‘next step’ after Kaggle competitions.
2. The alternative metric that I propose for Startcrowd, market performance, is measured very subjectively. Market value is inherently subjective: it depends on what other people think. However, I believe that it is the right metric to look at.
3. Kaggle staff is certainly aware about their platform issues, and I am very sympathetic to their pivots (kernels and datasets), although I left Kaggle before their launch. They look like ordinary forums and repositories, without any clear vision and purpose behind.
4. Thank you for this information about the Higgs boson competition, but my main point remains: taking as input 1942 people and 35772 submissions, to get only a couple of papers at the end, it still looks like an awful yield, which Startcrowd will try to improve.
I am learning massively from Startcrowd, and it is helping my career a lot. I will definitely come back to Startcrowd and I invite everybody else to do so! It would be nice to see you there too. The more the merrier!