I agree. I know they offer big downloadable database dumps of all their data. I tried working with them in the past and it was quite a process. I’m not up-to-speed on what they currently offer.
However, it could be a great business model for them to sell access to their data to companies for commercial use. Offer a robust API and clean downloads of data sets in a wide variety of forms —let people use this freely for research and non-profit, but I bet lots of companies would pay for it. This is not true not only of the user-generated content (e.g. to construct a concept graph) but also access to their traffic data (e.g. what articles are viewed most often, what articles are trending right now, etc).