Good post, accurate points. Indeed curious that Tensorflow is unoptimized for GPU. There is a possibility that the move to GPU was at odds with how Google ran its data-centers. That is, employing middle of the pack processors with low energy footprint. In fact, a recent tweet by Andrew Ng about Google exploring GPUs is quite revealing. Baidu had no issue acquiring thousands of GPUs, Google in contrast likely balked at the idea.
A big company like Google doesn’t really care about building solutions for niche applications. There will always be an opportunity for small players to survive so long as its below the noise level. There is also the question of access to data that likely certain markets will not allow Google to commoditize.
Yes, it’s a difficult market but it isn’t any different from when Microsoft controlled the desktop. Innovation will always happen elsewhere.