I am a self-published writer and I agree with what you are saying. I am there. I am here. World-building is my forte. World-building is why I write in the first place.
There are times when I wonder if people actually care about what I am trying to accomplish. Some might, I know. I have to believe that. But it is hard to focus on that because the world is so saturated with content, most of it ‘garbage’ (Sturgeon’s Law.) How can one find the gems? How do I know if my own work is a ‘gem?’ I believe it to be, of course, but that does not mean that others will. Getting support from the consumers is mind-bogglingly difficult. I can find other authors with the same plight. Finding readers and people willing to try is a challenge.
I don’t know how to fix it. I don’t know whether it should be fixed. All I know is that I love what I do.