Sure, in the long term, that is a possibility.

I have excellent credit, but was almost denied my mortgage because I didn’t have enough credit -- or the credit I had wasn’t old enough. Over the years I had opened a couple store cards, and later closed them because it made me uncomfortable to have credit lines I had no intention of using. If they had just say there, doing nothing, there would have been no problem. I tried to reopen one of those accounts but wasn’t able to. It very seriously could have cost me my house, and put off a new attempt at buying for several years.