This is a very interesting idea, and I can’t think of any real drawbacks of what you are suggesting.
As Eric Elliott said before, version numbers are for machines, so they don’t need to look “pretty” (nor short). To communicate information to humans in a concise way, we use release names instead.
Therefore I think we can dismiss those concerns about “wasting numbering space” and about the numbers easily becoming too large. I suspect those are just a manifestation of our psychological friction to change.
This new numbering scheme may look odd now, but only because we’re so used to the current one.