This is not totally true. Actually, it’s only true for a same Version Number.
For instance, while you work on v4.2, you submit 3 builds with 1, 2 and 3 build numbers.
Then, when you work on v4.3, you can also submit builds with 1, 2, or 3, as build number.
More info in Technical Note TN2420