Thanks for your attention to details! Fonts are different, and glyphs can vary. That’s a fact. The reason why a font x-height is literally its “x” height is that this is a lowercase letter that stands directly between the guides and has no rounded elements, which need optical compensation.
Letters “v”, “w”, “z”, and “i” (without a dot) have the same characteristics, but traditionally the term is “x-height”, not “v-height” or “z-height”.