Your experience seems similar to mine in a lot of ways. My “saying something” to my wife included the pervert bit, but I inevitably said stuff more directly about gender and how I saw myself too. I always thought of the concepts as separate — how I fantasized vs who I was/wanted to be.
This allowed me to believe, for a time, that my fantasies invalidated the other bits. That whole paraphilia/autogynephilia thing felt like an insurmountable barrier between me and what I wanted. It was, in my mind, the thing that made it possible for me to be a “weird guy” rather than a trans woman.
However, these days none of that rings true, and I am incredibly relieved to have rid myself of the vast majority of that pathological crap. Funny how being true to yourself can give you comfort and relief.
I empathize with your wish to not hurt your wife. I wanted nothing more than to stay with my wife. She was, and is, inseparable from who I am and my future. So I truly believe I could have kept my promise to not do anything she was uncomfortable with. But, I am incredibly glad I was not forced to choose between being me or being with her. Dysphoria is a boomerang — you can throw it away, but it will come back. I hope that you and your wife can find a way to break that cycle.