If it was “potential” as a mate vs. “potential” as a human being I would see this more charitably. However, written the way it is, if you flipped every instance of man to woman, or better yet, just “person”, you would rightly see this for the offensive it is.
Humans are just humans — don’t box them in as (losers who appear to be) having “potential” (but will never actually live up to it). You indicate that you just freed your writing, but someone who you dated earlier and broke up with might have the opinion that you “had potential” as a writer, but ultimately were a loser because you couldn’t come to it.
Anyway, don’t box people in with labels. It’s offensive no matter what gender it is applied to.