Sure, it is. If you’re specifically discriminating trans* women because they are trans*, it’s transphobia.
Of course you still can do it, and create those spaces. But if you discriminate people based on whether they are cis or not, you can’t at the same time claim the opposite.
You may not like being called transphobic, or think that is unfair. But it doesn’t make it less true.