I would have to disagree. Islamophobia, like homophobia or xenophobia, does not belong to the traditional definition of phobia. In these cases phobia more describes something akin to “aversion.” Whether it was a good idea for any of those words to acquire those particular, somewhat confusing, meanings is not for me to say. But rather islamophobia describes dislike of or prejudice against Islam or Muslims, especially as a political force. It doesn’t describe a fear, but rather a specific, prevalent form of xenophobia/racism/prejudice (that has its own name specifically because of its prevalence). (And before you say that Islam is not a race, that is true, but islamophobia is a very racialized phenomenon, as evidenced by the many Middle Eastern people with no outward sign of adherence to Islam who are caught up in the problem.)
Calling someone islamophobic is not like shaming someone for an uncontrollable fear. It is acknowledging the patterns of behavior and the words that that person is using to exhibit prejudice against Islam and Muslims. I agree with you that religions are not beneficial to the well-being of the world at large, but islamophobia — the prejudice that people show towards Islam — also affects Muslims and Middle Easterners in general in very real, personal ways (attacks, hate speech, being called on to answer for terrorists’ crimes) and so it must be called out for what it is and not allowed to become normalized.