I have no more problem differentiating a Muslim who wishes me harm from a Muslim who doesn’t wish me harm than I have differentiating a non-Muslim who wishes me harm from a non-Muslim who doesn’t wish me harm. Either one would be difficult to determine based on appearance alone. However, it would be a flagrant double standard to have a bias against one group due to their having some members who want to hurt me and not having a bias against another group for the same reason.
The world is full of people who want to do me harm, and who want to do you harm. In this country, that number includes far more non-Muslims than Muslims, even though they may not get the press that Muslims do when they commit crimes. Theoretically, anyone you encounter whom you don’t know well could have a desire to do you harm, and they will often look just like everyone else.
When Timothy McVeigh blew up the Murrah Building in Oklahoma City, did you form a bias against all white guys with crew cuts? When Eric Rudolph bombed the Atlanta Olympic Games and a couple of abortion clinics, did you form a bias against all evangelical Christians? When Jim David Adkisson barged into a Unitarian-Universalist church and started shooting people, did you form a bias against all conservatives who oppose liberalism? Or do you save your blanket biases for Muslims only?