Kyle, not to say that your experience is meaningless, but it may not be giving you the right odds to make this decision with.
There isn’t an easy solution, and I think the best solution is one you mentioned: we should be working with people we know and trust.
In the meantime, your solution about not having one-on-ones with a group you’re throwing a blanket of mistrust over reminds me of “Not All Men" and the stupid M&Ms bowl argument. Part of (if not most of) being a good person is assuming risk to help a greater cause, be it your job or community. It is assuming good will on the part of others, knowing full well that good will is not universal.
At worst, you can record one-on-ones so you have strong evidence of your (I assume) good behavior.