I was one of the primary (possibly the most veteran, now that I think about it) screen interviewers for the Facebook London office during its biggest growth phase (2012–2015). I got to see what you’ve described here first hand — both being too permissive at times and too restrictive at others. Over time, I found it was easier and easier to see the actual red flags and do high quality filtering without risking good candidates.
For instance, I initially felt that English language proficiency wasn’t a big deal. I had the opportunity to see the first, second, and third year evolutions of good coders who, despite ample opportunity to learn the language shown in the CV, really sucked at English. Waaay too much time was lost by tech leads and managers trying to buoy these engineers who, fundamentally, weren’t going to put in the effort to learn to communicate effectively. Often the root cause was a combination of hubris and lack of self-awareness. For me as an interviewer, I learned to be cautious of poor spoken English skills and to try to pick up signals about why a candidate might not be speaking English well.
Devoting energy towards making sure the screening interviewers are super calibrated and had the right disposition for the first engineering interaction with the candidates is really key. A really valuable strategy here is to leverage your most social engineers as screeners. They’ll be more resilient to rejecting many, many candidates. At FB London, I saw at least one other screener burn out from screening because he just couldn’t handle the variance and relative brutality of the role.
If you have a coder who can warmly chat with anyone and who doesn’t get demoralized by the high rejection rate of screening people, they’re a real value for screens. They’ll give your company the ability to start the engineering interaction with candidates on a high note, which makes the on-site interviews much lower stress for the candidate and makes it more likely that you can actually hire the good people you find.
By contrast, if you put a cold, only-good-code-matters interviewer as the screen, even if the person passes, you’re less likely to hire them unless they just happen to also have a similar perspective. Often the more senior and unique engineers have a more rounded perspective than this, which can leave the feeling cold at the screen even in the happy case.
Sorry this comment is so long. Fantastic write-up and something that way too many companies get sadly wrong to their own detriment.