Job Interviews At Big Corps Are Broken.
Or originally titled “Why Phone Screens for Hiring are Broken — Real Life Tests and Results.”
(This post is specifically about hiring in the tech industry.)
Context: At various companies that I’ve worked for, in the majority of cases, I’ve been an interviewer for potential new hires. These processes were very typical, phone screens as first filters, and then on-premises with various team members at different levels.
Certain roles are fairly easy to filter out at the initial screen level. For example, development roles are very binary. Either you can code or you don’t. But what about roles like design and product? or leadership roles? How do you quantify these? I left @WalmartLabs over 2 years ago. On leaving, I got the usual recruiter calls. I took a phone screen and got me thinking about this problem.
Is the current use of phone screens for hiring broken? Phone screening is not fun. It takes people off their regular work schedule to speak with a stranger with minimal social interaction. No one jumps for joy when ‘picked’ to do a phone screen. If a company is on a hiring spree, the volume can be significant.
For the candidate, phone communication is probably the worst platform to impress (unless you are interviewing for a sales role). The whole process is hit and miss. The mood of the screener can have a huge impact on the candidate’s career. Did the screener have the morning coffee? Was he/she arm twisted to take the call? (Personally, I’ve been guilty of rushing through phone screens to make meeting appointments.) Should screeners have accountability?
Phone screeners are gatekeepers for the company (culture, first impression, bar raiser, etc). Yet, the task (in general) seems to be filled by staff availability and less about suitability. Typically, it is optimized to save time and not necessary for the actual problem — getting the right candidate for the company. That was my theory. So I decided to do some real-life tests to get more insights.
The Tests
Disclaimer: Since there are many variables, it will be foolish to assume these are realistic scientific tests, but the results were surprising.
The constants are pay (if asked), resume, and cover letter (where required — basically reuse of the same template). I would apply for roles that realistically match my experience (product management vs development). I would choose a diverse set of companies (and avoid any conscious bias — companies I may not have considered for a real job search).
If invited in for a face to face interview, I would decline (and reveal my agenda where appropriate).
(Sidenote: My tests also revealed how broken the job search processes are. It is a game of how to get a resume in front of the right people — there are probably many potentially good candidates that never got past this stage. Playing the ‘game’ seems to be an art.)
The Results
The following background of the screeners is based on info found on LinkedIn. I want to point out that these results are not a criticism of the company or the screeners (whom in general, were extremely friendly).
Big Corp 1:
Phone screen 1: 1-hour video conference with an internal recruiter. Good info gathering and ‘sell’ the role
PS 2 : 45 mins with a Director of Product (previous experience starting own company. Needed life balance and joined big corp) Result: progress to next phone screen.
PS 3: 45 mins with Director of Product (one previous exp working for a large ‘startup’* in prod/dev role otherwise all large corps). Questions were all over the place — it seems he read too many ‘how to be a PM’ books. Result: Failed.
* IMHO ‘Startup’ is such an abused term. It is liberally used to give an impression of agile and entrepreneurial. It is a term I see used in many of the big corp job ads in this context. Many startups are far from agile. The reality is very different. Most are grinding with minimal resources.
Big Corp 2:
PS 1: 30 mins internal recruiter. Usual info gathering on education, why, etc. Obviously following an internal script. (The only one that asked about University and degree — seem to be an internal filter)
PS 2: 1 hour with Senior Product Lead (he co-founded a startup that was acquired by Big Corp 2). Super focused product questions. The only big corp in the tests that covered blue-sky thinking. Result: Call back to come on-site. Willing to wait 6 months if I change my mind.
Big Corp 3:
PS1: 15 mins with an external recruiter (turn around time was ~1 hour from me sending in the application). Casual observation: external recruiters are optimized and incentivized to get you hired. Internal recruiters came across as ‘just doing their job’ — much slower.
PS2: 1 hour with Head of Business (recent promotion, no previous start-up experience). Very academic approach — submit a paper of answers to questions before the phone screen. Phone conversation gave me a bad impression of the company (lack of experience of the interviewer in the product space — especially at a senior level). I called the phone screen off after appox 20 mins. Result: Failed (Recruiter called back to apologize and aware of the issue. The disadvantage of external recruiters — lack of control on matching screeners.)
Big Corp 4:
PS1: 30 mins with an internal junior recruiter (recently changed roles). On mute most of the time while I regurgitated my resume. Attempt to sell me GM role that I did not apply for. After the call, it was such a bad experience, I actually withdrew the application. (Curious how internal recruiters’ success are measured — I sense an opportunity to disrupt here.)
The next day was told my resume was passed around the company. Withdrew my application again from their internal job dashboard. A week later got a call (from a different internal recruiter) for a phone screen. Note: my application was still at the ‘candidate withdrew’ state.
PS2: 35 mins with Head of Product (no previous start-up experience). Very rushed. 15 mins into the call, realized that it was possibly a competitive information gathering call (questions was focused on specific project implementation details and less about my experiences). At this realization, I switched on my ‘dumb’ mode and held back. Result: Failed
Big Corps Summary: Similar results with four other Big Corps. If the screener had previously co-founded a start-up or worked for an early-stage startup, I would pass to the next stage (either another phone screen or on-site) If the screener had no previous startup experience, then I would fail. The application job titles (ranging from director to head of product) had no noticeable bias to the results.
10 applications to big corps:
4 failed
4 passed
1 never heard back
1 email reply ‘we’ll keep you on file’
Conclusion
My personal experience and pattern seem to suggest that if you want to hire candidates with ‘startup’ experience, have screeners with actual start-up experience. There is possibly an unintentional bias here — startup people hiring startup people.
For example, it will be interesting to run similar tests for candidates with MBAs. Will we see patterns of MBAs hiring MBAs? Will a screener with start-up experience unconsciously bias against the MBAs?
The neck of the hiring conversion funnel (which is where I place the screeners) is critically important. If not optimized correctly, the ideal candidate may have leaked. In reality, screeners (gatekeepers) may have a bigger impact in defining a company’s future and culture than other initiatives. (One possibility is to track a screeners’ history of results. The rate of passed vs hires vs failed. Tracking these data points may identify the right screeners.)
The counter-argument is that maybe the screeners did do the job right — I was correctly filtered out. I cannot disagree with that. My conclusion and result are based on a very small dataset. It will be interesting to revisit this with a bigger dataset. I am aware there are new startups (Comparably etc) using machine learning (ML) to provide more insights into the career space. Maybe we will have a future where ML will replace the concept of phone screening and do a better job of unbiased filtering.
