Dimension Reduction Without Reducing Your Professional Integrity
The Unexpected Twist in a Prestigious Medical Research Firm Interview
Recently, I found myself at the final stage of a hiring process with a renowned medical research firm. Having received positive feedback in the earlier rounds, I was optimistic and eager to showcase my expertise. However, the final interview took an unexpected and challenging turn that left me re-evaluating the entire experience.
The interviewer’s demeanor was markedly different from what I had encountered previously, almost bordering on rude. He grilled me intensely about my career choices and past projects in biological research, with a critical tone that was hard to ignore. When I shared my experience with a liquid biopsy startup, the conversation quickly became highly technical. More details can be found here.
I explained how we managed an overwhelming number of features through dimensional reduction using a process called saturation. The interviewer interrupted, questioning why we hadn’t used Principal Component Analysis (PCA).
My honest response — that PCA hadn’t worked for us — was met with skepticism. “You didn’t use it properly,” he asserted.
To defend our approach, I highlighted the limitations of PCA in biological contexts. PCA essentially performs a general rotation of the data, functioning as a linear transformation that preserves both orientation and the Euclidean metric. However, this method is only valid when the data conforms to Euclidean principles, which is often not the case in biological systems.
- PCA’s Assumption of Euclidean Structure
PCA assumes that the data has an inherent Euclidean structure, a condition that is rarely met in biological systems. Biological data typically exhibit complex, non-Euclidean relationships that cannot be accurately represented by simple linear transformations.
- The Issue with Linear Combinations
In biological contexts, linear combinations of variables often lack meaningful interpretation. For instance, what practical insight can be gained by adding twice the body temperature to three times the blood pressure level? Such combinations are abstract and fail to provide actionable information relevant to biological phenomena.
- Non-linear Feedback Loops
Many biological systems are characterized by non-linear feedback loops, which PCA is ill-equipped to capture. I used a saturation example to illustrate this: at low doses, there is little to no reaction, but beyond a specific threshold, saturation occurs, dramatically slowing the reaction rate. This behavior mirrors an activation function in neural networks, highlighting the data’s inherent non-linearity. Accurately modeling these non-linear responses, such as enzyme kinetics or hormone levels, is essential for making reliable predictions.
- Sensitivity to Non-linear Scaling
PCA is not invariant to non-linear scaling or changes in units. For example, a log transformation could significantly alter PCA results, leading to different interpretations.
- Broader Implications
This non-linear behavior is not confined to a single scenario. Similar patterns emerge in studies of opioid dosages, insulin response, and other pharmacokinetic phenomena. For instance, the relationship between drug dosage and effect is rarely linear, underscoring the importance of understanding these nuances for effective treatment planning.
It is worth noting that other metrics, such as the Manhattan or maximal metrics, do not allow for geometrical interpretation. This is because they are not derived from any inner product, which is necessary for defining an angle between two vectors.
To my surprise, my detailed explanation fell flat. The interviewer retorted, “I see your point, but I know that linear methods are working all across biology. I’d rather hire a mathematician who can explain why it works, not why it doesn’t.”
Needless to say, I didn’t get the job. While disappointed, I left feeling confident in my answers. If my honest, scientifically grounded perspective wasn’t appreciated, perhaps this wasn’t the right fit for me after all.
This experience taught me a valuable lesson: interviews aren’t just about showcasing your knowledge but also about aligning with the company’s methodologies and perspectives. While it’s crucial to stand by your expertise, it’s equally important to gauge the receptiveness of your audience. In this case, the interviewer’s rigid adherence to traditional methods overshadowed the opportunity for an innovative approach to complex biological data.
Reflecting on the interview, I realized the importance of finding a workplace that values critical thinking and the ability to challenge established norms when necessary. The right opportunity should not only appreciate my technical skills but also my willingness to think outside the box and apply unconventional solutions to unique problems.
In the end, not getting this job might have been a blessing in disguise. It’s a reminder that the right fit is not just about the prestige of the firm but also about mutual respect and a shared vision for innovation and progress. Moving forward, I’ll be more mindful of aligning with organizations that value a holistic and open-minded approach to problem-solving in the ever-evolving field of biological research.
