Taming the tongue: Unveiling and debiasing Language Models: A Deep Dive