PinnedNeuron to Graph: Interpreting Neurons in Large Language ModelsIn this note we focus on investigating “Neuron to Graph” an interpretability approach recently developed by Foote, Alex, et al. 2023 [1]…Nov 28, 2023Nov 28, 2023
PinnedExplaining Neurons in LLMs with LLMsThere have been recent efforts pioneered by OpenAI where they use an LLM, e.g., GPT-4 as a tool to explain the output of another LLM, e.g…Nov 15, 2023A response icon1Nov 15, 2023A response icon1
COPY SUPPRESSION: Mechanistic Interpretability Demystifies a Single Attention HeadThe purpose of this short note is to explore some of the recent advances of the mechanistic interpretability of large language models. In…Feb 26, 2024Feb 26, 2024
Learning to Rank: RankNet, LambdaRank, and LambdaMARTIn this note, we are revisiting “Learning to Rank” problem. In particular, we review the fundamental elements of famous ranking algorithms…Dec 21, 2023Dec 21, 2023
Is Automated Circuit Discovery Possible in LLMs?In the previous story we embarked on the mechanistic interpretability journey of LLMs. The goal of this note is to discuss a systematic way…Oct 29, 2023Oct 29, 2023
GPT-2 Small Mechanistic Interpretability on IOI TaskIn this short note we study recent advances in the mechanistic interpretability research at scale. In particular, we are interested in…Oct 16, 2023Oct 16, 2023
TabLLM: LLM Practicability on Tabular DataCan one possibly use LLMs on tabular data instead of just training a supervised model? In this short story, we study the applications of…Sep 13, 2023Sep 13, 2023
Financial LLMs: FinGPT & BloombergGPTIn this short note we look at LLMs that are specific for use cases in the financial sector. In particular, we discuss BloombergGPT [1]…Sep 5, 2023Sep 5, 2023
Understanding QLoRA & LoRA: Fine-tuning of LLMsIn this short note, we gently review LoRA [1] and QLoRA [2] papers. Fine-tuning LLMs is a popular subject these days. These two papers have…Aug 22, 2023Aug 22, 2023
