PinnedAshkan GolgoonGrokking in Neural Networks & Mechanistic InterpretabilitySep 28, 2023Sep 28, 2023
PinnedAshkan GolgoonNeuron to Graph: Interpreting Neurons in Large Language ModelsIn this note we focus on investigating “Neuron to Graph” an interpretability approach recently developed by Foote, Alex, et al. 2023 [1]…Nov 28, 2023Nov 28, 2023
PinnedAshkan GolgoonExplaining Neurons in LLMs with LLMsThere have been recent efforts pioneered by OpenAI where they use an LLM, e.g., GPT-4 as a tool to explain the output of another LLM, e.g…Nov 15, 20231Nov 15, 20231
Ashkan GolgoonCOPY SUPPRESSION: Mechanistic Interpretability Demystifies a Single Attention HeadThe purpose of this short note is to explore some of the recent advances of the mechanistic interpretability of large language models. In…Feb 26Feb 26
Ashkan GolgoonLearning to Rank: RankNet, LambdaRank, and LambdaMARTIn this note, we are revisiting “Learning to Rank” problem. In particular, we review the fundamental elements of famous ranking algorithms…Dec 21, 2023Dec 21, 2023
Ashkan GolgoonIs Automated Circuit Discovery Possible in LLMs?In the previous story we embarked on the mechanistic interpretability journey of LLMs. The goal of this note is to discuss a systematic way…Oct 29, 2023Oct 29, 2023
Ashkan GolgoonGPT-2 Small Mechanistic Interpretability on IOI TaskIn this short note we study recent advances in the mechanistic interpretability research at scale. In particular, we are interested in…Oct 16, 2023Oct 16, 2023
Ashkan GolgoonTabLLM: LLM Practicability on Tabular DataCan one possibly use LLMs on tabular data instead of just training a supervised model? In this short story, we study the applications of…Sep 13, 2023Sep 13, 2023
Ashkan GolgoonFinancial LLMs: FinGPT & BloombergGPTIn this short note we look at LLMs that are specific for use cases in the financial sector. In particular, we discuss BloombergGPT [1]…Sep 5, 2023Sep 5, 2023
Ashkan GolgoonUnderstanding QLoRA & LoRA: Fine-tuning of LLMsIn this short note, we gently review LoRA [1] and QLoRA [2] papers. Fine-tuning LLMs is a popular subject these days. These two papers have…Aug 22, 2023Aug 22, 2023
