Matthew KwanRe-architecting the human brainThe human brain is one of the most complex objects known to man. But the cerebral cortex — the big region on top where we store memories —…Mar 7Mar 7
Matthew KwanBuilding a Large Language Model using a HippocampusPart 4: Improving reliabilityFeb 25Feb 25
Matthew KwanBuilding a Large Language Model using a HippocampusPart 2: Algorithmic outlineJan 24Jan 24
Matthew KwanReverse-engineering the transformerI’ve spent a lot of time trying to understand the transformer model, and it only started making sense when I realized what it was trying to…Dec 31, 2023Dec 31, 2023
Matthew KwanOptimal dimensions for word embeddings, take twoIn a previous post I described a method for finding the optimal number of dimensions for word embeddings — tl;dr: increase the number of…Nov 29, 2023Nov 29, 2023
Matthew KwanWord tokenization as compressionIn a previous post I talked about finding the optimal number of dimensions for text embeddings. As part of that research I first had to…Oct 25, 2023Oct 25, 2023
Matthew KwanFinding the optimal number of dimensions for word embeddingsEmbeddings are vectors of numbers that represent a word. They have some useful properties for machine learning, such asSep 6, 2023Sep 6, 2023
Matthew KwanAI is not MathematicsThroughout history, Artificial Intelligence research has been held back by mathematicians. To get an AI paper published it needed…Aug 23, 2020Aug 23, 2020