Amina ShabbeerinTowards AIDeepspeed ZeRO-DP: distributed training for large modelsDeepspeed’s ZeRO (Zero Redundancy Optimizer) is a distributed training framework with a number of optimizations to easily train large deep…Jun 6Jun 6
Amina ShabbeerinTowards AIImplement Tensor Operations With PyTorch einsum: Basic to Self-attentionCode examples using einsum and visualizations for various tensor operationsMay 22May 22
Amina ShabbeerinTowards AILORA: Low-Rank Adaptation of Large Language ModelsIntroduction: This article explains LoRA [1], a parameter-efficient method for fine-tuning models to solve downstream tasks and the…Jul 31, 2023Jul 31, 2023
Amina Shabbeer2-minute Intro to JAXJAX is a machine learning research library that gives you hardware acceleration right out the box.Feb 27, 2023Feb 27, 2023