Zain ul Abideen – Medium

Zain ul Abideen

Exploring S1: Experiments and Findings

Introduction

Mar 11

Exploring S1: Experiments and Findings

Mar 11

Building a Coding agent to solve SWE-Bench

In our first attempt to solve SWE-bench problems, we ran into a lot of issues because the patches were being created before the actual…

Jan 17

Building a Coding agent to solve SWE-Bench

Jan 17

Introduction to SWE Bench & Patch Centric Approach

The Software Engineering (SWE) Bench was created to evaluate AI coding agents like Devin, which automate tasks such as bug fixes and code…

Jan 17

Introduction to SWE Bench & Patch Centric Approach

Jan 17

Q-GaLore | Memory-efficient Pre-training and Fine-tuning

Training or fine-tuning Large Language Models (LLMs) demands high-end GPUs due to massive datasets, optimizer states, and Billion…

Jul 20, 2024

Q-GaLore | Memory-efficient Pre-training and Fine-tuning

Jul 20, 2024

Coding Deepseek-V2 from Scratch in PyTorch

Implementation of Multi-head Latent Attention, Fine-Grained Expert Segmentation, and Shared Expert Isolation.

Jul 20, 2024

Coding Deepseek-V2 from Scratch in PyTorch

Jul 20, 2024

MHA vs MQA vs GQA vs MLA

Comparison of Deepseek’s new Multi-latent head attention with MHA, MQA, and GQA.

Jul 13, 2024

MHA vs MQA vs GQA vs MLA

Jul 13, 2024

Linear Rope vs NTK vs YaRN vs CoPE

Comparison of various positional embeddings.

Jul 13, 2024

Linear Rope vs NTK vs YaRN vs CoPE

Jul 13, 2024

Align Phi3 with CPO-SimPO

Align your LLM with less memory and speed efficient approach than DPO.

Jul 6, 2024

Align Phi3 with CPO-SimPO

Jul 6, 2024

Best LLM Inference Engine? TensorRT vs vLLM vs LMDeploy vs MLC-LLM

Benchmarking various LLM Inference Engines.

Jul 6, 2024

Best LLM Inference Engine? TensorRT vs vLLM vs LMDeploy vs MLC-LLM

Jul 6, 2024

MoE vs Dense vs Hybrid LLM Architectures

Train 600M MoE, Dense, Hybrid LLM Architectures.

Apr 29, 2024

MoE vs Dense vs Hybrid LLM Architectures

Apr 29, 2024

Zain ul Abideen

Zain ul Abideen

Machine Learning Engineer | I share what I learn. https://www.linkedin.com/in/zaiinulabideen/ | https://huggingface.co/abideen

Help
Status
About
Careers
Press
Blog
Privacy
Rules
Terms
Text to speech