Benjamin Marie – Medium

Benjamin Marie

Pinned

Published in

TDS Archive

Run Mixtral-8x7B on Consumer Hardware with Expert Offloading

Finding the right trade-off between memory usage and inference speed

Jan 11, 2024

Run Mixtral-8x7B on Consumer Hardware with Expert Offloading

Jan 11, 2024

Pre-Training Encoders: Bidirectional MLM Remains Important

CLM, MLM, or Both?

4d ago

Pre-Training Encoders: Bidirectional MLM Remains Important

4d ago

Supervised Fine-Tuning Improves LLM Reasoning at the Cost of Other Skills

RL vs SFT

6d ago

Supervised Fine-Tuning Improves LLM Reasoning at the Cost of Other Skills

6d ago

LLM Recovery without Checkpoints

A breakthrough for multi-GPU settings

Jun 27

LLM Recovery without Checkpoints

Jun 27

RAG with Qwen3 Embedding and Qwen3 Reranker

How to use embedding and reranker models to efficiently retrieve only the most relevant chunks or documents given a user query

Jun 26

RAG with Qwen3 Embedding and Qwen3 Reranker

Jun 26

No Verifier? No Problem: Reinforcement Learning with Reference Probabilities

RLPR: Extrapolating RLVR to General Domains without Verifiers

Jun 25

No Verifier? No Problem: Reinforcement Learning with Reference Probabilities

Jun 25

How Well Does Qwen3 Handle 4-bit and 2-bit Quantization?

Let’s review Qwen3 and check which one you should use

May 3

How Well Does Qwen3 Handle 4-bit and 2-bit Quantization?

May 3

Published in

TDS Archive

2-Bit VPTQ: 6.5x Smaller LLMs While Preserving 95% Accuracy

Very accurate 2-bit quantization for running 70B LLMs on a 24 GB GPU

Jan 31

2-Bit VPTQ: 6.5x Smaller LLMs While Preserving 95% Accuracy

Jan 31

OLMo 2 and Tulu 3 Quantized

State-of-the-art LLMs on your computer

Dec 3, 2024

OLMo 2 and Tulu 3 Quantized

Dec 3, 2024

Published in

Stackademic

Hymba: Combining Attention Heads and SSM Heads within the Same Layer

Faster and better LLMs

Dec 3, 2024

Hymba: Combining Attention Heads and SSM Heads within the Same Layer

Dec 3, 2024

Benjamin Marie

Benjamin Marie

Ph.D, research scientist in NLP/AI. Exclusive articles and all my AI notebooks on https://kaitchup.substack.com/

Following

Help
Status
About
Careers
Press
Blog
Privacy
Rules
Terms
Text to speech