Sean ShenginTowards Data ScienceBenchmarking LLM Inference BackendsComparing Llama 3 serving performance on vLLM, LMDeploy, MLC-LLM, TensorRT-LLM, and TGI10 min read·4 days ago--1--1
Sean ShenginTowards Data ScienceScaling AI Models Like You Mean ItStrategies for Overcoming the Challenges of Scaling Open-Source AI Models in Production11 min read·Apr 10, 2024--1--1