Sean ShenginTowards Data ScienceBenchmarking LLM Inference BackendsComparing Llama 3 serving performance on vLLM, LMDeploy, MLC-LLM, TensorRT-LLM, and TGIJun 171Jun 171
Sean ShenginTowards Data ScienceScaling AI Models Like You Mean ItStrategies for Overcoming the Challenges of Scaling Open-Source AI Models in ProductionApr 101Apr 101