Sign in Get started

SqueezeBits Team Blog

SqueezeBits Team Blog

Latest

[vLLM vs TensorRT-LLM] #6. Weight-Only Quantization

[vLLM vs TensorRT-LLM] #6. Weight-Only Quantization

This article provides a comparative analysis of the effects of weight-only quantization on vLLM and TensorRT-LLM frameworks.

Oct 31

Scrum Helper: SlackBot으로 Daily Scrum에 Jira 티켓 연동하기

Scrum Helper: SlackBot으로 Daily Scrum에 Jira 티켓 연동하기

안녕하세요! 스퀴즈비츠의 새내기 PM 김사랑입니다. Product Manager를 줄여서 말하는 PM은 제품을 기획하고 스프린트를 관리할 뿐만 아니라, 좋은 제품이 나올 수 있도록 일하기 좋은 환경을 만드는 역할도 합니다. 이번 글에서는 후자의…

Oct 29

[vLLM vs TensorRT-LLM] #5 Dynamic Sequence Lengths

[vLLM vs TensorRT-LLM] #5 Dynamic Sequence Lengths

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on performance with fixed and dynamic datasets.

Oct 29

[vLLM vs TensorRT-LLM] #4 Which Scheduler Wins? 🔥

[vLLM vs TensorRT-LLM] #4 Which Scheduler Wins? 🔥

This article provides a comparative analysis of schedulers in vLLM and TensorRT-LLM frameworks.

Oct 23

[vLLM vs TensorRT-LLM] #3 Understanding Sampling Methods and Their Performance Impact

[vLLM vs TensorRT-LLM] #3 Understanding Sampling Methods and Their Performance Impact

Large Language Models (LLMs) generate text by predicting the next token based on the context provided according to the probability…

Oct 17

[vLLM vs TensorRT-LLM] #2. Towards Optimal Batching for LLM Serving

[vLLM vs TensorRT-LLM] #2. Towards Optimal Batching for LLM Serving

In our previous article, we compared vLLM and TensorRT-LLM under default configurations and specific constraints, providing insights into…

Oct 10

[vLLM vs TensorRT-LLM] #1. An Overall Evaluation

[vLLM vs TensorRT-LLM] #1. An Overall Evaluation

vLLM and TensorRT-LLM are two leading frameworks for efficiently serving Large Language Models (LLMs). vLLM is a fast, user-friendly…

Sep 30

About SqueezeBits Team BlogLatest StoriesArchiveAbout MediumTermsPrivacyTeams