SqueezeBits

This article provides a comparative analysis of automatic prefix caching.

This article provides a comparative analysis of speculative decoding.

This article provides a comparative analysis of multi-LoRA serving capabilities of vLLM and TensorRT-LLM frameworks.

In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability.

This article provides a comparative analysis of different parallelism strategies on vLLM and TensorRT-LLM frameworks.

In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability.

This article provides a comparative analysis of the effects of KV cache quantization on vLLM and TensorRT-LLM frameworks.

This article provides a comparative analysis of the effects of weight-activation quantization on vLLM and TensorRT-LLM frameworks.

This article provides a comparative analysis of the effects of weight-only quantization on vLLM and TensorRT-LLM frameworks.

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on performance with fixed and dynamic datasets.

This article provides a comparative analysis of schedulers in vLLM and TensorRT-LLM frameworks.

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks with various sampling methods.

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on batching configurations and thoroughly examining the effects of maximum batch size and maximum number of tokens.

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks for serving LLMs, evaluating their performance based on key metrics like throughput, TTFT, and TPOT to offer insights for practitioners in optimizing LLM deployment strategies.

Estimating the cost savings from model compression.

An introduction to tokenizers and their implications in language models.

A brief review of the research paper co-authored by Jiwon Song*, Kyungseok Oh*, Taesu Kim, Hyungjun Kim, Yulhwa Kim, and Jae-Joon Kim, published February 2024.

Clarifying the misunderstandings in AI model compression

The Blackwell GPU from GTC 2024 was astonishing. Analysis of the Nvidia GPU evolution & what it means for GPU users.

Do I need to COMPRESS my AI model? : the short answer is “YES” — and here’s why.

AI model compression for acceleration is essential. The question is HOW? Here are 4 key methodologies.