SqueezeBits

In this article, we introduce how to run LLMs efficiently on Apple Silicon with disaggregated inference technique.

This article is about an open-source library for direct conversion of PyTorch models to TensorRT-LLM.