https://blog.squeezebits.com The official SqueezeBits Tech blog covering AI model compression and optimization, LLM and Edge AI systems, and deep technical insights into products like Yetter, Fits on Chips, and OwLite, along with research, engineering, and development updates. Mon, 10 Apr 2023 03:50:02 GMT inblog Tue, 14 Apr 2026 05:08:30 GMT https://image.inblog.dev?url=https%3A%2F%2Fsource.inblog.dev%2Fog_image%2F2026-01-07T04%3A48%3A26.787Z-d4f56888-0576-4c57-bdc3-f6f129134bb1&w=1920&q=85 https://blog.squeezebits.com https://blog.squeezebits.com/vllm-korea-meetup-2026-en Check out highlights from the 2nd vLLM Korea Meetup — open-source use cases and real-world production examples that showcase vLLM's technical maturity! Tue, 14 Apr 2026 08:54:00 GMT https://blog.squeezebits.com/vllm-korea-meetup-2026-en Event Goeun Kang https://blog.squeezebits.com/gtc-conference-booth-review-en Sharing GTC 2026 insights, which is the Largest AI Industry Conference for developers! If you’ve ever wondered what it’s like for an AI startup to run a booth at such a massive event, you won’t want to miss this! Fri, 27 Mar 2026 09:11:00 GMT https://blog.squeezebits.com/gtc-conference-booth-review-en Event Goeun Kang https://blog.squeezebits.com/reliable-synthetic-data-physical-ai-production Explore why Physical AI deployment needs synthetic data at scale with Squeezebits' research and discover how to overcome inference bottlenecks to accelerate Roboost Agent. Wed, 11 Mar 2026 06:49:00 GMT https://blog.squeezebits.com/reliable-synthetic-data-physical-ai-production Research Jongho LeeDaehyun AhnYeonjoon JungSemin KimSeungryeol Kim https://blog.squeezebits.com/reliable-synthetic-data-physical-ai Scaling Physical AI requires reliable synthetic data. Learn how RoBoost Agent integrates NVIDIA Cosmos to transform world models into trustworthy data engines for robotics and autonomous driving. Wed, 25 Feb 2026 08:22:00 GMT https://blog.squeezebits.com/reliable-synthetic-data-physical-ai Research Daehyun AhnJongho LeeYeonjoon JungSemin KimSeungryeol Kim https://blog.squeezebits.com/intel-gaudi-hands-on-workshop-en A recap of the Intel® Gaudi® hands-on workshop co-hosted by SqueezeBits and Lablup. AI model compression, fine-tuning, and vLLM serving on Gaudi® hardware with Backend.AI. Wed, 07 Jan 2026 07:34:00 GMT https://blog.squeezebits.com/intel-gaudi-hands-on-workshop-en Event Goeun Kang https://blog.squeezebits.com/introducing-atom-max-npu Introducing ATOM™-Max, rebellions’ next-generation NPU designed for high-performance AI inference. Learn how its runtime, profiling tools, and PyTorch-native integrations enable developers to run and serve models efficiently without sacrificing usability. Wed, 24 Dec 2025 08:30:00 GMT https://blog.squeezebits.com/introducing-atom-max-npu Tech Insight Huijong Jeong https://blog.squeezebits.com/vllm-hands-on-workshop-with-rebellions-squeezebits-en Rebellions and SqueezeBits Co-Host a vLLM Hands-on Workshop: Workshop Highlights, PyTorch Best Practices, Performance Optimization, and Developer First-Hand Tips Wed, 10 Dec 2025 03:00:00 GMT https://blog.squeezebits.com/vllm-hands-on-workshop-with-rebellions-squeezebits-en Event Goeun Kang https://blog.squeezebits.com/77516 Explore how the Yetter Inference Engine overcomes the limitations of step caching and model distillation for diffusion models. We analyze latency, diversity, quality, and negative-prompt handling to reveal what truly matters for scalable, real-time image generation. Fri, 31 Oct 2025 08:54:00 GMT https://blog.squeezebits.com/77516 Product Yeonjoon Jung https://blog.squeezebits.com/intel-gaudi-gemm-attention-performance Explore how Intel’s new Gaudi-3 compares to Gaudi-2, NVIDIA A100, and H100. We analyze real-world GEMM efficiency, attention performance, and LLM serving results to uncover what truly matters for AI inference and training workloads. Tue, 28 Oct 2025 00:30:00 GMT https://blog.squeezebits.com/intel-gaudi-gemm-attention-performance Tech Insight Taesu Kim https://blog.squeezebits.com/yetter-genai-api-service Meet 'Yetter': the generative AI API service built for speed, efficiency, and scalability. Powered by our optimization inference engine, it delivers reliable image, video, and future LLM services at a fraction of the cost. Thu, 02 Oct 2025 04:00:00 GMT https://blog.squeezebits.com/yetter-genai-api-service Product Seungryeol Kim https://blog.squeezebits.com/guided-decoding-performance-vllm-sglang The guide to LLM guided decoding! This deep-dive benchmark compares XGrammar and LLGuidance on vLLM and SGLang to help you find the optimal setup for generating structured output based on your use case. Tue, 16 Sep 2025 05:59:00 GMT https://blog.squeezebits.com/guided-decoding-performance-vllm-sglang Tech Insight Eunik Park https://blog.squeezebits.com/disaggregated-inference-on-apple-silicon-npu-prefill-and-gpu-decode-67176 In this article, we introduce how to run LLMs efficiently on Apple Silicon with disaggregated inference technique. Tue, 26 Aug 2025 00:00:00 GMT https://blog.squeezebits.com/disaggregated-inference-on-apple-silicon-npu-prefill-and-gpu-decode-67176 Tech Insight Jiwoong Choi https://blog.squeezebits.com/efficient-ai-study-meetup-by-squeezebits-en Efficient AI Study & Meetup recap: SqueezeBits' community study on AI model compression, featuring paper reviews, participant interviews, and networking from the offline meetup. Wed, 20 Aug 2025 08:27:00 GMT https://blog.squeezebits.com/efficient-ai-study-meetup-by-squeezebits-en Event https://blog.squeezebits.com/vocabulary-trimming-methods Trimming large multilingual vocabularies in Small Language Models (SLM) is a simple, low-risk way to boost efficiency to its limit. It accelerates the model inference significantly while keeping accuracy almost unchanged. Mon, 04 Aug 2025 09:00:00 GMT https://blog.squeezebits.com/vocabulary-trimming-methods Research Semin Kim https://blog.squeezebits.com/gralora-boosting-fine-tuning-accuracy LoRA excels at efficient fine-tuning but suffers at higher ranks due to gradient entanglement. We introduce GraLoRA, which addresses these issues through finer-grained, block-wise updates, significantly enhancing performance and expressivity without overhead. GraLoRA outperforms LoRA across tasks, achieving up to +8.5% improvement in HumanEval+ Pass@1. Mon, 21 Jul 2025 00:00:00 GMT https://blog.squeezebits.com/gralora-boosting-fine-tuning-accuracy Research Yeonjoon Jung https://blog.squeezebits.com/owlite-qualcomm-on-device-ai At SqueezeBits we have been empowering developers to efficiently deploy complex AI models while minimizing performance trade-offs with OwLite toolkit. With OwLite v2.5, we're excited to announce official support for Qualcomm Neural Network (QNN) through seamless integration with Qualcomm AI Hub. Thu, 03 Jul 2025 01:00:00 GMT https://blog.squeezebits.com/owlite-qualcomm-on-device-ai Product Eunik Park https://blog.squeezebits.com/bringing-npus-into-production SqueezeBits has partnered with Intel to make Gaudi NPUs more usable in practice. We optimized LLMs and diffusion models for Gaudi-2 and created yetter, a generative AI API service. Tue, 01 Jul 2025 07:52:00 GMT https://blog.squeezebits.com/bringing-npus-into-production Tech Insight https://blog.squeezebits.com/tokyo-japan-itwwek-2025-global-ai-expo-experience-en SqueezeBits at Japan IT Week Spring 2025 in Tokyo: AI model compression demos, OwLite and Fits on Chips introductions, Japan market entry experiences, and team stories from the frontline. Tue, 10 Jun 2025 07:50:00 GMT https://blog.squeezebits.com/tokyo-japan-itwwek-2025-global-ai-expo-experience-en Event Naeun Kim https://blog.squeezebits.com/how-to-quantize-transformerbased-model-for-tensorrt-deployment-55802 This article describes the experimental results of quantized Vision Transformer model and its variants with OwLite. Tue, 20 May 2025 05:02:00 GMT https://blog.squeezebits.com/how-to-quantize-transformerbased-model-for-tensorrt-deployment-55802 Product Daehyun Ahn https://blog.squeezebits.com/how-to-quantize-yolo-models-with-owlite-54076 This article describes the experimental results of quantized YOLO models with OwLite. Wed, 07 May 2025 00:47:00 GMT https://blog.squeezebits.com/how-to-quantize-yolo-models-with-owlite-54076 Product Daehyun Ahn https://blog.squeezebits.com/owlite-no-more-compromising-on-ai-performance-after-quantization-51779 Discover how OwLite simplifies AI model optimization with seamless integration and secure architecture. Fri, 11 Apr 2025 08:54:13 GMT https://blog.squeezebits.com/owlite-no-more-compromising-on-ai-performance-after-quantization-51779 Product Seungryeol Kim https://blog.squeezebits.com/intel-gaudi-5-flux1-on-gaudi2-50213 This article discusses inference efficiency when running the FLUX.1 models on Intel Gaudi-2 hardware. Wed, 02 Apr 2025 09:20:00 GMT https://blog.squeezebits.com/intel-gaudi-5-flux1-on-gaudi2-50213 Tech Insight Taesu Kim https://blog.squeezebits.com/global-ai-events-recap-squeezebits-en From Edge AI to NVIDIA GTC: Squeezebits team members share firsthand stories from global AI events, including networking insights, technical trends, and conference experiences. Wed, 02 Apr 2025 08:03:00 GMT https://blog.squeezebits.com/global-ai-events-recap-squeezebits-en Event Naeun Kim https://blog.squeezebits.com/tensorrtllm-goes-open-source-48780 With TensorRT-LLM now open source, we can finally take a deep dive into the secret sauce behind its impressive performance. Tue, 25 Mar 2025 16:00:00 GMT https://blog.squeezebits.com/tensorrtllm-goes-open-source-48780 Tech Insight Huijong Jeong https://blog.squeezebits.com/when-should-i-use-fits-on-chips-46717 This article describes when to use Fits on Chips toolkit with specific use cases. Mon, 10 Mar 2025 00:00:00 GMT https://blog.squeezebits.com/when-should-i-use-fits-on-chips-46717 Product Daehyun Ahn https://blog.squeezebits.com/fits-on-chips-saving-llm-costs-became-easier-than-ever-38187 This article introduces Fits on Chips, an LLMOps toolkit for performance evaluation. Wed, 26 Feb 2025 15:00:00 GMT https://blog.squeezebits.com/fits-on-chips-saving-llm-costs-became-easier-than-ever-38187 Product Seungryeol Kim https://blog.squeezebits.com/sleb-streamlining-llms-through-redundancy-verification-and-elimination-of-transformer-blocks-f2bb262342d6 A brief review of the research paper from our team, published at ICML 2024. Mon, 17 Feb 2025 00:00:00 GMT https://blog.squeezebits.com/sleb-streamlining-llms-through-redundancy-verification-and-elimination-of-transformer-blocks-f2bb262342d6 Research https://blog.squeezebits.com/the-missing-piece-of-tensorrtllm-42462 This article is about an open-source library for direct conversion of PyTorch models to TensorRT-LLM. Mon, 10 Feb 2025 00:00:00 GMT https://blog.squeezebits.com/the-missing-piece-of-tensorrtllm-42462 Tech Insight Jiwoong Choi https://blog.squeezebits.com/the-rise-and-fall-of-onnx-feat-pytorch-20-42184 This article explores the rise and fall of ONNX, from its early success as a unifying stasndard for AI frameworks to its gradual shift into a niche tool in the era of PyTorch 2.0. Thu, 06 Feb 2025 00:00:00 GMT https://blog.squeezebits.com/the-rise-and-fall-of-onnx-feat-pytorch-20-42184 Tech Insight Taesu Kim https://blog.squeezebits.com/vllm-vs-tensorrtllm-13-visionlanguage-models-40761 This article provides a comparative analysis of serving vision-language models on vLLM and TensorRT-LLM. Mon, 20 Jan 2025 00:00:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-13-visionlanguage-models-40761 Tech Insight Yeonjoon Jung https://blog.squeezebits.com/intel-gaudi-4-fp8-quantization--40269 In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability. Mon, 13 Jan 2025 11:14:00 GMT https://blog.squeezebits.com/intel-gaudi-4-fp8-quantization--40269 Tech Insight Minkyu Kim https://blog.squeezebits.com/intel-gaudi-3-performance-evaluation-with-synapseai-v119-39839 In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability. Mon, 06 Jan 2025 00:00:00 GMT https://blog.squeezebits.com/intel-gaudi-3-performance-evaluation-with-synapseai-v119-39839 Tech Insight Taesu Kim https://blog.squeezebits.com/vllm-vs-tensorrtllm-12-automatic-prefix-caching-38189 This article provides a comparative analysis of automatic prefix caching. Mon, 23 Dec 2024 02:15:41 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-12-automatic-prefix-caching-38189 Tech Insight Daehyun AhnYeonjoon JungTaesu KimHuijong Jeong https://blog.squeezebits.com/vllm-vs-tensorrtllm-11-speculative-decoding-37301 This article provides a comparative analysis of speculative decoding. Mon, 09 Dec 2024 03:26:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-11-speculative-decoding-37301 Tech Insight Daehyun AhnYeonjoon Jung https://blog.squeezebits.com/37065 This article provides a comparative analysis of multi-LoRA serving capabilities of vLLM and TensorRT-LLM frameworks. Thu, 05 Dec 2024 02:30:58 GMT https://blog.squeezebits.com/37065 Tech Insight Jongho Lee https://blog.squeezebits.com/36821 In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability. Mon, 02 Dec 2024 16:08:00 GMT https://blog.squeezebits.com/36821 Tech Insight Taesu Kim https://blog.squeezebits.com/vllm-vs-tensorrtllm-9-parallelism-strategies-36310 This article provides a comparative analysis of different parallelism strategies on vLLM and TensorRT-LLM frameworks. Tue, 26 Nov 2024 04:35:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-9-parallelism-strategies-36310 Tech Insight Changjun Lee https://blog.squeezebits.com/intel-gaudi-1-introduction-35414 In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability. Thu, 21 Nov 2024 04:28:00 GMT https://blog.squeezebits.com/intel-gaudi-1-introduction-35414 Tech Insight Taesu Kim https://blog.squeezebits.com/vllm-vs-tensorrtllm-8-kv-cache-quantization-35079 This article provides a comparative analysis of the effects of KV cache quantization on vLLM and TensorRT-LLM frameworks. Mon, 18 Nov 2024 03:01:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-8-kv-cache-quantization-35079 Tech Insight Jiwon Song https://blog.squeezebits.com/vllm-vs-tensorrtllm-7-weightactivation-quantization-34461 This article provides a comparative analysis of the effects of weight-activation quantization on vLLM and TensorRT-LLM frameworks. Mon, 11 Nov 2024 02:38:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-7-weightactivation-quantization-34461 Tech Insight Eunik Park https://blog.squeezebits.com/vllm-vs-tensorrtllm-6-weightonly-quantization-33728 This article provides a comparative analysis of the effects of weight-only quantization on vLLM and TensorRT-LLM frameworks. Fri, 01 Nov 2024 04:51:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-6-weightonly-quantization-33728 Tech Insight Jiwon Song https://blog.squeezebits.com/vllm-vs-tensorrtllm-5-dynamic-sequence-lengths--33410 This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on performance with fixed and dynamic datasets. Wed, 30 Oct 2024 02:19:10 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-5-dynamic-sequence-lengths--33410 Tech Insight Minkyu Kim https://blog.squeezebits.com/vllm-vs-tensorrtllm-4-which-scheduler-wins--33083 This article provides a comparative analysis of schedulers in vLLM and TensorRT-LLM frameworks. Thu, 24 Oct 2024 03:22:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-4-which-scheduler-wins--33083 Tech Insight Huijong Jeong https://blog.squeezebits.com/vllm-vs-tensorrtllm-3-understanding-sampling-methods-and-their-performance-impact-31921 This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks with various sampling methods. Fri, 18 Oct 2024 03:54:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-3-understanding-sampling-methods-and-their-performance-impact-31921 Tech Insight Daehyun Ahn https://blog.squeezebits.com/vllm-vs-tensorrtllm-2-towards-optimal-batching-for-llm-serving-31349 This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on batching configurations and thoroughly examining the effects of maximum batch size and maximum number of tokens. Fri, 11 Oct 2024 04:56:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-2-towards-optimal-batching-for-llm-serving-31349 Tech Insight Yeonjoon Jung https://blog.squeezebits.com/vllm-vs-tensorrtllm-1-an-overall-evaluation-30703 This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks for serving LLMs, evaluating their performance based on key metrics like throughput, TTFT, and TPOT to offer insights for practitioners in optimizing LLM deployment strategies. Tue, 01 Oct 2024 05:39:00 GMT https://blog.squeezebits.com/vllm-vs-tensorrtllm-1-an-overall-evaluation-30703 Tech Insight Yeonjoon Jung https://blog.squeezebits.com/how-much-can-we-save-through-compression-b675c60611b4 Estimating the cost savings from model compression. Wed, 26 Jun 2024 07:30:29 GMT https://blog.squeezebits.com/how-much-can-we-save-through-compression-b675c60611b4 Tech Insight https://blog.squeezebits.com/ai-lightweight-experience-it-exhibition-en SqueezeBits' IT exhibition recap: from AI model compression demos to hands-on OwLite experiences, booth visitor reactions, and more. Read our on-the-ground event story! Mon, 27 May 2024 08:44:00 GMT https://blog.squeezebits.com/ai-lightweight-experience-it-exhibition-en Event Naeun Kim https://blog.squeezebits.com/breaking-down-tokenizers-in-llms-5699a8122574 An introduction to tokenizers and their implications in language models. Thu, 16 May 2024 08:23:54 GMT https://blog.squeezebits.com/breaking-down-tokenizers-in-llms-5699a8122574 Tech Insight https://blog.squeezebits.com/accuracy-degradation-in-ai-compression-myth-or-truth-c7a94ec0bc92 Clarifying the misunderstandings in AI model compression Wed, 24 Apr 2024 05:19:41 GMT https://blog.squeezebits.com/accuracy-degradation-in-ai-compression-myth-or-truth-c7a94ec0bc92 Tech Insight https://blog.squeezebits.com/are-you-getting-everything-out-of-your-gpus-1f030a4a460f The Blackwell GPU from GTC 2024 was astonishing.Analysis of the Nvidia GPU evolution & what it means for GPU users. Tue, 23 Apr 2024 02:20:50 GMT https://blog.squeezebits.com/are-you-getting-everything-out-of-your-gpus-1f030a4a460f Tech Insight https://blog.squeezebits.com/things-to-check-if-your-business-utilizes-ai-53be650a1248 Do I need to COMPRESS my AI model? : the short answer is “YES” — and here’s why. Fri, 19 Apr 2024 04:41:25 GMT https://blog.squeezebits.com/things-to-check-if-your-business-utilizes-ai-53be650a1248 Tech Insight https://blog.squeezebits.com/4-types-of-ai-compression-methods-you-should-know-5d07759c60a7 AI model compression for acceleration is essential. The question is HOW? Here are 4 key methodologies. Mon, 15 Apr 2024 08:55:19 GMT https://blog.squeezebits.com/4-types-of-ai-compression-methods-you-should-know-5d07759c60a7 Tech Insight