# The official SqueezeBits Tech blog > The official SqueezeBits Tech blog covering AI model compression and optimization, LLM and Edge AI systems, and deep technical insights into products like Yetter, Fits on Chips, and OwLite, along with research, engineering, and development updates. ## Categories - [Event](https://blog.squeezebits.com/category/event) - [Product](https://blog.squeezebits.com/category/product) - [Tech Insight](https://blog.squeezebits.com/category/tech) ## Posts in Event - [[Conference Recap] Key Takeaways from AI Conference in Taipei & Singapore ](https://blog.squeezebits.com/ai-conference-recap-key-takeaways): Read SqueezeBits’ firsthand recap of our June 2026 conference visits in Taiwan and Singapore, where we confirmed growing demand for AI inference optimization and for generative AI infrastructure across Asian markets. - [2026 Modular Seoul Developer Meetup](https://blog.squeezebits.com/modular-seoul-developer-meetup-recap): Modular, a U.S. based AI unicorn, hosted its first Korea meetup with SqueezeBits. Read our recap of the first Seoul developer meetup! - [2026 Efficient AI Offline Meetup](https://blog.squeezebits.com/efficient-ai-meetup): Wrap up 8 weeks of online studies and take a look at how SqueezeBits makes an effort to maintain the AI compression community to expand! - [Recap: 2nd vLLM Korea Meetup 2026](https://blog.squeezebits.com/vllm-korea-meetup-highlights): Check out highlights from the 2nd vLLM Korea Meetup! open-source use cases and real-world production examples that showcase vLLM's technical maturity! - [Our Experience Running a Booth at GTC 2026](https://blog.squeezebits.com/gtc-conference-booth-review-en): Sharing GTC 2026 insights, which is the Largest AI Industry Conference for developers! If you’ve ever wondered what it’s like for an AI startup to run a booth at such a massive event, you won’t want to miss this! - [Intel® Gaudi® Hands-on Workshop | A Recap of the Gaudi Workshop with SqueezeBits x Lablup](https://blog.squeezebits.com/intel-gaudi-hands-on-workshop-en): A recap of the Intel® Gaudi® hands-on workshop co-hosted by SqueezeBits and Lablup. AI model compression, fine-tuning, and vLLM serving on Gaudi® hardware with Backend.AI. - [vLLM Hands-on Workshop with Rebellions & SqueezeBits: A Recap](https://blog.squeezebits.com/vllm-hands-on-workshop-with-rebellions-squeezebits-en): Rebellions and SqueezeBits Co-Host a vLLM Hands-on Workshop: Workshop Highlights, PyTorch Best Practices, Performance Optimization, and Developer First-Hand Tips! - [[Efficient AI Study] AI Model Compression Community Study and Meetup](https://blog.squeezebits.com/efficient-ai-study-meetup-by-squeezebits-en): Efficient AI Study & Meetup recap: SqueezeBits' community study on AI model compression, featuring paper reviews, participant interviews, and networking from the offline meetup. - [[Japan IT Week Spring 2025] What We Saw on the Global AI Frontline in Tokyo](https://blog.squeezebits.com/tokyo-japan-itwwek-2025-global-ai-expo-experience-en): SqueezeBits at Japan IT Week Spring 2025 in Tokyo: AI model compression demos, OwLite and Fits on Chips introductions, Japan market entry experiences, and team stories from the frontline. - [Field Notes from the Global AI Market: Our Overseas Event Recap](https://blog.squeezebits.com/global-ai-events-recap-squeezebits-en): From Edge AI to NVIDIA GTC: Squeezebits team members share firsthand stories from global AI events, including networking insights, technical trends, and conference experiences. - [Experiencing AI Model Compression Firsthand: Our IT Exhibition Story](https://blog.squeezebits.com/ai-lightweight-experience-it-exhibition-en): SqueezeBits' IT exhibition recap: from AI model compression demos to hands-on OwLite experiences, booth visitor reactions, and more. Read our on-the-ground event story! ## Posts in Tech Insight - [Reliable & Scalable Synthetic Data for Physical AI (Part 2): Making Cosmos 3.1 x Faster for Production](https://blog.squeezebits.com/reliable-synthetic-data-physical-ai-production): Explore why Physical AI deployment needs synthetic data at scale with Squeezebits' research and discover how to overcome inference bottlenecks to accelerate Roboost Agent. - [Reliable & Scalable Synthetic Data for Physical AI (Part 1): Taming NVIDIA Cosmos with RoBoost Agent](https://blog.squeezebits.com/reliable-synthetic-data-physical-ai): Scaling Physical AI requires reliable synthetic data. Learn how RoBoost Agent integrates NVIDIA Cosmos to transform world models into trustworthy data engines for robotics and autonomous driving. - [Introducing rebellions ATOM™-MAX](https://blog.squeezebits.com/introducing-atom-max-npu): Introducing ATOM™-Max, rebellions’ next-generation NPU designed for high-performance AI inference. Learn how its runtime, profiling tools, and PyTorch-native integrations enable developers to run and serve models efficiently without sacrificing usability. - [[Intel Gaudi] #6. GEMM, Attention, vLLM on Gaudi](https://blog.squeezebits.com/intel-gaudi-gemm-attention-performance): Explore how Intel’s new Gaudi-3 compares to Gaudi-2, NVIDIA A100, and H100. We analyze real-world GEMM efficiency, attention performance, and LLM serving results to uncover what truly matters for AI inference and training workloads. - [Guided Decoding Performance on vLLM and SGLang](https://blog.squeezebits.com/guided-decoding-performance-vllm-sglang): The guide to LLM guided decoding! This deep-dive benchmark compares XGrammar and LLGuidance on vLLM and SGLang to help you find the optimal setup for generating structured output based on your use case. - [Disaggregated Inference on Apple Silicon: NPU prefill and GPU decode](https://blog.squeezebits.com/disaggregated-inference-on-apple-silicon-npu-prefill-and-gpu-decode-67176): In this article, we introduce how to run LLMs efficiently on Apple Silicon with disaggregated inference technique. - [Vocabulary Trimming: An Easy and Effective Method for SLM Acceleration](https://blog.squeezebits.com/vocabulary-trimming-methods): Trimming large multilingual vocabularies in Small Language Models (SLM) is a simple, low-risk way to boost efficiency to its limit. It accelerates the model inference significantly while keeping accuracy almost unchanged. - [GraLoRA: Boosting Fine-Tuning Accuracy Without Extra Cost](https://blog.squeezebits.com/gralora-boosting-fine-tuning-accuracy): LoRA excels at efficient fine-tuning but suffers at higher ranks due to gradient entanglement. We introduce GraLoRA, which addresses these issues through finer-grained, block-wise updates, significantly enhancing performance and expressivity without overhead. GraLoRA outperforms LoRA across tasks, achieving up to +8.5% improvement in HumanEval+ Pass@1. - [Bringing NPUs into Production: Our Journey with Intel Gaudi](https://blog.squeezebits.com/bringing-npus-into-production): SqueezeBits has partnered with Intel to make Gaudi NPUs more usable in practice. We optimized LLMs and diffusion models for Gaudi-2 and created yetter, a generative AI API service. - [[Intel Gaudi] #5. FLUX.1 on Gaudi-2](https://blog.squeezebits.com/intel-gaudi-5-flux1-on-gaudi2-50213): This article discusses inference efficiency when running the FLUX.1 models on Intel Gaudi-2 hardware. - [TensorRT-LLM Goes Open Source!](https://blog.squeezebits.com/tensorrtllm-goes-open-source-48780): With TensorRT-LLM now open source, we can finally take a deep dive into the secret sauce behind its impressive performance. - [SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks ](https://blog.squeezebits.com/sleb-streamlining-llms-through-redundancy-verification-and-elimination-of-transformer-blocks-f2bb262342d6): A brief review of the research paper from our team, published at ICML 2024. - [The Missing Piece of TensorRT-LLM](https://blog.squeezebits.com/the-missing-piece-of-tensorrtllm-42462): This article is about an open-source library for direct conversion of PyTorch models to TensorRT-LLM. - [The Rise and Fall of ONNX (feat. PyTorch 2.0)](https://blog.squeezebits.com/the-rise-and-fall-of-onnx-feat-pytorch-20-42184): This article explores the rise and fall of ONNX, from its early success as a unifying stasndard for AI frameworks to its gradual shift into a niche tool in the era of PyTorch 2.0. - [[vLLM vs TensorRT-LLM] #13. Vision-Language Models](https://blog.squeezebits.com/vllm-vs-tensorrtllm-13-visionlanguage-models-40761): This article provides a comparative analysis of serving vision-language models on vLLM and TensorRT-LLM. - [[Intel Gaudi] #4. FP8 Quantization ](https://blog.squeezebits.com/intel-gaudi-4-fp8-quantization--40269): In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability. - [[Intel Gaudi] #3. Performance Evaluation with SynapseAI v1.19](https://blog.squeezebits.com/intel-gaudi-3-performance-evaluation-with-synapseai-v119-39839): In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability. - [[vLLM vs TensorRT-LLM] #12. Automatic Prefix Caching](https://blog.squeezebits.com/vllm-vs-tensorrtllm-12-automatic-prefix-caching-38189): This article provides a comparative analysis of automatic prefix caching. - [[vLLM vs TensorRT-LLM] #11. Speculative Decoding](https://blog.squeezebits.com/vllm-vs-tensorrtllm-11-speculative-decoding-37301): This article provides a comparative analysis of speculative decoding. - [[vLLM vs TensorRT-LLM] #10 Serving Multiple LoRAs at Once](https://blog.squeezebits.com/37065): This article provides a comparative analysis of multi-LoRA serving capabilities of vLLM and TensorRT-LLM frameworks. - [[Intel Gaudi] #2. Graph Compiler and Overall Performance Evaluation](https://blog.squeezebits.com/36821): In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability. - [[vLLM vs TensorRT-LLM] #9. Parallelism Strategies](https://blog.squeezebits.com/vllm-vs-tensorrtllm-9-parallelism-strategies-36310): This article provides a comparative analysis of different parallelism strategies on vLLM and TensorRT-LLM frameworks. - [[Intel Gaudi] #1. Introduction](https://blog.squeezebits.com/intel-gaudi-1-introduction-35414): In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability. - [[vLLM vs TensorRT-LLM] #8. KV Cache Quantization](https://blog.squeezebits.com/vllm-vs-tensorrtllm-8-kv-cache-quantization-35079): This article provides a comparative analysis of the effects of KV cache quantization on vLLM and TensorRT-LLM frameworks. - [[vLLM vs TensorRT-LLM] #7. Weight-Activation Quantization](https://blog.squeezebits.com/vllm-vs-tensorrtllm-7-weightactivation-quantization-34461): This article provides a comparative analysis of the effects of weight-activation quantization on vLLM and TensorRT-LLM frameworks. - [[vLLM vs TensorRT-LLM] #6. Weight-Only Quantization](https://blog.squeezebits.com/vllm-vs-tensorrtllm-6-weightonly-quantization-33728): This article provides a comparative analysis of the effects of weight-only quantization on vLLM and TensorRT-LLM frameworks. - [[vLLM vs TensorRT-LLM] #5. Dynamic Sequence Lengths ](https://blog.squeezebits.com/vllm-vs-tensorrtllm-5-dynamic-sequence-lengths--33410): This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on performance with fixed and dynamic datasets. - [[vLLM vs TensorRT-LLM] #4. Which Scheduler Wins? 🔥](https://blog.squeezebits.com/vllm-vs-tensorrtllm-4-which-scheduler-wins--33083): This article provides a comparative analysis of schedulers in vLLM and TensorRT-LLM frameworks. - [[vLLM vs TensorRT-LLM] #3. Understanding Sampling Methods and Their Performance Impact](https://blog.squeezebits.com/vllm-vs-tensorrtllm-3-understanding-sampling-methods-and-their-performance-impact-31921): This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks with various sampling methods. - [[vLLM vs TensorRT-LLM] #2. Towards Optimal Batching for LLM Serving](https://blog.squeezebits.com/vllm-vs-tensorrtllm-2-towards-optimal-batching-for-llm-serving-31349): This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on batching configurations and thoroughly examining the effects of maximum batch size and maximum number of tokens. - [[vLLM vs TensorRT-LLM] #1. An Overall Evaluation](https://blog.squeezebits.com/vllm-vs-tensorrtllm-1-an-overall-evaluation-30703): This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks for serving LLMs, evaluating their performance based on key metrics like throughput, TTFT, and TPOT to offer insights for practitioners in optimizing LLM deployment strategies. - [How much can we save through compression? ](https://blog.squeezebits.com/how-much-can-we-save-through-compression-b675c60611b4): Estimating the cost savings from model compression. - [‘Breaking Down’ Tokenizers in LLMs ](https://blog.squeezebits.com/breaking-down-tokenizers-in-llms-5699a8122574): An introduction to tokenizers and their implications in language models. - [Accuracy Degradation in AI Compression: Myth or Truth? ](https://blog.squeezebits.com/accuracy-degradation-in-ai-compression-myth-or-truth-c7a94ec0bc92): Clarifying the misunderstandings in AI model compression - [Are you getting everything out of your GPUs?](https://blog.squeezebits.com/are-you-getting-everything-out-of-your-gpus-1f030a4a460f): The Blackwell GPU from GTC 2024 was astonishing. Analysis of the Nvidia GPU evolution & what it means for GPU users. - [Things to check if your business utilizes AI ](https://blog.squeezebits.com/things-to-check-if-your-business-utilizes-ai-53be650a1248): Do I need to COMPRESS my AI model? : the short answer is “YES” — and here’s why. - [AI Compression for Acceleration: 4 Key Methods. ](https://blog.squeezebits.com/4-types-of-ai-compression-methods-you-should-know-5d07759c60a7): AI model compression for acceleration is essential. The question is HOW? Here are 4 key methodologies. ## Posts in Product - [Winning both speed and quality: How Yetter deals with diffusion models](https://blog.squeezebits.com/77516): Explore how the Yetter Inference Engine overcomes the limitations of step caching and model distillation for diffusion models. We analyze latency, diversity, quality, and negative-prompt handling to reveal what truly matters for scalable, real-time image generation. - [Yetter, the GenAI API service: AI Optimization, Out of the Box](https://blog.squeezebits.com/yetter-genai-api-service): Meet 'Yetter': the generative AI API service built for speed, efficiency, and scalability. Powered by our optimization inference engine, it delivers reliable image, video, and future LLM services at a fraction of the cost. - [OwLite Meets Qualcomm Neural Network: Unlocking On-Device AI Performance](https://blog.squeezebits.com/owlite-qualcomm-on-device-ai): At SqueezeBits we have been empowering developers to efficiently deploy complex AI models while minimizing performance trade-offs with OwLite toolkit. With OwLite v2.5, we're excited to announce official support for Qualcomm Neural Network (QNN) through seamless integration with Qualcomm AI Hub. - [How to Quantize Transformer-based model for TensorRT Deployment](https://blog.squeezebits.com/how-to-quantize-transformerbased-model-for-tensorrt-deployment-55802): This article describes the experimental results of quantized Vision Transformer model and its variants with OwLite. - [How to Quantize YOLO models with OwLite](https://blog.squeezebits.com/how-to-quantize-yolo-models-with-owlite-54076): This article describes the experimental results of quantized YOLO models with OwLite. - [OwLite: No More Compromising on AI Performance After Quantization](https://blog.squeezebits.com/owlite-no-more-compromising-on-ai-performance-after-quantization-51779): Discover how OwLite simplifies AI model optimization with seamless integration and secure architecture. - [When Should I Use Fits on Chips?](https://blog.squeezebits.com/when-should-i-use-fits-on-chips-46717): This article describes when to use Fits on Chips toolkit with specific use cases. - [Fits on Chips: Saving LLM Costs Became Easier Than Ever](https://blog.squeezebits.com/fits-on-chips-saving-llm-costs-became-easier-than-ever-38187): This article introduces Fits on Chips, an LLMOps toolkit for performance evaluation.