Ternary Large Language Models store every weight as one of three values: -1, 0, or +1. The information-theoretic minimum for this is log2(3), approximately 1.58 bits per weight. But a new paper from Intel researchers shows that real-world ternary models are far from uniform — and that asymmetry can be exploited to store weights even more compactly.
The researchers measured the actual symbol distribution across 29 ternary LLM models and found that zeros account for up to 51.5 percent of all weights. This is the key insight: treating all three symbols as equally likely wastes storage because the distribution is heavily skewed toward zero.
They introduce BITCOS, a distribution-adaptive layout consisting of a dense presence bitmap plus a compacted sign vector. The cost is 2 minus z bits per weight element, where z is the zero density. On the sparsest models tested, BITCOS reaches 1.485 bits per weight — below the 1.58-bit theoretical floor that assumes equiprobable symbols.
Beyond storage compression, BITCOS enables faster computation. The researchers present optimized unpacking sequences for AVX-512, AVX2, and Intel Xe2 GPUs. Against production state-of-the-art ternary matrix-vector multiplication kernels, BITCOS delivers up to 1.28x speedup. End-to-end LLM inference tests across five platforms show decode throughput improvements of up to 1.18x on CPUs and 1.27x on GPUs.
For the AI industry's push toward efficient inference, this matters: ternary models are already being deployed at scale, and any reduction in memory bandwidth or compute cost directly improves the economics of running large language models.