The Era of 1-bit LLMs: Three Symbols, 1.58 Bits, Zero Multiplies

1Why "1-bit"?

Ask an LLM to generate a token and nearly every number it touches is a weight: 7 billion of them for a 7B model, 70 billion for a 70B. These weights are read from memory once per forward pass, and on modern hardware reading is the expensive part. A 4.6 pJ multiply-add costs less than a fifth of the energy of moving a single bit out of DRAM (~20 pJ) [4]. Scale that across billions of weights and the arithmetic all but disappears next to the memory traffic.

So the most direct efficiency question for inference isn’t “how fast can we multiply,” it’s “how few bits can we read per weight?” The standard answer since the 2010s has been progressive quantization: FP32 → FP16 → INT8 → 4-bit. At each step quality barely moved and throughput grew. But 4-bit is where the naive path stalls: a full integer multiplier still costs real die area and energy, and going below 4 bits with uniform grids historically broke accuracy.

Enter the observation that changed the framing: weights don’t need to approximate arbitrary values. They can be restricted at train time.

2BitNet b1.58

BitNet b1.58 [1, 2] trains a transformer whose linear layers use only weights from the ternary set

$$\{-1, 0, +1\},$$

scaled per-layer by a detached mean-absolute magnitude $\gamma = \mathrm{mean}\,|W|$. Values are snapped to the nearest ternary symbol in the forward pass; the backward pass passes gradients through unchanged (a straight-through estimator), optionally clipped. Crucially, the quantizer is applied during training, not after: the model learns to be robust to its own discrete weights.

The headline result (hence the “1.58” in the name) is that a model trained this way, at the 3B–7B scale, closes most of the perplexity gap to full precision by simply scaling up: low-bit weights under-parameterize each layer, so you buy back capacity with more layers and parameters. Ternary weights are cheap enough that scaling up is affordable.

3What 1.58 bits actually means

$\log_2 3 = 1.585$. That is the information content of a symbol chosen from three equally likely states. The name “1.58-bit” is an idealization, not a packing: real hardware doesn’t do 1.585-bit containers. In practice you store each ternary weight in a compact code and pay packing overhead, or you accept a two-bit container and burn the unused state.

Two (sic) numbers are worth internalizing. A 7B model at FP16 needs ~14 GB of weights; at 1.58 bits it needs ~1.4 GB. And in the energy model above, that single change dominates every other saving available in the system.

4Kernels are the product

Ternary weights change the kind of arithmetic, not just the precision. A dot product of ternary weights with integer activations is

$$\sum_j a_j \cdot w_j, \qquad w_j \in \{-1, 0, 1\},$$

which contains no multiplications at all: each term is a select (which slope to apply), an optional negate, and an integer add. That is the core of Bitnet.cpp [3]: a kernel stack that packs ternary weights densely, replays activation tiles from cache, and issues pure add/compare instructions. The same trick applies to quaternary grids with power-of-two intermediates: multiply by $1/2$ becomes a one-bit shift, which is exactly the direction TetraNet [5] explores.

The general lesson: for low-bit LLMs, the kernel is the product. The weight alphabet decides the arithmetic; the kernel decides whether the alphabet’s promise survives contact with hardware. A theoretically-clean grid with a slow implementation loses to a clever ternary kernel every time.

5The two-bit frontier

Here is the loose thread. Exactly two bits store four states; cast into a ternary grid, the fourth state sits idle: you pay for 2.00 bits of DRAM and enjoy 1.58 bits of representation. Ternary in an ideal entropy code saves ~21% of that traffic (2.00 vs. 1.58 bits per weight), but the packing machinery is nontrivial.

What if the fourth state worked for you instead? A symmetric quaternary grid

$$\mathcal{Q}_c = \{-1, -c, c, 1\}, \qquad c \in \{1/4, 1/2\}$$

uses all four states, keeps two-bit packing, and, because the intermediates are powers of two, preserves the multiply-free dot product: +c and −c cost one or two arithmetic shifts. Our TetraNet pilot [5] found both variants beat a matched BitNet-style ternary baseline on TinyStories perplexity at equal 9.4M-scale budgets, at essentially the same analytical energy (≈1.02 mJ/token vs. ≈1.02 for packed ternary). The fourth state isn’t free physics; it’s free representation.

6Where the research lands

Stepping back, three claims are now well-supported, and three are not.

Well-supported. (1) Training-time quantization with ternary weights scales far better than post-hoc compression. (2) Multiply-free kernels are a real, deployable systems trick (Bitnet.cpp executes them on commodity CPUs). (3) Memory traffic, not FLOPs, is the binding constraint for edge inference.

Not yet established. (1) Whether quaternary grids beat ternary at scale: the TetraNet evidence is pilot-grade (one seed, ~9M parameters). (2) Whether the ~0.2% energy parity at equal packing survives real cache hierarchies. (3) Whether low-bit training preserves capabilities beyond perplexity (reasoning, instruction following, long-context) at 7B+ scale.

The honest summary of 2026’s low-bit landscape: 1.58 bits is real, shipping, and roughly a floor, and the two-bit register has one seat left. We think it should be filled.