Weights × budget stack politely.
AWQ-INT4 plus a tight budget costs about the sum of the two on their own. No significant accuracy interaction on any benchmark: the bootstrap 95% CIs all span zero.
CostPerThought
Everyone stacks the "nearly free" speedups for reasoning models on faith. I stacked all of them and checked the bill.
Weight quantization
FP16 → AWQ-INT4. Smaller weights, less memory. Supposedly "free".
KV-cache quantization
FP16 → FP8 for the attention cache. That matters a lot when the model thinks for thousands of tokens.
Reasoning budget cap
"Budget forcing": cut the thinking phase off at 75 / 50 / 25% of its natural length.
Each one usually gets validated alone. In production they all get switched on together. Nobody had published what happens when you stack them, or what that costs in real dollars on real rented GPUs rather than a FLOPs estimate. So I ran the full factorial: 2 weight precisions × 2 KV precisions × 4 budgets, across math, science QA, and code.
Stronger blue = better (more accurate, cheaper, or faster). Striped = FP8 garbage. Hatched = can't even run.
FP16AWQ-INT4FP8 KVBigger dot = bigger budget
What I found
Two "free" knobs weren't.
FP8 KV-cache broke everything.
Uncalibrated fp8_e4m3 produced degenerate output, with accuracy at or near 0%. Four separate fixes all failed on vLLM 0.11.0. It needs offline scale calibration, which is a real hidden setup cost for a knob sold as nearly free.
"All three on" can't even run.
AWQ needs fp16 activations. FP8-KV attention needs bf16. In vLLM 0.11.0 the combination everyone assumes they're running is literally un-runnable.
0× cheaper
$ per correct answer on MATH from capping the reasoning budget alone. That was the real cost lever.
~0× slower
the int4 dequant path vs fp16 on a memory-rich GPU. On roomy hardware, AWQ made $ per correct worse.
Show & tell
Presenting it at the LaunchPad AI Symposium.
launchpad-group.jpg
launchpad-poster-1.jpg
launchpad-poster-2.jpg
KAIROS →