MEGACRUNCH / PUBLIC RESEARCH SURFACE

MEGA CRUNCH

Compress the model.
Not the behavior.

STATED PLAINLY

The distinction is the whole programme and it is not yet proven at a scale that would matter. We have shown it on one transformer block of a 1.7B model. Whether behavior survives compression across a whole network is an open question, and we have not answered it.

THE FIRST QUANT IS NOT THE ANSWER ↓

THESIS / 01

Quantization chooses
a compressed state.
MegaCrunch trains it.

STATED PLAINLY

Ordinary quantization is a rounding decision made once, per weight, with no reference to what the model does afterwards. We treat that decision as an opening position and optimise against behavior instead. The cost is that this is a search rather than a formula: it takes compute, and the best state we find is not the best state that exists — only the best one we reached.

PRIMITIVE / MC-Q1_RT STATE / LEGAL Q1 BUDGET / FIXED SERIALIZED CEILING

Direct quantization picks one legal state and stops. The byte budget does not move — the same bytes can hold many states, and ordinary quantization takes the first. MegaCrunch searches for the state those bits can actually learn: not weights that resemble BF16, but the legal state that best preserves what BF16 does.

PROCESS / 02

No glowing brain.
No magic compression button.

01 SEED
Take the ordinary Q1 state. It is the starting point, not the result.
02 MEASURE
Compare against BF16 on final behavior, not on how closely the weights resemble it.
03 TRAIN
Search the states those same bytes can legally hold for the one that behaves best.
04 VERIFY
Keep the byte ceiling, lineage, and behavioral measurements attached to the result.

REWRITES / BINARY STATE REWRITES / GROUP SCALE STRUCTURE

STATED PLAINLY

MEASURE is the expensive step and the one we had wrong first. Every cheap proxy we tried — hidden-state error, distance from the original weights — ranked candidate states in close to the reverse of how they actually behaved. The measurement has to run the model and look at what comes out, which is most of why this is slow.

DESIGN RULE / 03

Quantization is an initialization.
Not an endpoint.

Internal resemblance is a proxy. Final behavior is the authority — ranking candidate states by hidden-state error came out close to the reverse of ranking them by what the model actually does.

STATED PLAINLY

We believed that re-observing tensors between rounds let them co-adapt, and that the joint arm was better for that reason. Three controlled experiments failed to establish it. Roughly four fifths of the advantage we had attributed to co-adaptation was simply more optimisation, and the remaining fifth is not statistically resolved. Re-observation does change what tensors learn, substantially — but we can no longer say it makes the result better, and our earlier line about tails and robustness is withdrawn.

CONTROL / COMPUTE-MATCHED ARMS METHOD / RE-OBSERVATION

INPUT / DIRECT Q1 STATE OBJECTIVE / BEHAVIOR AT FIXED BYTES OUTPUT / MEASURED RECOVERY

STATUS / 04

ACTIVE RESEARCH

Research first.
Marketing second.

PUBLIC POSITION

The work is evolving. Claims should survive exact byte ceilings, provenance checks, and repeated measurement before they become product language.

STATED PLAINLY

What would have to be true first: a whole model rather than one block, a public benchmark ladder rather than KL against BF16, and results that survive somebody else running them. None of those are done. Until they are, this is a research note with numbers attached.

BACK TO QPG GROUP ↙

Measured

Developmental · Qwen3-1.7B · one transformer block

Subject

What was recovered

Model	Qwen3-1.7B
Stored parameters	1,720,574,976
Recovered unit	One transformer block · 7 projections
Recovered parameters	50,300,000
Serialized state	7,082,750 bytes · 7.08 MB
Representation	Legal Q1 · fixed byte budget

Full-block recovery

Current result

Direct Q1 destroyed the block. The same byte budget, retrained, restored it.
Metric	Direct Q1	MC-Q1_RT	BF16
Final-logit KL vs BF16	15.807306	0.213703 – 0.236497	0
KL damage removed	—	98.5 – 98.6%	—
Final-logit relative MSE	3.581940	—	0
Block-output relative MSE	0.929811	0.275000 – 0.294106	0
BF16 top-1 agreement	0 / 101	79.21 – 81.19%	100%
Non-finite logits	0	0	0
Serialized bytes	baseline	identical	—

The two compute-matched arms behind that range.
Arm	Final KL	BF16 top-1	Block rel. MSE
Independent	0.236497	81.19%	0.294106
Joint · re-observed	0.213703	79.21%	0.275000

Not statistically distinguishable on final KL. The result is the recovery,
not the arrangement that produced it. Neither arm is preferred.

Output distribution

The collapse, and its repair

Direct Q1 was not noisy. It became confidently degenerate.
Quantity	Direct Q1	MC-Q1_RT	BF16
Unique argmax tokens / 101	8	67	68
Most-common token frequency	88.1%	9.9%	9.9%
Mean output entropy	0.611	2.125	1.915
Logit variance	17.992	8.180	8.676
Dominant token, positions held	89 / 101	10 / 101	10 / 101
Mean entropy, nats	0.6114	2.1250	1.9150
Most-common token identity	Differs from BF16	Same as BF16	—

State movement

What the optimizer actually does

The useful Q1 state is not near the direct projection, and how far it moves depends on the tensor.
Tensor class	Hamming from direct Q1
Attention projections	~26 – 28%
FFN down	36.5%
FFN gate	Very large group-scale drift
What moves	Binary state and group scale structure both

Ranking candidate states by hidden-state error came out close to the reverse
of ranking them by final KL. Internal numerical resemblance is a proxy.
Final model behavior is the authority.

Attribution

Corrected after controls

Most of the earlier joint advantage was deeper optimisation, not co-adaptation.
Component	KL	Share
Original joint advantage	0.024630	100%
Continued optimisation	0.019593	~79.5%
Re-observation — not statistically resolved	0.005037	~20.5%

Re-observation changes what tensors learn, without a resolved final-quality gain.
Round-two local improvement	Independent	Re-observed
Attention output	2.26%	41.10%
FFN gate	10.47%	19.02%
FFN up	2.36%	12.35%
FFN down	12.30%	73.87%

The attention-arm ladder the attribution above was derived from.
Arm	Final KL
Independent · round 1	0.072563
Independent · round 2	0.052970
Joint · round 2	0.047933

Implementation · CPU

Bit-identical acceleration

Metric	Before	After
Projection objective	41.1 s	6.4 s · 6.39×
Share of wall time	32.4%	7.3%
End-to-end run	108.6 – 127 s	87.8 s · 1.24 – 1.45×
Theoretical GPU ceiling	3.09×	13.64×
Recovered-state Hamming difference	—	0
Scale drift	—	0%
Fit objective	0.7096768573	0.7096768573

Implementation · GPU

Intel Arc B580 · Vulkan SPIR-V f64

The GPU reproduces the CPU's recovered symbol state exactly, not approximately.
Check	CPU	GPU
Symbols recovered	2,097,152	2,097,152
Symbols differing	—	0
Accepted flips	1,317,971	1,317,971
Hamming to direct Q1	537,019	537,019
Decisions accepted	7,821 / 32,768	7,821 / 32,768
Decision disagreements	—	0
Worst margin-relative difference	—	2.222 × 10⁻¹¹
Symbol-phase time	11.640 s	3.234 s · 3.60×
Transfer share of runtime	—	~0.6%
Resident VRAM	—	42.4 MB + 26.4 MB scratch

Where the GPU time goes, and what the device had to provide.
Upload	0.018 s
Compute	3.589 s
Readback	0.003 s
Shader compile	0.065 s
Run-to-run range	3.178 – 3.610 s
shaderFloat64 · shaderInt64	Both true
Determinism	Exact final symbol-state equality

Projected, not measured — the scale phase is still on CPU.
Symbol phase only, overall	~1.49×
Symbol and scale, overall	~2.4×
An 86-minute arm, projection only	~36 minutes

GLSL has no arbitrary f64 pow, so the GPU path runs only at tail_weight_power 1.0.
Every other exponent fails closed to CPU rather than silently approximating.

REGIME / BEHAVIORAL ATTRACTOR PHASE / SYMBOL PHASE / SCALE RECOVERY GUARD / FAIL CLOSED PARAMETER / tail_weight_power

Developmental · one-block result · not a whole-model benchmark.
~98.5% refers to final-logit KL damage removed, not BF16 capability retained.
Scale recovery is not yet on GPU — roughly 38% of full-block work remains on CPU.
An earlier ~21× GPU figure was measured against a single-threaded baseline and is withdrawn.
Co-adaptation is not demonstrated to improve final quality. No Q2–Q8 ladder has been run.

STATED PLAINLY

What these tables do not show: any whole-model result, any comparison against Q2 through Q8, and any benchmark a third party would recognise. The KL figures are against BF16 on one block of one model, over 101 positions. They are real and they are small. They are not evidence that a model compressed this way is good — only that this block stopped being broken.