CAT-YOKO-KERNEL

Operator source, tests and measured evidence for CAT-YOKO on ROCm. This archive contains no model weights. Matching cat_yoko/ core source is included so the standalone runners can hash and import the exact implementation they test. Model overlays are published separately in CAT-YOKO.

The 2026-10-04 update includes opt-in document-split FP32 attention and cached BF16 CPU-shadow Adam, alongside shared-storage frozen MoE. These are ordinary PyTorch modules with process-local contexts. They are not newly built or validated kernels.get_kernel() packages. The existing HF Kernels source project, configuration, tests and direct-source validation are preserved; this Hub repository remains model-type and has no published build variants or fatwheel at this revision.

Latest measured adoption

Hardware: RX 7900 XTX / gfx1100, PyTorch 2.9.1+ROCm 6.4. B0 real-text updates use BF16 parameters, FP32 MATH attention and CPU FP32 Adam moments.

Measurement	Result	Scope
Split attention	Sample median latency reduction: 6.64% warm, 7.78% cold	Unweighted real-row operator samples; includes input backward and assembly
Cached CPU Adam	11.02% lower steady-state optimizer latency	Same-state fixed-gradient microcheck; about 418 MiB extra host cache
Both, synchronized complete updates	+3.032% median, +2.546% aggregate throughput	Same-source old/new/old, 20 updates each, first 5 discarded
Both, Trainer-loop median	+3.033% throughput	The same short bracketed window

The automatic 5% performance gate was not met. Adoption followed the explicit user request after numerical and recovery validation. These short-window measurements do not establish sustained or corpus-wide speedup.

Split attention's initial GPU microcheck and the later long/single-document check each passed 14/14 tested comparisons with byte-exact output/dQ/dK/dV. The initial check omitted four memory-filtered fixtures; the supplemental check had zero skips and covers those long-document and native fallback cases. The cached optimizer passed five fixed-gradient updates of all 132 parameters with byte-exact BF16 weights and 264 FP32 moments.

Full-model native/reference-repeat checks precede installation. All 132 gradients, global gradient, loss and sampled outputs pass the original limits: loss absolute error 0.02; output/per-tensor/global gradient relative L2 5%. Against the unoptimized native graph, the packed/shared candidate is not byte-exact: global gradient relative L2 is 1.4949%, loss error 0.000927925.

A separate strict deterministic control executes two real updates from the same step-53143 source. Old and new methods end with all 132 BF16 weights and 264 FP32 moments byte-exact, maximum moment difference zero. Ordinary-policy old/old 20-step trajectories already differ; the two-step control does not claim byte-identical ordinary-policy trajectories. Production retains deterministic_algorithms=False; --deterministic-training is a separate default-off option for controlled arithmetic checks.

Adoption report, operator microchecks, and selected evidence index retain the limitations and actual source hashes. The archived production audit is a snapshot at 2026-10-04T05:31:24Z: 106 updates to 53249 and the first verified new-backend checkpoint at 53201. It is not a completed-corpus or current live status claim. No language-quality gain is attributed to the operators.

Run and test

Install a ROCm-compatible PyTorch build before the remaining dependencies. Use Python 3.10 or newer; run from this repository root.

git clone https://huggingface.co/AvrovaDonz/CAT-YOKO-KERNEL
cd CAT-YOKO-KERNEL
python -m pip install -r requirements.txt
python -m unittest operators.rocm.test_split_attention operators.rocm.test_packed_attention operators.rocm.test_cpu_adam_cached
python -m unittest operators.rocm.test_round3_candidate_bench operators.rocm.test_round3_switch

With an external MiniCPM5-2B-Base, a full B0 trainable overlay including native Adam/RNG/cursor state, and separate packed training/evaluation data:

python operators/rocm/round3_candidate_bench.py   --base /path/to/MiniCPM5-2B-Base --resume /path/to/trainable.pt   --out /path/to/separate-experiment --data /path/to/train.bin   --eval-data /path/to/eval.bin --eos-id 1   --moe-layout shared-storage --packed-attention   --parity-seqs 4096 --seq-len 4096 --run-steps 20   --deterministic-parity --reference-repeat --save-optim   --split-attention --cached-cpu-adam --save-every 0   --save-every-seconds 300 --keep-last 3 --wait-gpu-idle

Use --parity-only to validate without updates. Omit one or both new operator flags for individual or old-operator comparisons. Synchronized comparisons require --sync-update-timing on every variant. The runner forwards native checkpoint formats and keeps 300-second/three-checkpoint defaults. The private BF16 host cache is rebuilt after optimizer load and is not serialized; writes through unversioned storage aliases require explicit cache invalidation. Shared-storage/cached-Adam continuation is B0 only; rebuild and validate the native graph before B1/B2. Attention remains FP32 MATH; importing installs no patches. Under ROCm, PyTorch still names the device cuda.

History and provenance

The 2026-10-02 card and old ledgers remain historical evidence. Their 4.45× result measured shared storage plus removal of block offload on DummyStream, not the latest operator-only gain or real-text quality. Historical download scripts and docs/ROCM_TRAIN.md are retained; use the current model repository's release manifest to select an optimizer-bearing real-text overlay rather than an old DummyStream release.

PROVENANCE.json binds current sources and the preceding Hub revision; SHA256SUMS covers the complete archive except itself. Executed frozen runner copies and later local audit gates are distinguished. The Apache-2.0 license and existing HF builder source/validation files are preserved. Upstream: CAT-YOKO.