aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-INT8

Qwen3-ForcedAligner-0.6B — CoreML INT8

CoreML conversion of Qwen/Qwen3-ForcedAligner-0.6B with INT8 kmeans palettization on the two transformer bundles, for Apple Silicon.

Predicts word-level timestamps for audio + text pairs in a single non-autoregressive forward pass.

Files

File	Role	Size
audio_encoder.mlmodelc / .mlpackage	24-layer block-attention encoder (INT8-palettized); fixed 30 s mel input	~304 MB
text_decoder.mlmodelc / .mlpackage	28-layer non-AR decoder + 5000-class classify head (INT8-palettized); fixed T=768	~427 MB
embed_tokens.fp16.bin	Raw little-endian fp16 token embedding table, shape [152 064, 1024] (not palettized)	~297 MB
config.json	Runtime config — variant, classify_num, timestamp_segment_time, fixed shapes	small
vocab.json, merges.txt, tokenizer_config.json	Qwen3 BPE tokenizer files	~5 MB

The embed-tokens table is shipped as a raw fp16 binary rather than a CoreML package — the Swift runtime memory-maps the file and gathers rows with vImageConvert_Planar16FtoPlanarF, which costs ~0.5 ms per alignment instead of the ~70 ms an mlpackage round-trip would cost. Keeping the embedding at fp16 (not palettized) preserves classify-time accuracy across the full vocabulary; the palettization budget is spent on the two large transformer bundles, where it matters most for disk and ANE memory.

Usage (Swift via speech-swift)

import Qwen3ASR

let aligner = try await CoreMLForcedAligner.fromPretrained(
    modelId: "aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-INT8")
let aligned = try aligner.align(
    audio: pcmSamples,
    text: "Can you guarantee that the replacement part will be shipped tomorrow?",
    sampleRate: 16000,
    language: "English")

CLI: speech align audio.wav --engine coreml --aligner-model aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-INT8

Historical performance (M2 Max, 64 GB, debug build, 20 s clip)

Metric	Value
RTF	0.014 (69× faster than real-time)
Peak RSS	697 MB
Median alignment time	~289 ms

Per-stage profile (COREML_ALIGN_PROFILE=1):

mel=150ms  encoder=78ms  embedding=0.5ms  splice=0ms  decoder=54ms  argmax=7ms

The Swift decoder defaults to .cpuAndGPU; the encoder defaults to .cpuAndNeuralEngine. Explicit .cpuOnly uses the separate CPU-compatible bundle below.

Architecture

Same model as the upstream Qwen3-ForcedAligner. Inference is non-autoregressive: one forward pass over a chat-template prompt (system + audio + assistant + word<timestamp><timestamp> slots), then argmax at the timestamp positions, then LIS monotonicity correction.

The causal mask is baked into the exported graph as a constant with a finite -1e4 fill value (not -inf) so the fp16 softmax cannot produce NaN.

Variants

• This repo: INT8 — kmeans-palettized text decoder + audio encoder, ~880 MB, similar RTF, ~35% lower peak memory than FP16.
• Companion: aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-FP16 — full precision, ~1.7 GB, highest fidelity.

For pure MLX paths (no CoreML), see the 4bit, 5bit, 8bit, and bf16 siblings under the same HuggingFace org.

CPU-compatible decoder

text_decoder_cpu_v1.mlmodelc and text_decoder_cpu_v1.mlpackage are additional decoder bundles for CPU-only execution. Their 113 RMS-normalization calculations use Float32; learned weights and the existing model input/output types are preserved. The original text_decoder bundles remain available for GPU execution.

The legacy decoder returns all-zero logits with CPU-only execution on macOS 15.8.2. The Float32-normalization INT8 graph matches all 22 corrected word boundaries in the reference fixture on that runtime. The release validation manifest records the supported checks; validation on other devices is separate.

Updated speech-swift loaders select the versioned bundle for explicit .cpuOnly execution. Run one online model load to populate it; an existing legacy cache does not contain the new CPU bundle. Local/custom exports can provide the same bundle name to select it. The versioned bundle must not be changed in place; future graph changes require a new bundle version.

CPU release validation

Both versioned INT8 and FP16 CPU bundles match all 22 corrected reference word boundaries on macOS 15.8.2 (24H232), with finite, nonzero logits. The checks use the packaged model files at immutable commits; see the verification run. Local M5 Pro/macOS 26.6.2 CPU and existing GPU paths also match the reference boundaries. Physical-iPhone validation remains separate. The historical performance figures above predate these CPU bundles and do not describe CPU-only execution.