aufklarer/Breeze-TTS-2-MLX-bf16

Breeze TTS 2 (BF16)

Derived from Breeze TTS 2 by BreezeBlue and licensed for research and non-commercial use only.

Native MLX checkpoint for speech-swift. English and Chinese synthesis, natural-language voice design, reference-guided voice cloning and direction, mono 24 kHz PCM, and incremental audio chunks.

Source: BreezeBlue/Breeze-TTS-2. The model and self-hosted outputs are restricted to research and non-commercial use. See LICENSE and NOTICE. This is an independently converted checkpoint.

Conversion

Unquantized BF16 Linear weights; embeddings, normalization weights, and depth codebook heads retain upstream precision. The Qwen3 audio tokenizer is copied unchanged. Unused legacy codec and text embeddings are removed; tied audio embeddings are stored once. Immutable provenance, file hashes, and conversion details are in export_config.json. No inference quality or performance claim follows from export alone.

Swift

import BreezeTTS

let model = try await BreezeTTSModel.fromPretrained(modelId: "aufklarer/Breeze-TTS-2-MLX-bf16")
let samples = try await model.generate(
    text: "The quick brown fox jumps over the lazy dog.",
    instruction: "A clear, calm English narrator.")

CLI

speech speak "The quick brown fox jumps over the lazy dog." --engine breeze \
  --breeze-model-id aufklarer/Breeze-TTS-2-MLX-bf16 --instruct "A clear English narrator" --stream -o speech.wav

Voice cloning requires --voice-sample reference.wav --breeze-reference-text "Exact transcript". Add --instruct to direct the reference voice. English and Chinese are inferred from text. Use one generation at a time per model. Reaching the frame cap before EOS raises an error.

Optimized Swift runtime — 2026-10-30

The updated speech-swift runtime uses the same checkpoint weights. Matched release measurements on Apple M5 Pro, 48 GiB unified memory, macOS 26.6.2, with compiled Metal shaders and the shared 2 GiB reusable MLX cache ceiling:

Metric	Measured
Warm median, first decoded PCM	272 ms
Warm median RTF (wall/audio)	0.868
Peak RSS	6.06 GiB
Peak physical footprint	9.22 GiB

The sentence is “The quick brown fox jumps over the lazy dog.” with instruction “A clear, calm English narrator.”, seed 42, CFG 4, and 3 / 25 frame chunks. Timings are medians of three warm requests after an initial request in a fresh process. Each optimized clip is 3.20 s. Model loading and WAV writing are excluded; first PCM also excludes playback buffering and output-device latency. Memory values include loading and all requests. These are workload-specific results.

The default-chunk paragraph measures RTF 0.853 and 272 ms first PCM for 9.20 s of speech. One initial frame reduces short-sentence first PCM to 138 ms (RTF 0.862); 25-frame follow-ups still require a larger playback buffer. BF16 now uses batched matrix-vector projections for two-vector inputs. Accumulation rounding changes the seeded waveform and duration (the original short clip was 3.68 s). Set SPEECH_SWIFT_BREEZE_GEMV=0 before loading to compare the original projection path. Other precisions retain their projection path.

The final matched series produced 45 WAVs across both variants, representing 11 distinct waveforms. All repeats within a fixed runtime and chunk schedule are byte-identical. Independent Qwen3-ASR reports zero normalized WER/CER for all 11, covering the short sentence, paragraph, alternative chunks, cloning, and direction. Fifteen isolated real-model/CLI/tokenizer checks passed with no skips; the local CI-equivalent unit run completed 1,933 tests with 42 skips and no failures. This verifies content, not general acoustic quality or speaker identity.

See the full benchmark and reproduction steps. The dated results, validation.json, and sample files below describe the earlier runtime and remain available as historical records.

Measured Swift smoke test — 2026-10-24

Apple M5 Pro, 48 GiB RAM, macOS 26.6.2, release build with compiled Metal shaders. English voice design, 3.68 seconds of speech, seed 42, guidance 4, 3 frames in the first chunk and 25 thereafter. Three warm repeats.

Metric	Measured
Model load, cached local files	1.391 s
First request, first decoded audio	488 ms
Warm median, first decoded audio	462 ms
Warm median RTF (wall/audio)	1.258
Peak RSS	6.06 GiB
Peak physical footprint	10.75 GiB

RSS can undercount GPU mappings. Physical footprint includes MLX allocations and reusable buffers. Other texts and reference modes can use more memory. All measured clips for this checkpoint have zero normalized WER/CER with aufklarer/Qwen3-ASR-0.6B-MLX-4bit; see validation.json for transcripts, per-request timing, memory, and numerical comparisons with official PyTorch. All 16 measured clips across both published variants match the previous validated WAV files byte for byte. The full local CI unit selection passed 1,739 tests with 41 skips; all ten Breeze runtime/CLI E2E checks and three exporter checks passed. This is a deterministic smoke test, not a general quality or speaker-identity benchmark.

Native Swift runtime, CLI, and regression coverage: speech-swift #526.

Audio samples

• design-bf16

The unchanged Qwen3 audio tokenizer is separately licensed under Apache-2.0; see audio_tokenizer/LICENSE for its source attribution and full license.