ajh-code/VoxCPM2-FP8-W8A8

V1 · VoxCPM2 · FP8 W8A8

A complete VoxCPM2 FP8 download: static E4M3 weights, AudioVAE, tokenizer, loader, compiled runtime, llama-compatible TTS API, installer, client, and integrity manifest.

Calibrated static W8A8 cuts the loaded VRAM and complete weight footprint by about 42% while retaining strong English quality in the limited listening evaluation.

Original VoxCPM2 · AJH Code on Hugging Face · Arands.com · updates

---

Download

Component	Purpose	Size
model-fp8.safetensors	Complete static-FP8 VoxCPM2 state, including AudioVAE	2.84 GB
Tokenizer and configuration	Local text processing and exact FP8 reconstruction policy	3.8 MB
Loader, server, launcher, installer, and client	Ready-to-run project-local runtime	<1 MB
Complete repository	No separate BF16 model, codec, or weight download	about 2.84 GB

All weights required by the bundled server are in this repository. The root config.json preserves the VoxCPM2 architecture metadata and is intentionally downloaded by the quick-start path.

Quick start

Tested on Linux x86-64, Python 3.12, CUDA 13.0, PyTorch 2.11.0+cu130, TorchAO 0.17.0+cu130, and NVIDIA Blackwell SM120.

hf download ajh-code/VoxCPM2-FP8-W8A8 \
  --local-dir voxcpm2-fp8-w8a8
cd voxcpm2-fp8-w8a8
./install.sh
./launch_fp8_server.sh

The server listens on all interfaces at port 8080, matching the bundled llama VoxCPM2 TTS contract. Set VOXCPM_HOST=127.0.0.1 to restrict it to the local machine, or set VOXCPM_PORT to use another port. Protect or firewall the service before exposing it beyond a trusted network.

The launcher uses nvidia-smi to select the compatible GPU with the most free VRAM. To choose one exact GPU yourself:

CUDA_VISIBLE_DEVICES=0 ./launch_fp8_server.sh

The first launch performs a one-time TorchInductor compilation warmup. On the tested RTX 5080, artifact load took about 9 seconds and the initial warmup took about 70 seconds. Wait for the server-ready event before sending traffic. Compiled kernels are cached inside the downloaded repository.

Generate speech

The included client uses the same API and has no dependency beyond Python's standard library:

python client.py \
  --text "VoxCPM2 is running from the complete FP8 download." \
  --output output.wav

Equivalent llama/OpenAI-style request:

curl http://127.0.0.1:8080/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "voxcpm",
    "input": "VoxCPM2 is running from the complete FP8 download.",
    "voice": "default",
    "response_format": "wav",
    "seed": 42,
    "cfg_value": 2.0,
    "inference_timesteps": 10,
    "max_steps": 200,
    "temperature": 1.0
  }' \
  --output output.wav

Useful endpoints:

Endpoint	Purpose
GET /health or GET /v1/health	Service health
GET /v1/audio/speech/models	Loaded model metadata
POST /v1/audio/speech	Complete WAV or float32 PCM response
POST /v1/audio/speech/stream	Chunked PCM16 WAV response

The model aliases voxcpm and voxcpm2 are both accepted. The voice field is reserved for compatibility and is currently ignored.

Voice cloning

Use only reference audio you have the right and consent to use. A clean WAV and its exact transcript provide continuation-style cloning:

python client.py \
  --reference-audio reference.wav \
  --reference-text "The exact words spoken in reference.wav." \
  --text "This sentence uses the reference voice." \
  --seed 42 \
  --output clone.wav

Supplying reference audio without --reference-text uses reference-only cloning. Reference WAV data is decoded and VAE-encoded in memory by the server; it is not retained as a request file.

Quantization policy

V1 applies calibrated static per-tensor E4M3 W8A8 FP8 to 424 large linear modules across the base LM, residual LM, local encoder, and local diffusion transformer.

Scope	Stored/executed precision
424 selected large linear projections	static E4M3 FP8 weights and activations
Boundary projections, embeddings, normalization, stop prediction, and small heads	original BF16/FP32 policy
AudioVAE and scalar quantization path	original precision, bundled in the checkpoint
KV cache	BF16, reduced to 512 positions by the serving profile

The artifact stores raw FP8 tensors, weight scales, and calibrated activation scales in safetensors. It reconstructs the TorchAO tensor subclasses locally and does not expand into or download a second BF16 checkpoint.

Measured performance

The fixed-work benchmark generated exactly 32 audio patches / 5.12 seconds of 48 kHz audio with ten diffusion steps. These are local RTX 5080 measurements, not universal hardware claims.

Measurement	Compiled BF16	FP8 W8A8 V1
Median generation time	0.559 s	0.668 s
Real-time factor	0.109	0.130
Loaded PyTorch allocation	5.027 GB	2.909 GB
Peak PyTorch allocation	5.378 GB	3.261 GB
Complete weight/artifact bytes	4.961 GB	2.843 GB

V1 saves 42.1% loaded VRAM, 39.4% peak VRAM, and 42.7% artifact bytes in that comparison. BF16 remained 19.5% faster on the RTX 5080, so this release is a memory/package optimization rather than a claim that FP8 beats BF16 latency on every GPU. Warm short API smoke requests completed in about 0.16–0.28 seconds after compilation.

Limited English quality test

The release candidate was evaluated with six matched, counterbalanced English prompts covering conversation, questions and numbers, expressive contrast, articulation, reflective cadence, and long-form stability.

Blind result	BF16	FP8 V1	Tie
Frozen machine-listening pilot	1	4	1
First human listener	3	2	1

The human listener described the last BF16 selection as very close and judged the overall comparison a perceptual wash after reveal. No severe FP8 regression was reported. This is a one-listener, single-seed pilot—not a broad MOS study or proof of universal parity.

Validated scope

Gate	Result
Standalone packaging	Fresh load succeeds without the BF16 source model or external AudioVAE weights
Native FP8 execution	Static E4M3 projections execute through TorchAO/TorchInductor FP8 paths
API compatibility	WAV, float32 PCM, chunked streaming, cloning, continuation, health, and model discovery passed
English quality	Six-pair machine and one-listener human pilots found no systematic FP8 regression
Hardware	Runtime requires CUDA compute capability 8.9 or newer; tested on RTX 5080 SM120
Multilingual	Upstream supports multiple languages, but this V1 quantized release is qualified only for English

Known limitations

• FP8 V1 is slower than compiled BF16 in the device-matched RTX 5080 test.
• The first launch has a substantial one-time compilation warmup.
• Quality evidence is limited to English, one human listener, and one matched seed per prompt.
• The 512-position serving cache is sized for the tested request envelope, not arbitrarily long synthesis.
• /v1/voxcpm2/init dynamic GGUF reload is not applicable to this fixed FP8 artifact. Restart the server to change artifacts.
• CUDA, PyTorch, TorchAO, and compiler behavior can vary by driver and GPU.

Validate the download

python validate_release.py

MANIFEST.json records the byte size and SHA-256 of every distributed file except itself. Validation also checks the source revision, model checksum, artifact format, 424-module FP8 policy, and safetensors header. Hashing the 2.84 GB checkpoint takes a little while.

License and attribution

VoxCPM2 and this derivative are distributed under the Apache License 2.0. You may use, modify, and redistribute this release subject to that license. Keep the included LICENSE and NOTICE, identify your changes, and do not imply endorsement by OpenBMB or AJH Code.

Attribution request: if you reuse or modify this FP8 W8A8 work, please credit ajh-code and link to this Hugging Face page. This is a request from the release author, not an additional restriction on the Apache-2.0 permissions.

Do not use generated speech or voice cloning for impersonation, fraud, deception, or other harmful activity. Clearly disclose AI-generated audio when appropriate.