Index-Echo-S2TT-2B-GGUF

Official GGUF conversion of the text LLM backbone (llm/) of IndexTeam/Index-Echo-S2TT-2B, part of the Index-Echo speech-to-text translation (S2TT) model family by bilibili.

Note: LLM backbone only. The audio tower, connector and speech-synthesis stack are not included - llama.cpp cannot run the full speech pipeline. For full speech-to-text translation (S2TT) inference, use the original checkpoint or the FP8 build in Index-Echo-S2TT-2B-FP8, which ships the complete pipeline with an FP8-quantized LLM.

Converted with llama.cpp (master, 2026-10); static post-training quantization, all bit-widths in this single repository.

Provided files

(sorted by size)

File	Size	Notes
mmproj-Q8_0	0.36 GB	multi-modal projector supplement
mmproj-f16	0.67 GB	multi-modal projector supplement
Q2_K	0.99 GB	2-bit, significant quality loss
Q3_K_S	1.05 GB	3-bit, noticeable quality loss
Q3_K_M	1.13 GB	3-bit, noticeable quality loss
Q3_K_L	1.20 GB	3-bit, noticeable quality loss
IQ4_XS	1.23 GB	4-bit I-quant, small
Q4_K_S	1.25 GB	4-bit, slightly smaller/faster
Q4_K_M	1.31 GB	4-bit, good balance, recommended
Q5_K_S	1.42 GB	5-bit, low quality loss
Q5_K_M	1.45 GB	5-bit, low quality loss
Q6_K	1.61 GB	6-bit, very low quality loss
Q8_0	2.08 GB	8-bit, near-lossless
f16	3.90 GB	16-bit, lossless conversion baseline

The mmproj files are the multi-modal projector for the vision tower; they are only needed for image-input use with llama-mtmd-cli. For text-only translation, the plain GGUF files suffice.

Usage

# Run the recommended Q4_K_M directly:
llama serve -hf IndexTeam/Index-Echo-S2TT-2B-GGUF:Q4_K_M
llama cli -hf IndexTeam/Index-Echo-S2TT-2B-GGUF:Q4_K_M

Consistency validation

Before release, each quantization tier was validated on GPU (NVIDIA A100) against the F16 conversion: per-token KL divergence / RMS delta-p via llama-perplexity, plus greedy-generation spot checks against the original BF16 weights (transformers reference). Generation outputs of Q4_K_M matched the reference almost verbatim.

Converted and published by the Index team, 2026-10-03.