Preliminary quants for the model (Q2_K_S is an early quant and is not imatrixed, the rest are).

Perplexity scores (20 chunks):

Q8_0: Final estimate: PPL = 8.1500 +/- 0.30810

IQ4_NL: Final estimate: PPL = 8.2485 +/- 0.31326

IQ3_XS: Final estimate: PPL = 8.3266 +/- 0.30716

IQ2_M: Final estimate: PPL = 9.1081 +/- 0.33962

IQ2_XXS: Final estimate: PPL = 10.2483 +/- 0.38654 (I'd guess for the desperate)

GGUF

Model size

80B params

Architecture

qwen3next

Hardware compatibility

2-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ilintar/Qwen3-Next-80B-A3B-Instruct-GGUF

Base model

Quantized

(46)

this model