corupta/DeepSeek-R1-0528-Qwen3-8B-int8-AutoRound-gptq-inc

Model Details

This model is an int8 model with group_size 128 and symmetric quantization of deepseek-ai/DeepSeek-R1-0528-Qwen3-8B generated by intel/auto-round algorithm.

Please follow the license of the original model.

Evaluate the model

auto-round --eval  --model "corupta/DeepSeek-R1-0528-Qwen3-8B-int8-AutoRound-gptq-inc" --eval_bs 16  --tasks leaderboard_ifeval,leaderboard_mmlu_pro,gsm8k,lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,arc_easy,arc_challenge,cmmlu,ceval-valid

Metric	BF16	INT8(auto-round)	INT8 (auto-round-best)
Avg	0.5958	?	?
arc_challenge	0.5137	?	?
arc_easy	0.7908	?	?
boolq	0.8498	?	?
ceval-valid	0.7296	?	?
cmmlu	0.7159	?	?
gsm8k	0.8211	?	?
hellaswag	0.5781	?	?
lambada_openai	0.5544	?	?
leaderboard_ifeval	0.2731	?	?
leaderboard_mmlu_pro	0.4115	?	?
openbookqa	0.3020	?	?
piqa	0.7617	?	?
truthfulqa_mc1	0.3562	?	?
winogrande	0.6835	?	?

Reproduce the model

Here is the sample command to reproduce the model

auto-round
--model_name deepseek-ai/DeepSeek-R1-0528-Qwen3-8B \
--device 0 \
--bits 8 \
--format "auto_gptq" \
--enable_torch_compile \
--dataset codeparrot/github-code-clean \
--output_dir "./tmp_autoround"