Update config.json for sglang
#3
by
ayachinenefan
- opened
In sglang, the original ignore method is not effective. It needs to be modified by following the configuration format of https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct-FP8.
hi
@ayachinenefan
the vision tower is unquantized which is why the all the vision layers are listed in the ignore list as they are. It seems like you're keeping some layers as is while removing the model. prefix in some cases?
@dsikka Yes, in sglang, using the original config.json will cause the non-quantized layers in the vision part to become invalid, resulting in the vision part also being quantized. To mimic the approach of this link, https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct-FP8, the prefix in the config.json needs to be removed.