|
|
--- |
|
|
library_name: exllamav2 |
|
|
license: apache-2.0 |
|
|
license_link: https://huggingface.co/Qwen/Qwen3-235B-A22B/blob/main/LICENSE |
|
|
pipeline_tag: text-generation |
|
|
base_model: Qwen/Qwen3-235B-A22B |
|
|
base_model_relation: quantized |
|
|
tags: |
|
|
- exl2 |
|
|
--- |
|
|
|
|
|
Exllamav2 quantization of [Qwen/Qwen3-235B-A22B](https://huggingface.co/Qwen/Qwen3-235B-A22B) |
|
|
|
|
|
Quantized using commit 68976a0 of the dev branch of [exllamav2](https://github.com/turboderp-org/exllamav2). Support for this architecture does not appear to be in the main branch as of this writing. To use this model, please either build the dev branch from source, or wait for a future release. |
|
|
|
|
|
[2.25 bpw h6](https://huggingface.co/MikeRoz/Qwen3-235B-A22B-exl2/tree/2.25bpw_H6) 63.580 GiB |
|
|
[3.00 bpw h6](https://huggingface.co/MikeRoz/Qwen3-235B-A22B-exl2/tree/3.00bpw_H6) 83.410 GiB |
|
|
[4.00 bpw h6](https://huggingface.co/MikeRoz/Qwen3-235B-A22B-exl2/tree/4.00bpw_H6) 110.628 GiB |
|
|
[measurement.json](https://huggingface.co/MikeRoz/Qwen3-235B-A22B-exl2/blob/main/measurement.json) |
|
|
|
