Qwen/Qwen3-VL-8B-Instruct
Image-Text-to-Text
•
9B
•
Updated
•
290k
•
•
334
Hoang Duc Viet
ducviet00
AI & ML interests
None yet
Recent Activity
new activity
7 days ago
florence-community/Florence-2-base:How were the models converted?
liked
a model
8 days ago
Qwen/Qwen3-VL-8B-Instruct
liked
a model
8 days ago
PaddlePaddle/PaddleOCR-VL
Organizations
How were the models converted?
1
#1 opened 8 days ago
by
xzuyn
liked
2
models
8 days ago
liked
a
model
12 days ago
upvoted
a
collection
about 1 month ago
liked
a
model
about 1 month ago
florence checkpoints
🚀
1
1
#1 opened about 2 months ago
by
mervenoyan
How to convert non finetuned florence2 model weights to this new native format?
2
#1 opened about 2 months ago
by
fcakyon
liked
2
models
about 2 months ago
liked
a
model
2 months ago
upvoted
a
paper
2 months ago
updated
2
models
2 months ago
liked
a
model
2 months ago
Can we upstream this architecture to the Transformers repo?
#5 opened 2 months ago
by
ducviet00
liked
a
model
2 months ago
reacted to
badaoui's
post with 🚀
2 months ago
Post
3183
Is there a "one-size-fits-all" recipe for quantizing Large Language Models? 🤔
As part of my ongoing work in mixed-precision quantization, I've been exploring this question by measuring layer-by-layer sensitivity. The goal is to see if we can find universal rules for which layers can be quantized aggressively without impacting performance.The results are fascinating and reveal two key insights:
1️⃣ Sensitivity profiles are like architectural "fingerprints." Models from the same family share strikingly similar sensitivity patterns. As you can see in the charts below for the Gemma and SmolLM families, the ranking and relative sensitivity of the layers remain remarkably consistent. This suggests that the underlying architecture is a primary driver of a model's quantization behavior.
2️⃣ A "universal" mixed-precision quantization strategy is challenging. While models within a family are similar, these "fingerprints" change dramatically when comparing different architectures like LLaMA, Qwen, and StableLM. This highlights the difficulty in creating a generalized mixed-precision configuration that works optimally across all model families.
However, there is one near-universal truth we uncovered: the mlp.down_proj layer consistently emerges as one of the most sensitive components across all models studied.
This finding strongly resonates with the work in "The Super Weight in Large Language Models" (by Mengxia Yu et al.). The paper identifies that functionally critical parameters, or "super weights," are concentrated in these down_proj layers. Our empirical results provide clear validation for this theory, showing these layers are highly intolerant to precision loss.
In short, while every architecture has a unique sensitivity profile, a fingerprint shaped not only by its core design but also by its specific training dataset and optimization approach, some components remain universally critical!
What are your thoughts?
As part of my ongoing work in mixed-precision quantization, I've been exploring this question by measuring layer-by-layer sensitivity. The goal is to see if we can find universal rules for which layers can be quantized aggressively without impacting performance.The results are fascinating and reveal two key insights:
1️⃣ Sensitivity profiles are like architectural "fingerprints." Models from the same family share strikingly similar sensitivity patterns. As you can see in the charts below for the Gemma and SmolLM families, the ranking and relative sensitivity of the layers remain remarkably consistent. This suggests that the underlying architecture is a primary driver of a model's quantization behavior.
2️⃣ A "universal" mixed-precision quantization strategy is challenging. While models within a family are similar, these "fingerprints" change dramatically when comparing different architectures like LLaMA, Qwen, and StableLM. This highlights the difficulty in creating a generalized mixed-precision configuration that works optimally across all model families.
However, there is one near-universal truth we uncovered: the mlp.down_proj layer consistently emerges as one of the most sensitive components across all models studied.
This finding strongly resonates with the work in "The Super Weight in Large Language Models" (by Mengxia Yu et al.). The paper identifies that functionally critical parameters, or "super weights," are concentrated in these down_proj layers. Our empirical results provide clear validation for this theory, showing these layers are highly intolerant to precision loss.
In short, while every architecture has a unique sensitivity profile, a fingerprint shaped not only by its core design but also by its specific training dataset and optimization approach, some components remain universally critical!
What are your thoughts?
- 4 replies
liked
a
dataset
3 months ago
upvoted
an
article
3 months ago
Article
Vision Language Model Alignment in TRL ⚡️
•
96