Qwen

/

Qwen3-VL-32B-Instruct

like 115

Follow

Qwen 56.5k

Edit: The latest llama.cpp seems to have fixed this.

---

I'm posting this here instead of a specific GGUF repo as this affects all GGUFs.

I've been testing the Qwen3 VL 32B GGUFs with Japanese OCR tasks and they've been having issues while the same model on Qwen Chat is able to OCR the text correctly. Here's an example to demonstrate:

Prompt:
OCR the text in this image.

Edit: Try the below prompt instead, it makes the difference more apparent
Transcribe the text in this image.

First, the correct OCR from Qwen3 VL 32B on Qwen Chat:

ざこ
ざこ
ざこざこ
くだらない存在
あわれだね

However, when the GGUF conversion of the model is given the same task it produces incorrect OCR for the first 2 text bubbles. There are a few characters it might say it is instead, but they're all wrong.

Qwen3 VL 32B Instruct:

ごめん
ごめん
ざこざこ
くだらない存在
あわれだね

or

ご
ご
ざこざこ
くだらない存在
あわれだね

Giving the same task to Qwen3 VL 32B Thinking is even more revealing:

So, let's look at the image. It's a pink background with white speech bubbles containing Japanese text. Let's read each bubble one by one.

First bubble: ごき (but wait, the characters are ごき? Wait no, let's check. The first bubble has ごき? Wait, no, the characters are ごき? Wait, no, looking at the image: first bubble is ごき? Wait, no, the first two characters are ご and き? Wait, no, let's see. Wait, the first bubble: the text is ごき? Wait, no, the first bubble's text is ごき? Wait, no, the first bubble has two characters: ご and き? Wait, no, let's check again.

Wait, the first speech bubble: the text is ごき? Wait, no, the first bubble: the characters are ご and き, but written as ごき? Wait, no, in the image, the first bubble has ごき? Wait, no, let's look carefully. The first bubble: the text is ごき? Wait, no, the first two characters are ご and き, but maybe it's ごき. Wait, no, let's list each bubble:

1. First bubble: ごき (but wait, the order: in Japanese, the characters are written in the bubble. Let's check each bubble's text.

First bubble: ごき → but wait, no, the first bubble's text is ごき? Wait, no, looking at the image, the first speech bubble has ご and き, so ごき. Wait, no, the first bubble: the characters are ご and き, so ごき. Wait, but let's check the actual image. The first bubble: the text is ごき? Wait, no, the first bubble's text is ごき. Wait, no, the first two characters: ご and き, so ごき. Wait, no, the first bubble's text is

As you can see in the reasoning block the model is really struggling to parse the first characters and loops infinitely.


Key Points:

• I've also tested GGUFs of other models in the Qwen3 VL series, such as 30B-A3B and 8B dense. They're able to OCR the text perfectly, so this is likely an issue specific to the 32B version.
• This occurs on all GGUFs by all GGUF providers (both text model and mmproj), including those I converted myself. I even tried to use different llama.cpp binaries, including official releases and those I built myself. There is no improvement. So this is likely an issue further up.
• All testing was done with Q8_0 quants (f16/bf16 mmproj) except for once when I downloaded unsloth's Q5_XL quant to see if that helped (it didn't).
• I used a variety of sampler settings when testing.
• I only tested Japanese OCR, but my intuition is that if there are issues with this then there are probably issues with other things as well.

Those are my findings, and I hope they are able to help the Qwen team in some way. This is a pretty quick test to perform, so I encourage others to try it and see for themselves.