spills reasoning into code
with qwen's official parameter settings for code/precise tasks, the model starts spilling its chain of thought in code. i have noticed this happens in comments. it starts to describe what it is about to do next, in a comment, (// in cpp) and then it will fall into a long conversation, quite funny at time where it is attempting to talk itself out of the loop. only experienced it with this intel-provdided model.
is there a quick fix or tuning of parameters that helps?
Would you please provide some prompts (and parameter settings for code/precise tasks you mentioned)that can reproduce this issue?
it is a frequent occurrence. serving on vllm, fp8 kv cache, nothing really special. as conversation drags, it occurs often. i feel it should be easy to reproduce? i am on nightly vllm and flashinfer, if that helps.