fix: update `generation_config.json`to default to stochastic sampling (temp 0.15)

#24
by casinca - opened

Hello,

This is a mirror PR for Devstral-Small, based on: https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512/discussions/18

adds the required hparam arguments to enable stochastic sampling (temp 0.15) rather than greedy decoding in the generation_config.json.
So when users load the mistralai/Devstral-Small-2-24B-Instruct-2512 model, they automatically get the default sampling settings intended by Mistral.

Motivation: Not all users might know about these sampling hparams and what they do, defaulting to what Mistral recommends, could lower complaints about potential poor generations/model performances.

Opened as a separate PR in case you want to keep it greedy by default, this is originally linked to: https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512/discussions/9
(It's true that low temp is close to greedy but this is still stochastic nonetheless)

Mistral AI_ org

Thanks !

juliendenize changed pull request status to merged

Sign up or log in to comment