Mungert

/

granite-3.2-8b-instruct-GGUF

like 4

https://readyforquantum.com/huggingface_gguf_selection_guide.html does this answer your question?

no you see :D
granite-bf16-Q8
granite-bf16_Q8

???

Ah yes I see sorry that is confusing. A bit of mix up in changing format there. Huggingface requested that I reduce disk space usage. So I tried to clean up all the repos seems this has been left behind. The _ is the better format as it boosts important layers to either bf16 or f16 depending on the prefix bf16 or f16 . And the - version only boosts the embeddings layer. I will leave these as they are as its useful to have both. All my newer repos have a very much restricted selection of quants to reduce disk usage. Newer repos only have the _ format ie boosting all important layers.