How did you manage to make quants of these transformers?
#1
by
Winnougan
- opened
I was wondering if it's possible to make GGUF quants of any illustrious model on a Windows machine. I can make fp8 and GGUF using Calcuis's nodes in Comfyui, but I am struggling to find any way to make q4 quants of models in Windows. What tools did you use? Can you point me in the right direction - I have a lot of models that I'd like to quantize. Thanks
I think you are in CivitAI, but here is the article that I've written on how I?m creating the files - https://civitai.com/articles/22952