Please quantize the model.

#14
by VaLtEc-BoY - opened

Quantize the model to FP8, Q6, Q8, Q4.TKs!!!

Thank you very much!

Sign up or log in to comment