~8GB model files compared to BF16 ~11GB, it's almost larger than 8bit quants. Even if the accuracy is better than normal bnb 4bit, I'm not sure it's really worthy
· Sign up or log in to comment
