turboderp commited on
Commit
9756067
·
verified ·
1 Parent(s): af4b952

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -2
README.md CHANGED
@@ -4,5 +4,19 @@ license: apache-2.0
4
  EXL3 quants of [ERNIE-4.5-300B-A47B-PT](https://huggingface.co/baidu/ERNIE-4.5-300B-A47B-PT)
5
 
6
  [2.00 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/2.0bpw)
7
- [2.25 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/2.25bpw) (appears to be (a bit) broken)
8
- [4.00 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/4.0bpw)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  EXL3 quants of [ERNIE-4.5-300B-A47B-PT](https://huggingface.co/baidu/ERNIE-4.5-300B-A47B-PT)
5
 
6
  [2.00 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/2.0bpw)
7
+ [2.10 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/2.1bpw) (optimized)
8
+ [2.25 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/2.25bpw) (optimized)
9
+ [2.50 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/2.5bpw) (optimized)
10
+ [3.00 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/3.0bpw)
11
+ [4.00 bits per weight](https://huggingface.co/turboderp/ERNIE-4.5-300B-A47B-PT-exl3/tree/4.0bpw)
12
+
13
+
14
+ Quant | Weights/VRAM | Perplexity | KL-div
15
+ ---------|--------------|------------|-----------
16
+ 2.00 bpw | 70.2 GB | 7.4131 | 0.5283
17
+ 2.10 bpw | 73.4 GB | 6.7507 | 0.2202
18
+ 2.25 bpw | 78.6 GB | 6.5576 | 0.2074
19
+ 2.50 bpw | 87.8 GB | 6.3504 | 0.1899
20
+ 3.00 bpw | 104.9 GB | 5.8913 | 0.1547
21
+ 4.00 bpw | 139.5 GB | 5.8132 | 0.0717
22
+ Original | 597.1 GB | 5.4131 |