Hugging Face's logo

vazish
/
all-MiniLM-L6-v2-fine-tuned

Sentence Similarity
sentence-transformers
ONNX
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:429643
loss:CosineSimilarityLoss
Eval Results
text-embeddings-inference
Model card Files
xet
Community
vazish commited on
Commit
35dd1fc
·
unverified ·
1 Parent(s): 7dd909b

update model

Browse files
Files changed (12) hide show
  1. README.md +148 -58
  2. config.json +4 -2
  3. model.safetensors +1 -1
  4. onnx/model.onnx +3 -0
  5. onnx/model_bnb4.onnx +3 -0
  6. onnx/model_fp16.onnx +3 -0
  7. onnx/model_int8.onnx +3 -0
  8. onnx/model_q4.onnx +3 -0
  9. onnx/model_q4f16.onnx +3 -0
  10. onnx/model_quantized.onnx +3 -0
  11. onnx/model_uint8.onnx +3 -0
  12. quantize_config.json +18 -0
README.md CHANGED
@@ -4,37 +4,56 @@ tags:
4
  - sentence-similarity
5
  - feature-extraction
6
  - generated_from_trainer
7
- - dataset_size:49800
8
  - loss:CosineSimilarityLoss
9
  base_model: sentence-transformers/all-MiniLM-L6-v2
10
  widget:
11
- - source_sentence: 99designs - Logo, Web, and Graphic Design Contests for Freelancers
12
  sentences:
13
- - Twitch - Live Streaming Platform for Gamers, Creators, and Communities
14
- - Yandex Mail - Russian Email Service with Built-In Translation
15
- - arXiv.org - Preprint Server for Physics, Mathematics, and Computer Science
16
- - source_sentence: BBC News - Comprehensive Global Reporting and Analysis
17
  sentences:
18
- - Indeed - Aggregated Job Listings, Company Reviews, and Salaries
19
- - ClickUp - All-in-One Productivity and Project Management Platform
20
- - Motherly - Empowering Articles and Classes for Modern Mothers
21
- - source_sentence: Trivago - Compare Accommodation Prices Across Multiple Sites
22
  sentences:
23
- - Mint - Budgeting App and Bill Tracking with Automatic Bank Sync
24
- - Dailymotion - Global Video Hosting and Sharing Platform
25
- - Zero to Three - Early Childhood Development and Care Information
26
- - source_sentence: ProtonMail - End-to-End Encrypted Email for Enhanced Privacy
27
  sentences:
28
- - Kelley Blue Book - Vehicle Valuations, Consumer Reviews, and Insights
29
- - GitHub - Code Hosting, Pull Requests, and Collaborative Development
30
- - Delish - Fun, Creative Recipes and Easy Meal Inspiration
31
- - source_sentence: AngelList Talent - Startups Hiring Engineers, Designers, and Marketers
32
  sentences:
33
- - Stitcher - Podcast App for Comedy, News, True Crime, and More
34
- - TaskRabbit - Hire Freelancers for Home Improvement and Errands
35
- - Dribbble - Discover UI Shots, Branding Projects, and Design Concepts
36
  pipeline_tag: sentence-similarity
37
  library_name: sentence-transformers
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  ---
39
 
40
  # SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
@@ -63,7 +82,7 @@ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [s
63
 
64
  ```
65
  SentenceTransformer(
66
- (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
67
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
68
  (2): Normalize()
69
  )
@@ -84,12 +103,12 @@ Then you can load this model and run inference.
84
  from sentence_transformers import SentenceTransformer
85
 
86
  # Download from the 🤗 Hub
87
- model = SentenceTransformer("vazish/all-MiniLM-L6-v2-fine-tuned")
88
  # Run inference
89
  sentences = [
90
- 'AngelList Talent - Startups Hiring Engineers, Designers, and Marketers',
91
- 'Stitcher - Podcast App for Comedy, News, True Crime, and More',
92
- 'Dribbble - Discover UI Shots, Branding Projects, and Design Concepts',
93
  ]
94
  embeddings = model.encode(sentences)
95
  print(embeddings.shape)
@@ -125,6 +144,19 @@ You can finetune this model on your own dataset.
125
  *List how the model may foreseeably be misused and address what users ought not to do with the model.*
126
  -->
127
 
 
 
 
 
 
 
 
 
 
 
 
 
 
128
  <!--
129
  ## Bias, Risks and Limitations
130
 
@@ -166,7 +198,8 @@ You can finetune this model on your own dataset.
166
  ### Training Hyperparameters
167
  #### Non-Default Hyperparameters
168
 
169
- - `num_train_epochs`: 2
 
170
  - `multi_dataset_batch_sampler`: round_robin
171
 
172
  #### All Hyperparameters
@@ -176,8 +209,8 @@ You can finetune this model on your own dataset.
176
  - `do_predict`: False
177
  - `eval_strategy`: no
178
  - `prediction_loss_only`: True
179
- - `per_device_train_batch_size`: 8
180
- - `per_device_eval_batch_size`: 8
181
  - `per_gpu_train_batch_size`: None
182
  - `per_gpu_eval_batch_size`: None
183
  - `gradient_accumulation_steps`: 1
@@ -189,7 +222,7 @@ You can finetune this model on your own dataset.
189
  - `adam_beta2`: 0.999
190
  - `adam_epsilon`: 1e-08
191
  - `max_grad_norm`: 1
192
- - `num_train_epochs`: 2
193
  - `max_steps`: -1
194
  - `lr_scheduler_type`: linear
195
  - `lr_scheduler_kwargs`: {}
@@ -264,7 +297,7 @@ You can finetune this model on your own dataset.
264
  - `fp16_backend`: auto
265
  - `push_to_hub_model_id`: None
266
  - `push_to_hub_organization`: None
267
- - `mp_parameters`:
268
  - `auto_find_batch_size`: False
269
  - `full_determinism`: False
270
  - `torchdynamo`: None
@@ -291,32 +324,89 @@ You can finetune this model on your own dataset.
291
  </details>
292
 
293
  ### Training Logs
294
- | Epoch | Step | Training Loss |
295
- |:------:|:-----:|:-------------:|
296
- | 0.0803 | 500 | 0.0197 |
297
- | 0.1606 | 1000 | 0.0153 |
298
- | 0.2410 | 1500 | 0.0069 |
299
- | 0.3213 | 2000 | 0.0028 |
300
- | 0.4016 | 2500 | 0.0012 |
301
- | 0.4819 | 3000 | 0.0008 |
302
- | 0.5622 | 3500 | 0.0007 |
303
- | 0.6426 | 4000 | 0.0006 |
304
- | 0.7229 | 4500 | 0.0006 |
305
- | 0.8032 | 5000 | 0.0005 |
306
- | 0.8835 | 5500 | 0.0004 |
307
- | 0.9639 | 6000 | 0.0004 |
308
- | 1.0442 | 6500 | 0.0004 |
309
- | 1.1245 | 7000 | 0.0003 |
310
- | 1.2048 | 7500 | 0.0002 |
311
- | 1.2851 | 8000 | 0.0002 |
312
- | 1.3655 | 8500 | 0.0005 |
313
- | 1.4458 | 9000 | 0.0001 |
314
- | 1.5261 | 9500 | 0.0001 |
315
- | 1.6064 | 10000 | 0.0001 |
316
- | 1.6867 | 10500 | 0.0001 |
317
- | 1.7671 | 11000 | 0.0001 |
318
- | 1.8474 | 11500 | 0.0001 |
319
- | 1.9277 | 12000 | 0.0001 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
320
 
321
 
322
  ### Framework Versions
@@ -361,4 +451,4 @@ You can finetune this model on your own dataset.
361
  ## Model Card Contact
362
 
363
  *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
364
- -->
 
4
  - sentence-similarity
5
  - feature-extraction
6
  - generated_from_trainer
7
+ - dataset_size:429643
8
  - loss:CosineSimilarityLoss
9
  base_model: sentence-transformers/all-MiniLM-L6-v2
10
  widget:
11
+ - source_sentence: Oracle Cloud - Infrastructure and Platform Services for Enterprises
12
  sentences:
13
+ - PulseAudio - Ubuntu Wiki
14
+ - Documentation page not found - Read the Docs
15
+ - Dwarf Fortress beginner tips - Video Games on Sports Illustrated
16
+ - source_sentence: Suggest opt in User Test - Google Slides
17
  sentences:
18
+ - ReleaseEngineering/TryServer - MozillaWiki
19
+ - Dwarf Fortress beginner tips - Video Games on Sports Illustrated
20
+ - Tutanota - Private Mailbox with End-to-End Encryption and Calendar
21
+ - source_sentence: https://portal.naviabenefits.com/part/prioritytasks.aspx
22
  sentences:
23
+ - What to Expect - Pregnancy and Parenting Tips, Week-by-Week Guides
24
+ - Parents.com - Articles, Recipes, and Ideas for Family Activities
25
+ - Pinterest - Boards for Collecting and Sharing Inspiration on Any Topic
26
+ - source_sentence: ‎Apple Music - Web Player
27
  sentences:
28
+ - BMW Connected Drive - Home Assistant
29
+ - Mary Stewart Phillips (1862-1928) - Find a Grave Memorial
30
+ - Sky Sports - Football, Formula 1, Cricket, and More
31
+ - source_sentence: Tidal - High-Fidelity Music Streaming with Master Quality Audio
32
  sentences:
33
+ - Walmart - Everyday Low Prices on Groceries, Electronics, and More
34
+ - Notion - Integrated Workspace for Notes, Tasks, Databases, and Wikis
35
+ - Ambient Dreams Playlist on Amazon Music
36
  pipeline_tag: sentence-similarity
37
  library_name: sentence-transformers
38
+ metrics:
39
+ - pearson_cosine
40
+ - spearman_cosine
41
+ model-index:
42
+ - name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
43
+ results:
44
+ - task:
45
+ type: semantic-similarity
46
+ name: Semantic Similarity
47
+ dataset:
48
+ name: Unknown
49
+ type: unknown
50
+ metrics:
51
+ - type: pearson_cosine
52
+ value: 0.9822505655251419
53
+ name: Pearson Cosine
54
+ - type: spearman_cosine
55
+ value: 0.2607864200673379
56
+ name: Spearman Cosine
57
  ---
58
 
59
  # SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
 
82
 
83
  ```
84
  SentenceTransformer(
85
+ (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
86
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
87
  (2): Normalize()
88
  )
 
103
  from sentence_transformers import SentenceTransformer
104
 
105
  # Download from the 🤗 Hub
106
+ model = SentenceTransformer("vazish/all-MiniLM-L6-v2-fine-tuned_0")
107
  # Run inference
108
  sentences = [
109
+ 'Tidal - High-Fidelity Music Streaming with Master Quality Audio',
110
+ 'Walmart - Everyday Low Prices on Groceries, Electronics, and More',
111
+ 'Notion - Integrated Workspace for Notes, Tasks, Databases, and Wikis',
112
  ]
113
  embeddings = model.encode(sentences)
114
  print(embeddings.shape)
 
144
  *List how the model may foreseeably be misused and address what users ought not to do with the model.*
145
  -->
146
 
147
+ ## Evaluation
148
+
149
+ ### Metrics
150
+
151
+ #### Semantic Similarity
152
+
153
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
154
+
155
+ | Metric | Value |
156
+ |:--------------------|:-----------|
157
+ | pearson_cosine | 0.9823 |
158
+ | **spearman_cosine** | **0.2608** |
159
+
160
  <!--
161
  ## Bias, Risks and Limitations
162
 
 
198
  ### Training Hyperparameters
199
  #### Non-Default Hyperparameters
200
 
201
+ - `per_device_train_batch_size`: 32
202
+ - `per_device_eval_batch_size`: 32
203
  - `multi_dataset_batch_sampler`: round_robin
204
 
205
  #### All Hyperparameters
 
209
  - `do_predict`: False
210
  - `eval_strategy`: no
211
  - `prediction_loss_only`: True
212
+ - `per_device_train_batch_size`: 32
213
+ - `per_device_eval_batch_size`: 32
214
  - `per_gpu_train_batch_size`: None
215
  - `per_gpu_eval_batch_size`: None
216
  - `gradient_accumulation_steps`: 1
 
222
  - `adam_beta2`: 0.999
223
  - `adam_epsilon`: 1e-08
224
  - `max_grad_norm`: 1
225
+ - `num_train_epochs`: 3
226
  - `max_steps`: -1
227
  - `lr_scheduler_type`: linear
228
  - `lr_scheduler_kwargs`: {}
 
297
  - `fp16_backend`: auto
298
  - `push_to_hub_model_id`: None
299
  - `push_to_hub_organization`: None
300
+ - `mp_parameters`:
301
  - `auto_find_batch_size`: False
302
  - `full_determinism`: False
303
  - `torchdynamo`: None
 
324
  </details>
325
 
326
  ### Training Logs
327
+ | Epoch | Step | Training Loss | spearman_cosine |
328
+ |:------:|:-----:|:-------------:|:---------------:|
329
+ | 0.0372 | 500 | 0.0218 | - |
330
+ | 0.0745 | 1000 | 0.0151 | - |
331
+ | 0.1117 | 1500 | 0.0113 | - |
332
+ | 0.1490 | 2000 | 0.0076 | - |
333
+ | 0.1862 | 2500 | 0.0063 | - |
334
+ | 0.2234 | 3000 | 0.0054 | - |
335
+ | 0.2607 | 3500 | 0.0045 | - |
336
+ | 0.2979 | 4000 | 0.0041 | - |
337
+ | 0.3351 | 4500 | 0.0027 | - |
338
+ | 0.3724 | 5000 | 0.0028 | - |
339
+ | 0.4096 | 5500 | 0.0026 | - |
340
+ | 0.4469 | 6000 | 0.0021 | - |
341
+ | 0.4841 | 6500 | 0.0019 | - |
342
+ | 0.5213 | 7000 | 0.0022 | - |
343
+ | 0.5586 | 7500 | 0.0017 | - |
344
+ | 0.5958 | 8000 | 0.0018 | - |
345
+ | 0.6331 | 8500 | 0.0015 | - |
346
+ | 0.6703 | 9000 | 0.0015 | - |
347
+ | 0.7075 | 9500 | 0.0018 | - |
348
+ | 0.7448 | 10000 | 0.0014 | - |
349
+ | 0.7820 | 10500 | 0.0017 | - |
350
+ | 0.8192 | 11000 | 0.0012 | - |
351
+ | 0.8565 | 11500 | 0.0014 | - |
352
+ | 0.8937 | 12000 | 0.001 | - |
353
+ | 0.9310 | 12500 | 0.0011 | - |
354
+ | 0.9682 | 13000 | 0.001 | - |
355
+ | 1.0054 | 13500 | 0.0009 | - |
356
+ | 1.0427 | 14000 | 0.0011 | - |
357
+ | 1.0799 | 14500 | 0.001 | - |
358
+ | 1.1172 | 15000 | 0.0009 | - |
359
+ | 1.1544 | 15500 | 0.0008 | - |
360
+ | 1.1916 | 16000 | 0.001 | - |
361
+ | 1.2289 | 16500 | 0.0011 | - |
362
+ | 1.2661 | 17000 | 0.0011 | - |
363
+ | 1.3033 | 17500 | 0.0006 | - |
364
+ | 1.3406 | 18000 | 0.0011 | - |
365
+ | 1.3778 | 18500 | 0.0008 | - |
366
+ | 1.4151 | 19000 | 0.0011 | - |
367
+ | 1.4523 | 19500 | 0.0009 | - |
368
+ | 1.4895 | 20000 | 0.0011 | - |
369
+ | 1.5268 | 20500 | 0.0009 | - |
370
+ | 1.5640 | 21000 | 0.0009 | - |
371
+ | 1.6013 | 21500 | 0.0008 | - |
372
+ | 1.6385 | 22000 | 0.0005 | - |
373
+ | 1.6757 | 22500 | 0.001 | - |
374
+ | 1.7130 | 23000 | 0.0008 | - |
375
+ | 1.7502 | 23500 | 0.0007 | - |
376
+ | 1.7874 | 24000 | 0.0007 | - |
377
+ | 1.8247 | 24500 | 0.0008 | - |
378
+ | 1.8619 | 25000 | 0.001 | - |
379
+ | 1.8992 | 25500 | 0.0009 | - |
380
+ | 1.9364 | 26000 | 0.0008 | - |
381
+ | 1.9736 | 26500 | 0.0009 | - |
382
+ | 2.0109 | 27000 | 0.0007 | - |
383
+ | 2.0481 | 27500 | 0.0006 | - |
384
+ | 2.0854 | 28000 | 0.0007 | - |
385
+ | 2.1226 | 28500 | 0.0006 | - |
386
+ | 2.1598 | 29000 | 0.0007 | - |
387
+ | 2.1971 | 29500 | 0.001 | - |
388
+ | 2.2343 | 30000 | 0.0006 | - |
389
+ | 2.2715 | 30500 | 0.0006 | - |
390
+ | 2.3088 | 31000 | 0.001 | - |
391
+ | 2.3460 | 31500 | 0.0007 | - |
392
+ | 2.3833 | 32000 | 0.0008 | - |
393
+ | 2.4205 | 32500 | 0.0006 | - |
394
+ | 2.4577 | 33000 | 0.0007 | - |
395
+ | 2.4950 | 33500 | 0.0007 | - |
396
+ | 2.5322 | 34000 | 0.001 | - |
397
+ | 2.5694 | 34500 | 0.0007 | - |
398
+ | 2.6067 | 35000 | 0.0007 | - |
399
+ | 2.6439 | 35500 | 0.0008 | - |
400
+ | 2.6812 | 36000 | 0.0007 | - |
401
+ | 2.7184 | 36500 | 0.0006 | - |
402
+ | 2.7556 | 37000 | 0.0007 | - |
403
+ | 2.7929 | 37500 | 0.0007 | - |
404
+ | 2.8301 | 38000 | 0.0005 | - |
405
+ | 2.8674 | 38500 | 0.0009 | - |
406
+ | 2.9046 | 39000 | 0.0006 | - |
407
+ | 2.9418 | 39500 | 0.0007 | - |
408
+ | 2.9791 | 40000 | 0.0008 | - |
409
+ | -1 | -1 | - | 0.2608 |
410
 
411
 
412
  ### Framework Versions
 
451
  ## Model Card Contact
452
 
453
  *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
454
+ -->
config.json CHANGED
@@ -1,10 +1,12 @@
1
  {
2
- "_name_or_path": "sentence-transformers/all-MiniLM-L6-v2",
 
3
  "architectures": [
4
  "BertModel"
5
  ],
6
  "attention_probs_dropout_prob": 0.1,
7
  "classifier_dropout": null,
 
8
  "gradient_checkpointing": false,
9
  "hidden_act": "gelu",
10
  "hidden_dropout_prob": 0.1,
@@ -19,7 +21,7 @@
19
  "pad_token_id": 0,
20
  "position_embedding_type": "absolute",
21
  "torch_dtype": "float32",
22
- "transformers_version": "4.48.2",
23
  "type_vocab_size": 2,
24
  "use_cache": true,
25
  "vocab_size": 30522
 
1
  {
2
+ "_attn_implementation_autoset": true,
3
+ "_name_or_path": "/content/model",
4
  "architectures": [
5
  "BertModel"
6
  ],
7
  "attention_probs_dropout_prob": 0.1,
8
  "classifier_dropout": null,
9
+ "export_model_type": "transformer",
10
  "gradient_checkpointing": false,
11
  "hidden_act": "gelu",
12
  "hidden_dropout_prob": 0.1,
 
21
  "pad_token_id": 0,
22
  "position_embedding_type": "absolute",
23
  "torch_dtype": "float32",
24
+ "transformers_version": "4.46.3",
25
  "type_vocab_size": 2,
26
  "use_cache": true,
27
  "vocab_size": 30522
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2dd1d2164e062efaed33b1435c12840cf0469da12d8930615b4ab37b996ec8a1
3
  size 90864192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cce795656a2f5901d70c4ea6568b7c98a136a9f653ec4e8e499ac26270beffcb
3
  size 90864192
onnx/model.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:506ac8fdb54c3d5401fc03fb7cc135553857779e2cc518dfb3341cf113ebe257
3
+ size 90448255
onnx/model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77f96020b60846dbd7f86538280364c6f5e7d1fce2a1753debe0fa28a5b2a1dc
3
+ size 53958494
onnx/model_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:492935c401e7ae513a959c9433dd204f159b501593bf732507b94d73081ae14d
3
+ size 45317631
onnx/model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d8575a3ba5d8a44884f56579d2046d5883e4bbef1923840f7b6b561bc6a6918
3
+ size 22999753
onnx/model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:235764145072d6bc5e6ab9f61f17e893f47dd02cc6eef3a5a1967fbfbae31523
3
+ size 54621818
onnx/model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5a462c3bbf68ad9d80c3367342d67fba38b871f683bd78a741479e9df6017dba
3
+ size 30061282
onnx/model_quantized.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d8575a3ba5d8a44884f56579d2046d5883e4bbef1923840f7b6b561bc6a6918
3
+ size 22999753
onnx/model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d985ea995e76faa0d7f542db38a3e08ecfedfc8b9cfa1143e8c13398803019d
3
+ size 22999753
quantize_config.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "modes": [
3
+ "fp16",
4
+ "q8",
5
+ "int8",
6
+ "uint8",
7
+ "q4",
8
+ "q4f16",
9
+ "bnb4"
10
+ ],
11
+ "per_channel": true,
12
+ "reduce_range": true,
13
+ "block_size": null,
14
+ "is_symmetric": true,
15
+ "accuracy_level": null,
16
+ "quant_type": 1,
17
+ "op_block_list": null
18
+ }