File size: 22,134 Bytes
e6c0e6b d74d520 e6c0e6b d74d520 e6c0e6b d74d520 e6c0e6b d74d520 e6c0e6b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 |
---
library_name: transformers
language:
- he
license: apache-2.0
base_model: ivrit-ai/whisper-large-v3-turbo
tags:
- hf-asr-leaderboard
- generated_from_trainer
metrics:
- wer
model-index:
- name: he-cantillation
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# he-cantillation
This model is a fine-tuned version of [ivrit-ai/whisper-large-v3-turbo](https://huggingface.co/ivrit-ai/whisper-large-v3-turbo) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 5.0244
- Wer: 97.8059
- Avg Precision Exact: 0.0463
- Avg Recall Exact: 0.1014
- Avg F1 Exact: 0.0598
- Avg Precision Letter Shift: 0.0622
- Avg Recall Letter Shift: 0.1383
- Avg F1 Letter Shift: 0.0805
- Avg Precision Word Level: 0.0777
- Avg Recall Word Level: 0.1656
- Avg F1 Word Level: 0.0970
- Avg Precision Word Shift: 0.1542
- Avg Recall Word Shift: 0.3497
- Avg F1 Word Shift: 0.1988
- Precision Median Exact: 0.0227
- Recall Median Exact: 0.0625
- F1 Median Exact: 0.0357
- Precision Max Exact: 1.0
- Recall Max Exact: 1.0
- F1 Max Exact: 1.0
- Precision Min Exact: 0.0
- Recall Min Exact: 0.0
- F1 Min Exact: 0.0
- Precision Min Letter Shift: 0.0
- Recall Min Letter Shift: 0.0
- F1 Min Letter Shift: 0.0
- Precision Min Word Level: 0.0
- Recall Min Word Level: 0.0
- F1 Min Word Level: 0.0
- Precision Min Word Shift: 0.0
- Recall Min Word Shift: 0.0
- F1 Min Word Shift: 0.0
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 2
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 60000
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Avg Precision Exact | Avg Recall Exact | Avg F1 Exact | Avg Precision Letter Shift | Avg Recall Letter Shift | Avg F1 Letter Shift | Avg Precision Word Level | Avg Recall Word Level | Avg F1 Word Level | Avg Precision Word Shift | Avg Recall Word Shift | Avg F1 Word Shift | Precision Median Exact | Recall Median Exact | F1 Median Exact | Precision Max Exact | Recall Max Exact | F1 Max Exact | Precision Min Exact | Recall Min Exact | F1 Min Exact | Precision Min Letter Shift | Recall Min Letter Shift | F1 Min Letter Shift | Precision Min Word Level | Recall Min Word Level | F1 Min Word Level | Precision Min Word Shift | Recall Min Word Shift | F1 Min Word Shift |
|:-------------:|:------:|:-----:|:---------------:|:--------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------:|:------------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:|
| No log | 0.0002 | 1 | 7.1581 | 109.4023 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0.0568 | 0.3754 | 2500 | 2.0970 | 96.9729 | 0.0613 | 0.0775 | 0.0671 | 0.0842 | 0.1078 | 0.0923 | 0.1037 | 0.1324 | 0.1135 | 0.2206 | 0.2995 | 0.2485 | 0.0345 | 0.05 | 0.04 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0314 | 0.7508 | 5000 | 2.3109 | 96.4492 | 0.0709 | 0.0953 | 0.0792 | 0.0932 | 0.1273 | 0.1045 | 0.1089 | 0.1502 | 0.1225 | 0.2270 | 0.3279 | 0.2605 | 0.0357 | 0.0588 | 0.0435 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.018 | 1.1261 | 7500 | 2.5849 | 97.4587 | 0.0458 | 0.0776 | 0.0548 | 0.0637 | 0.1096 | 0.0766 | 0.0775 | 0.1319 | 0.0926 | 0.1673 | 0.2959 | 0.2038 | 0.025 | 0.0455 | 0.0331 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0075 | 1.5015 | 10000 | 3.1851 | 97.9489 | 0.0397 | 0.0824 | 0.0507 | 0.0539 | 0.1151 | 0.0693 | 0.0664 | 0.1414 | 0.0847 | 0.1476 | 0.3210 | 0.1893 | 0.0204 | 0.05 | 0.0317 | 1.0 | 1.0 | 0.8571 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0153 | 1.8769 | 12500 | 2.7215 | 96.5921 | 0.0645 | 0.0950 | 0.0740 | 0.0868 | 0.1307 | 0.1001 | 0.1080 | 0.1624 | 0.1228 | 0.2293 | 0.3653 | 0.2688 | 0.0323 | 0.0625 | 0.0417 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0108 | 2.2523 | 15000 | 3.2672 | 97.2092 | 0.0544 | 0.1084 | 0.0685 | 0.0710 | 0.1448 | 0.0901 | 0.0848 | 0.1708 | 0.1064 | 0.1695 | 0.3629 | 0.2190 | 0.0270 | 0.0714 | 0.0392 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0077 | 2.6276 | 17500 | 3.2673 | 97.1625 | 0.0551 | 0.1014 | 0.0684 | 0.0749 | 0.1379 | 0.0922 | 0.0895 | 0.1631 | 0.1091 | 0.1845 | 0.3500 | 0.2292 | 0.0278 | 0.0625 | 0.0392 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0179 | 3.0030 | 20000 | 3.2682 | 97.2967 | 0.0516 | 0.0918 | 0.0634 | 0.0712 | 0.1274 | 0.0875 | 0.0869 | 0.1538 | 0.1061 | 0.1856 | 0.3430 | 0.2305 | 0.0278 | 0.0625 | 0.0385 | 1.0 | 0.8125 | 0.6667 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0066 | 3.3784 | 22500 | 3.5714 | 97.3726 | 0.0454 | 0.0955 | 0.0584 | 0.0621 | 0.1316 | 0.0796 | 0.0742 | 0.1559 | 0.0944 | 0.1542 | 0.3349 | 0.1987 | 0.0227 | 0.0556 | 0.0345 | 0.8571 | 1.0 | 0.8571 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0042 | 3.7538 | 25000 | 3.8114 | 97.9270 | 0.0404 | 0.0894 | 0.0525 | 0.0558 | 0.1247 | 0.0726 | 0.0677 | 0.1516 | 0.0883 | 0.1420 | 0.3347 | 0.1890 | 0.0222 | 0.0588 | 0.0345 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0014 | 4.1291 | 27500 | 4.0215 | 96.8212 | 0.0506 | 0.1030 | 0.0654 | 0.0667 | 0.1376 | 0.0863 | 0.0801 | 0.1627 | 0.1023 | 0.1651 | 0.3500 | 0.2142 | 0.0263 | 0.0667 | 0.0385 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0021 | 4.5045 | 30000 | 4.0509 | 98.0262 | 0.0395 | 0.0905 | 0.0523 | 0.0545 | 0.1243 | 0.0716 | 0.0666 | 0.1525 | 0.0873 | 0.1443 | 0.3387 | 0.1904 | 0.0217 | 0.0625 | 0.0345 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0028 | 4.8799 | 32500 | 3.9264 | 98.0685 | 0.0381 | 0.0844 | 0.0492 | 0.0513 | 0.1175 | 0.0667 | 0.0632 | 0.1427 | 0.0812 | 0.1319 | 0.3158 | 0.1729 | 0.0 | 0.0 | 0.0 | 0.7778 | 1.0 | 0.8750 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0027 | 5.2553 | 35000 | 4.5942 | 97.9532 | 0.0446 | 0.0991 | 0.0585 | 0.0593 | 0.1323 | 0.0770 | 0.0731 | 0.1586 | 0.0924 | 0.1465 | 0.3347 | 0.1893 | 0.0227 | 0.0667 | 0.0348 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0019 | 5.6306 | 37500 | 4.1899 | 97.8249 | 0.0437 | 0.1073 | 0.0581 | 0.0569 | 0.1405 | 0.0753 | 0.0679 | 0.1646 | 0.0884 | 0.1375 | 0.3415 | 0.1811 | 0.0204 | 0.0588 | 0.0331 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0011 | 6.0060 | 40000 | 4.5501 | 98.1939 | 0.0396 | 0.0917 | 0.0528 | 0.0555 | 0.1270 | 0.0728 | 0.0689 | 0.1550 | 0.0890 | 0.1460 | 0.3409 | 0.1913 | 0.0196 | 0.0588 | 0.0317 | 0.7778 | 1.0 | 0.8235 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0008 | 6.3814 | 42500 | 4.3012 | 97.9882 | 0.0421 | 0.0935 | 0.0552 | 0.0577 | 0.1284 | 0.0753 | 0.0710 | 0.1548 | 0.0909 | 0.1450 | 0.3337 | 0.1898 | 0.0217 | 0.0588 | 0.0339 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0007 | 6.7568 | 45000 | 4.2077 | 97.8409 | 0.0456 | 0.0924 | 0.0583 | 0.0623 | 0.1262 | 0.0792 | 0.0756 | 0.1512 | 0.0949 | 0.1578 | 0.3287 | 0.2014 | 0.0244 | 0.0625 | 0.0357 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0007 | 7.1321 | 47500 | 4.5387 | 97.6308 | 0.0510 | 0.1027 | 0.0652 | 0.0679 | 0.1377 | 0.0863 | 0.0815 | 0.1632 | 0.1026 | 0.1655 | 0.3474 | 0.2120 | 0.0263 | 0.0667 | 0.0385 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.001 | 7.5075 | 50000 | 4.3632 | 98.0670 | 0.0457 | 0.0830 | 0.0565 | 0.0635 | 0.1169 | 0.0785 | 0.0792 | 0.1419 | 0.0955 | 0.1735 | 0.3260 | 0.2133 | 0.0253 | 0.0556 | 0.0357 | 1.0 | 1.0 | 0.8 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0004 | 7.8829 | 52500 | 4.4452 | 97.6060 | 0.0445 | 0.0961 | 0.0580 | 0.0599 | 0.1288 | 0.0772 | 0.0723 | 0.1532 | 0.0921 | 0.1513 | 0.3325 | 0.1962 | 0.0233 | 0.0625 | 0.0351 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0001 | 8.2583 | 55000 | 4.5731 | 97.7607 | 0.0475 | 0.1018 | 0.0614 | 0.0635 | 0.1380 | 0.0821 | 0.0787 | 0.1659 | 0.0992 | 0.1603 | 0.3549 | 0.2071 | 0.025 | 0.0667 | 0.0377 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0001 | 8.6336 | 57500 | 4.9017 | 97.6148 | 0.0474 | 0.1006 | 0.0609 | 0.0625 | 0.1370 | 0.0811 | 0.0777 | 0.1637 | 0.0978 | 0.1550 | 0.3454 | 0.2009 | 0.025 | 0.0667 | 0.0377 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0 | 9.0090 | 60000 | 5.0244 | 97.8059 | 0.0463 | 0.1014 | 0.0598 | 0.0622 | 0.1383 | 0.0805 | 0.0777 | 0.1656 | 0.0970 | 0.1542 | 0.3497 | 0.1988 | 0.0227 | 0.0625 | 0.0357 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
### Framework versions
- Transformers 4.49.0
- Pytorch 2.7.0+cu126
- Datasets 2.12.0
- Tokenizers 0.20.1
|