a4e127996b33b76d46245a3f7c62e65a

This model is a fine-tuned version of distilbert/distilbert-base-multilingual-cased on the contemmcm/trec dataset. It achieves the following results on the evaluation set:

• Loss: 0.3003
• Data Size: 1.0
• Epoch Runtime: 6.2788
• Accuracy: 0.9417
• F1 Macro: 0.9497

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Accuracy	F1 Macro
No log	0	0	1.8197	0	0.7426	0.1	0.0722
No log	1	170	1.7374	0.0078	1.0096	0.2771	0.0779
No log	2	340	1.6659	0.0156	1.4510	0.3063	0.1278
No log	3	510	1.5055	0.0312	1.5827	0.5125	0.3271
No log	4	680	1.0641	0.0625	1.6405	0.6979	0.5777
0.0732	5	850	0.4897	0.125	2.0105	0.8771	0.7368
0.0732	6	1020	0.2355	0.25	2.6787	0.925	0.7760
0.2675	7	1190	0.1985	0.5	3.8552	0.9458	0.9361
0.2	8.0	1360	0.1434	1.0	6.4770	0.9583	0.9635
0.114	9.0	1530	0.1352	1.0	6.1807	0.9688	0.9719
0.0557	10.0	1700	0.1454	1.0	6.2951	0.9729	0.9685
0.0295	11.0	1870	0.2158	1.0	6.1631	0.9625	0.9681
0.0353	12.0	2040	0.1926	1.0	6.2203	0.9708	0.9741
0.0192	13.0	2210	0.3003	1.0	6.2788	0.9417	0.9497

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1