train_qnli_101112_1760638091

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.0427
• Num Input Tokens Seen: 207147488

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0444	1.0	23567	0.0706	10356896
0.0836	2.0	47134	0.0562	20715296
0.0168	3.0	70701	0.0514	31065184
0.0576	4.0	94268	0.0478	41428128
0.0825	5.0	117835	0.0459	51784320
0.0492	6.0	141402	0.0446	62144160
0.1004	7.0	164969	0.0444	72511552
0.043	8.0	188536	0.0440	82864256
0.0755	9.0	212103	0.0427	93220320
0.0695	10.0	235670	0.0436	103572992
0.0495	11.0	259237	0.0433	113924768
0.0236	12.0	282804	0.0437	124282240
0.0065	13.0	306371	0.0443	134645600
0.0056	14.0	329938	0.0431	145000704
0.0076	15.0	353505	0.0439	155349152
0.0326	16.0	377072	0.0435	165706304
0.0318	17.0	400639	0.0443	176064704
0.0614	18.0	424206	0.0445	186423936
0.0107	19.0	447773	0.0444	196786464
0.0409	20.0	471340	0.0443	207147488

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4