train_svamp_101112_1760638004

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.2142
• Num Input Tokens Seen: 1430592

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
2.4702	1.0	158	2.4417	71552
1.5492	2.0	316	1.5887	142960
0.8332	3.0	474	0.9196	214432
0.4267	4.0	632	0.5329	286000
0.2682	5.0	790	0.3736	357888
0.1618	6.0	948	0.3068	429456
0.1109	7.0	1106	0.2751	501136
0.1032	8.0	1264	0.2582	573104
0.1209	9.0	1422	0.2467	644752
0.1224	10.0	1580	0.2368	716192
0.0463	11.0	1738	0.2299	787200
0.1375	12.0	1896	0.2260	858736
0.1195	13.0	2054	0.2231	930160
0.0919	14.0	2212	0.2209	1001792
0.0831	15.0	2370	0.2172	1073248
0.0659	16.0	2528	0.2180	1144672
0.0959	17.0	2686	0.2169	1216160
0.0386	18.0	2844	0.2155	1287728
0.0871	19.0	3002	0.2143	1359120
0.0834	20.0	3160	0.2142	1430592

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4