train_hellaswag_101112_1760638082

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 0.4609
• Num Input Tokens Seen: 218373904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4551	1.0	8979	0.4630	10920768
0.4637	2.0	17958	0.4625	21838176
0.4666	3.0	26937	0.4629	32764400
0.0354	4.0	35916	0.0856	43678848
0.0452	5.0	44895	0.0701	54599824
0.0537	6.0	53874	0.0636	65511392
0.1214	7.0	62853	0.0612	76440704
0.0844	8.0	71832	0.0587	87358976
0.0144	9.0	80811	0.0609	98270752
0.0191	10.0	89790	0.0642	109182912
0.054	11.0	98769	0.0669	120100304
0.0905	12.0	107748	0.0749	131010176
0.0035	13.0	116727	0.0840	141940544
0.0013	14.0	125706	0.0791	152867360
0.0087	15.0	134685	0.0999	163792736
0.0002	16.0	143664	0.1159	174711344
0.0036	17.0	152643	0.1191	185629312
0.0002	18.0	161622	0.1384	196536784
0.0002	19.0	170601	0.1391	207451408
0.0002	20.0	179580	0.1406	218373904

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4