train_hellaswag_101112_1760638083

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 0.0606
• Num Input Tokens Seen: 218373904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0089	1.0	8979	0.0911	10920768
0.0957	2.0	17958	0.0606	21838176
0.1047	3.0	26937	0.0671	32764400
0.0001	4.0	35916	0.0873	43678848
0.0939	5.0	44895	0.0949	54599824
0.0267	6.0	53874	0.1401	65511392
0.0	7.0	62853	0.1377	76440704
0.0018	8.0	71832	0.1116	87358976
0.1397	9.0	80811	0.1159	98270752
0.0	10.0	89790	0.1133	109182912
0.0	11.0	98769	0.1682	120100304
0.0	12.0	107748	0.1336	131010176
0.0	13.0	116727	0.1828	141940544
0.0	14.0	125706	0.2001	152867360
0.0	15.0	134685	0.2017	163792736
0.0	16.0	143664	0.2334	174711344
0.0	17.0	152643	0.2356	185629312
0.0	18.0	161622	0.2409	196536784
0.0	19.0	170601	0.2430	207451408
0.0	20.0	179580	0.2429	218373904

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4