train_winogrande_1754652171

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 0.7101
• Num Input Tokens Seen: 30830624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2283	0.5	4545	0.2313	1541600
0.2302	1.0	9090	0.2314	3081600
0.2308	1.5	13635	0.2313	4623680
0.2309	2.0	18180	0.2324	6165104
0.2314	2.5	22725	0.2314	7706064
0.2335	3.0	27270	0.2313	9248016
0.2233	3.5	31815	0.2326	10789584
0.2319	4.0	36360	0.2312	12330800
0.2292	4.5	40905	0.2312	13871920
0.2294	5.0	45450	0.2317	15413776
0.2339	5.5	49995	0.2313	16954320
0.2283	6.0	54540	0.2313	18496992
0.2334	6.5	59085	0.2312	20039264
0.2323	7.0	63630	0.2312	21579792
0.2355	7.5	68175	0.2312	23122160
0.2328	8.0	72720	0.2313	24664400
0.2313	8.5	77265	0.2309	26207280
0.2305	9.0	81810	0.2306	27747856
0.23	9.5	86355	0.2304	29287888
0.2315	10.0	90900	0.2307	30830624

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1