1340d36ede0c81b6593f76f4319ef191

This model is a fine-tuned version of albert/albert-large-v2 on the nyu-mll/glue dataset. It achieves the following results on the evaluation set:

• Loss: 2.3085
• Data Size: 1.0
• Epoch Runtime: 12.9833
• Mse: 2.3093
• Mae: 1.2894
• R2: -0.0330

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Mse	Mae	R2
No log	0	0	8.3843	0	1.5129	8.3856	2.4799	-2.7512
No log	1	179	3.4279	0.0078	1.8985	3.4289	1.5488	-0.5339
No log	2	358	2.6515	0.0156	1.7839	2.6523	1.3766	-0.1865
No log	3	537	2.4839	0.0312	1.9344	2.4845	1.2990	-0.1114
No log	4	716	2.3451	0.0625	2.2913	2.3459	1.2966	-0.0494
No log	5	895	2.2858	0.125	3.0195	2.2866	1.2935	-0.0229
0.1597	6	1074	2.2858	0.25	4.4264	2.2866	1.2838	-0.0229
2.1809	7	1253	2.6532	0.5	7.3157	2.6539	1.3473	-0.1872
2.0324	8.0	1432	2.7158	1.0	13.0868	2.7164	1.3496	-0.2151
2.2306	9.0	1611	2.3085	1.0	12.9833	2.3093	1.2894	-0.0330

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.1