fine-tuned-DatasetQAS-IDK-MRC-with-indobert-base-uncased-with-ITTL-with-freeze-LR-1e-05

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0641
• Exact Match: 65.1832
• F1: 70.4300

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
6.31	0.49	36	2.5380	50.0	50.0
3.7784	0.98	72	1.9420	49.8691	49.8691
2.192	1.48	108	1.8160	48.8220	50.2574
2.192	1.97	144	1.7327	48.8220	51.7831
1.9694	2.46	180	1.6016	48.9529	53.9425
1.7482	2.95	216	1.4743	54.4503	58.8011
1.5835	3.45	252	1.3414	57.4607	62.1340
1.5835	3.94	288	1.2884	58.2461	63.2663
1.4053	4.44	324	1.2001	60.4712	65.7454
1.2521	4.93	360	1.1738	60.7330	66.8019
1.2521	5.42	396	1.1551	60.8639	66.5188
1.1668	5.91	432	1.1128	63.8743	68.7421
1.097	6.41	468	1.0863	63.7435	69.1933
1.0251	6.9	504	1.0843	63.0890	68.4132
1.0251	7.4	540	1.0507	66.3613	71.1898
1.0031	7.89	576	1.0732	63.6126	68.7281
0.9616	8.38	612	1.0689	65.0524	70.2879
0.9616	8.87	648	1.0376	66.2304	71.2857
0.9322	9.37	684	1.0474	65.7068	70.7885
0.9227	9.86	720	1.0641	65.1832	70.4300

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2