bart-large-cnn-finetuned-scientific-articles

This model is a fine-tuned version of facebook/bart-large-cnn on the scientific_papers dataset. It achieves the following results on the evaluation set:

• Loss: 2.6456
• Rouge1: 33.8477
• Rouge2: 11.8866
• Rougel: 20.1038
• Rougelsum: 30.5011

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 9
• eval_batch_size: 9
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.3695	1.0	56	2.8464	32.1056	10.3835	18.7541	29.2623
2.7639	2.0	112	2.6667	31.2657	10.758	18.9862	28.3279
2.5169	3.0	168	2.6219	33.226	11.4766	19.5923	30.0664
2.2985	4.0	224	2.6029	32.8562	11.5606	19.8616	29.7606
2.0851	5.0	280	2.6456	33.8477	11.8866	20.1038	30.5011

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1