README.md

metadata

library_name: transformers
license: apache-2.0
base_model: google-t5/t5-base
tags:
  - generated_from_trainer
datasets:
  - generator
metrics:
  - accuracy
model-index:
  - name: google_t5_language_ID
    results:
      - task:
          type: text2text-generation
          name: Sequence-to-sequence Language Modeling
        dataset:
          name: generator
          type: generator
          config: default
          split: train
          args: default
        metrics:
          - type: accuracy
            value: 0.6179074697593216
            name: Accuracy

google_t5_language_ID

This model is a fine-tuned version of google-t5/t5-base on the generator dataset. It achieves the following results on the evaluation set:

• Loss: 0.5429
• Accuracy: 0.6179
• F1 Macro: 0.3389
• F1 Weighted: 0.5774
• Precision Macro: 0.3873
• Recall Macro: 0.3627

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine_with_restarts
• lr_scheduler_warmup_steps: 1000
• training_steps: 60000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro	F1 Weighted	Precision Macro	Recall Macro
0.1943	0.0083	500	0.6981	0.4018	0.3139	0.3488	0.4624	0.3616
0.0812	0.0167	1000	0.7371	0.4086	0.3323	0.3446	0.5179	0.3940
0.049	0.025	1500	0.7806	0.4534	0.3793	0.3793	0.5316	0.4534
0.0518	0.0333	2000	0.5042	0.5845	0.5071	0.5258	0.5576	0.5637
0.0452	0.0417	2500	0.5120	0.6204	0.5554	0.5554	0.6496	0.6204
0.0288	0.05	3000	0.4798	0.6018	0.5230	0.5618	0.6077	0.5603
0.0341	0.0583	3500	0.4764	0.6098	0.5456	0.5658	0.6528	0.5881
0.0762	0.0667	4000	0.4389	0.6251	0.5296	0.5688	0.6091	0.5820
0.0189	0.075	4500	0.4167	0.6681	0.6068	0.6068	0.7167	0.6681
0.0235	0.0833	5000	0.4673	0.6599	0.6018	0.6018	0.7393	0.6599
0.0274	0.0917	5500	0.3304	0.6958	0.6102	0.6555	0.6868	0.6478
0.0198	0.1	6000	0.4752	0.6569	0.5877	0.6095	0.7165	0.6335
0.0246	0.1083	6500	0.4657	0.6540	0.5800	0.6015	0.6400	0.6306
0.0241	0.1167	7000	0.5429	0.6179	0.3389	0.5774	0.3873	0.3627

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1