---
library_name: transformers
license: mit
base_model: roberta-base
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: SST-2-FULL_FT-seed20
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# SST-2-FULL_FT-seed20

This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2716
- Accuracy: 0.9450

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Accuracy |
|:-------------:|:------:|:-----:|:---------------:|:--------:|
| 0.4018        | 0.0950 | 200   | 0.2495          | 0.9106   |
| 0.2966        | 0.1900 | 400   | 0.2164          | 0.9232   |
| 0.2766        | 0.2850 | 600   | 0.2299          | 0.9186   |
| 0.2381        | 0.3800 | 800   | 0.2115          | 0.9312   |
| 0.2268        | 0.4751 | 1000  | 0.2481          | 0.9186   |
| 0.2281        | 0.5701 | 1200  | 0.2841          | 0.9220   |
| 0.2133        | 0.6651 | 1400  | 0.2135          | 0.9300   |
| 0.2094        | 0.7601 | 1600  | 0.2200          | 0.9289   |
| 0.2083        | 0.8551 | 1800  | 0.1958          | 0.9381   |
| 0.1899        | 0.9501 | 2000  | 0.2282          | 0.9278   |
| 0.1811        | 1.0451 | 2200  | 0.2251          | 0.9255   |
| 0.1462        | 1.1401 | 2400  | 0.2134          | 0.9220   |
| 0.1543        | 1.2352 | 2600  | 0.2590          | 0.9243   |
| 0.1451        | 1.3302 | 2800  | 0.2907          | 0.9197   |
| 0.1481        | 1.4252 | 3000  | 0.2570          | 0.9220   |
| 0.1382        | 1.5202 | 3200  | 0.3125          | 0.9243   |
| 0.1543        | 1.6152 | 3400  | 0.2263          | 0.9312   |
| 0.1427        | 1.7102 | 3600  | 0.2303          | 0.9312   |
| 0.1412        | 1.8052 | 3800  | 0.2113          | 0.9404   |
| 0.1623        | 1.9002 | 4000  | 0.2011          | 0.9358   |
| 0.1411        | 1.9952 | 4200  | 0.2147          | 0.9335   |
| 0.1122        | 2.0903 | 4400  | 0.2686          | 0.9358   |
| 0.1047        | 2.1853 | 4600  | 0.2368          | 0.9346   |
| 0.1067        | 2.2803 | 4800  | 0.2754          | 0.9323   |
| 0.1138        | 2.3753 | 5000  | 0.2170          | 0.9358   |
| 0.1079        | 2.4703 | 5200  | 0.2897          | 0.9220   |
| 0.1039        | 2.5653 | 5400  | 0.2880          | 0.9255   |
| 0.1217        | 2.6603 | 5600  | 0.2261          | 0.9346   |
| 0.0957        | 2.7553 | 5800  | 0.2597          | 0.9358   |
| 0.1075        | 2.8504 | 6000  | 0.2263          | 0.9358   |
| 0.0994        | 2.9454 | 6200  | 0.2328          | 0.9415   |
| 0.0969        | 3.0404 | 6400  | 0.2429          | 0.9358   |
| 0.0809        | 3.1354 | 6600  | 0.2401          | 0.9427   |
| 0.0815        | 3.2304 | 6800  | 0.2416          | 0.9438   |
| 0.0836        | 3.3254 | 7000  | 0.2341          | 0.9438   |
| 0.078         | 3.4204 | 7200  | 0.2346          | 0.9438   |
| 0.0783        | 3.5154 | 7400  | 0.2831          | 0.9415   |
| 0.0797        | 3.6105 | 7600  | 0.2649          | 0.9358   |
| 0.0838        | 3.7055 | 7800  | 0.2499          | 0.9415   |
| 0.0792        | 3.8005 | 8000  | 0.3017          | 0.9358   |
| 0.0769        | 3.8955 | 8200  | 0.2704          | 0.9404   |
| 0.0838        | 3.9905 | 8400  | 0.2652          | 0.9369   |
| 0.056         | 4.0855 | 8600  | 0.3180          | 0.9323   |
| 0.0504        | 4.1805 | 8800  | 0.3403          | 0.9358   |
| 0.0607        | 4.2755 | 9000  | 0.3380          | 0.9312   |
| 0.0688        | 4.3705 | 9200  | 0.2830          | 0.9404   |
| 0.0608        | 4.4656 | 9400  | 0.2693          | 0.9438   |
| 0.0559        | 4.5606 | 9600  | 0.2850          | 0.9346   |
| 0.0603        | 4.6556 | 9800  | 0.2716          | 0.9450   |
| 0.0588        | 4.7506 | 10000 | 0.2574          | 0.9438   |
| 0.0598        | 4.8456 | 10200 | 0.2678          | 0.9415   |
| 0.062         | 4.9406 | 10400 | 0.2675          | 0.9427   |


### Framework versions

- Transformers 4.54.1
- Pytorch 2.5.1+cu121
- Datasets 4.0.0
- Tokenizers 0.21.4