Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+base_model: gpt2-medium
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GPT2LMHeadModel",
+    "parent_library": "transformers.models.gpt2.modeling_gpt2"
+  },
+  "base_model_name_or_path": "gpt2-medium",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e080b6018d76cf35e380d38c6d986e6242e3d34fb8b0e94622e054c202fb2587
+size 25172088

checkpoint-3125/README.md

@@ -0,0 +1,202 @@
+---
+base_model: gpt2-medium
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

checkpoint-3125/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GPT2LMHeadModel",
+    "parent_library": "transformers.models.gpt2.modeling_gpt2"
+  },
+  "base_model_name_or_path": "gpt2-medium",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-3125/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:808eb6c7977cb683352c4d6e4f4509c0b6bbf25932055faca1b9d2c3fd96995e
+size 25172088

checkpoint-3125/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33b5cff42bfd7cca5a61241e4983ca3d59d3c0fa36588b98b5393a3201b8401f
+size 50372538

checkpoint-3125/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b19bb2054b92e3b341aeff130324419c0b360c2bd88cd65e4248549686c44ce5
+size 14244

checkpoint-3125/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:392855cc9cbe029377262097ef598767921e2a3bc6937822c989a7603ee182c3
+size 1064

checkpoint-3125/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-3125/tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-3125/trainer_state.json

@@ -0,0 +1,2217 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 3125,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0032,
+      "grad_norm": 10.0,
+      "learning_rate": 2.132196162046908e-06,
+      "loss": 28.3701,
+      "step": 10
+    },
+    {
+      "epoch": 0.0064,
+      "grad_norm": 10.0,
+      "learning_rate": 4.264392324093816e-06,
+      "loss": 24.5207,
+      "step": 20
+    },
+    {
+      "epoch": 0.0096,
+      "grad_norm": 10.0,
+      "learning_rate": 6.396588486140726e-06,
+      "loss": 22.6885,
+      "step": 30
+    },
+    {
+      "epoch": 0.0128,
+      "grad_norm": 10.0,
+      "learning_rate": 8.528784648187633e-06,
+      "loss": 25.2666,
+      "step": 40
+    },
+    {
+      "epoch": 0.016,
+      "grad_norm": 10.0,
+      "learning_rate": 1.0660980810234541e-05,
+      "loss": 23.0791,
+      "step": 50
+    },
+    {
+      "epoch": 0.0192,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 1.2793176972281452e-05,
+      "loss": 23.3554,
+      "step": 60
+    },
+    {
+      "epoch": 0.0224,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 1.4925373134328357e-05,
+      "loss": 23.0232,
+      "step": 70
+    },
+    {
+      "epoch": 0.0256,
+      "grad_norm": 10.0,
+      "learning_rate": 1.7057569296375266e-05,
+      "loss": 21.8077,
+      "step": 80
+    },
+    {
+      "epoch": 0.0288,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 1.9189765458422178e-05,
+      "loss": 20.0271,
+      "step": 90
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 2.1321961620469083e-05,
+      "loss": 17.0629,
+      "step": 100
+    },
+    {
+      "epoch": 0.0352,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 2.345415778251599e-05,
+      "loss": 18.8398,
+      "step": 110
+    },
+    {
+      "epoch": 0.0384,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 2.5586353944562904e-05,
+      "loss": 15.8197,
+      "step": 120
+    },
+    {
+      "epoch": 0.0416,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 2.771855010660981e-05,
+      "loss": 14.2088,
+      "step": 130
+    },
+    {
+      "epoch": 0.0448,
+      "grad_norm": 10.0,
+      "learning_rate": 2.9850746268656714e-05,
+      "loss": 14.0394,
+      "step": 140
+    },
+    {
+      "epoch": 0.048,
+      "grad_norm": 10.0,
+      "learning_rate": 3.1982942430703626e-05,
+      "loss": 14.3635,
+      "step": 150
+    },
+    {
+      "epoch": 0.0512,
+      "grad_norm": 10.0,
+      "learning_rate": 3.411513859275053e-05,
+      "loss": 11.0378,
+      "step": 160
+    },
+    {
+      "epoch": 0.0544,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 3.624733475479744e-05,
+      "loss": 10.8982,
+      "step": 170
+    },
+    {
+      "epoch": 0.0576,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 3.8379530916844355e-05,
+      "loss": 9.9566,
+      "step": 180
+    },
+    {
+      "epoch": 0.0608,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 4.051172707889126e-05,
+      "loss": 10.2806,
+      "step": 190
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 4.2643923240938166e-05,
+      "loss": 9.8932,
+      "step": 200
+    },
+    {
+      "epoch": 0.0672,
+      "grad_norm": 10.0,
+      "learning_rate": 4.477611940298508e-05,
+      "loss": 8.1511,
+      "step": 210
+    },
+    {
+      "epoch": 0.0704,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 4.690831556503198e-05,
+      "loss": 8.07,
+      "step": 220
+    },
+    {
+      "epoch": 0.0736,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 4.904051172707889e-05,
+      "loss": 8.0589,
+      "step": 230
+    },
+    {
+      "epoch": 0.0768,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 5.117270788912581e-05,
+      "loss": 6.8857,
+      "step": 240
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 5.330490405117271e-05,
+      "loss": 6.7318,
+      "step": 250
+    },
+    {
+      "epoch": 0.0832,
+      "grad_norm": 10.0,
+      "learning_rate": 5.543710021321962e-05,
+      "loss": 7.0872,
+      "step": 260
+    },
+    {
+      "epoch": 0.0864,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 5.756929637526652e-05,
+      "loss": 6.5861,
+      "step": 270
+    },
+    {
+      "epoch": 0.0896,
+      "grad_norm": 10.0,
+      "learning_rate": 5.970149253731343e-05,
+      "loss": 6.3828,
+      "step": 280
+    },
+    {
+      "epoch": 0.0928,
+      "grad_norm": 10.0,
+      "learning_rate": 6.183368869936035e-05,
+      "loss": 6.0198,
+      "step": 290
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 6.396588486140725e-05,
+      "loss": 6.2239,
+      "step": 300
+    },
+    {
+      "epoch": 0.0992,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 6.609808102345416e-05,
+      "loss": 5.7589,
+      "step": 310
+    },
+    {
+      "epoch": 0.1024,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 6.823027718550106e-05,
+      "loss": 5.4196,
+      "step": 320
+    },
+    {
+      "epoch": 0.1056,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 7.036247334754798e-05,
+      "loss": 5.6412,
+      "step": 330
+    },
+    {
+      "epoch": 0.1088,
+      "grad_norm": 10.0,
+      "learning_rate": 7.249466950959489e-05,
+      "loss": 5.2791,
+      "step": 340
+    },
+    {
+      "epoch": 0.112,
+      "grad_norm": 10.0,
+      "learning_rate": 7.46268656716418e-05,
+      "loss": 5.5189,
+      "step": 350
+    },
+    {
+      "epoch": 0.1152,
+      "grad_norm": 10.0,
+      "learning_rate": 7.675906183368871e-05,
+      "loss": 5.0969,
+      "step": 360
+    },
+    {
+      "epoch": 0.1184,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 7.889125799573562e-05,
+      "loss": 5.2089,
+      "step": 370
+    },
+    {
+      "epoch": 0.1216,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.102345415778252e-05,
+      "loss": 6.2735,
+      "step": 380
+    },
+    {
+      "epoch": 0.1248,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.315565031982943e-05,
+      "loss": 5.1972,
+      "step": 390
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.528784648187633e-05,
+      "loss": 4.791,
+      "step": 400
+    },
+    {
+      "epoch": 0.1312,
+      "grad_norm": 10.0,
+      "learning_rate": 8.742004264392325e-05,
+      "loss": 5.139,
+      "step": 410
+    },
+    {
+      "epoch": 0.1344,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.955223880597016e-05,
+      "loss": 5.1234,
+      "step": 420
+    },
+    {
+      "epoch": 0.1376,
+      "grad_norm": 9.726828575134277,
+      "learning_rate": 9.168443496801706e-05,
+      "loss": 4.763,
+      "step": 430
+    },
+    {
+      "epoch": 0.1408,
+      "grad_norm": 10.0,
+      "learning_rate": 9.381663113006397e-05,
+      "loss": 5.0725,
+      "step": 440
+    },
+    {
+      "epoch": 0.144,
+      "grad_norm": 10.0,
+      "learning_rate": 9.594882729211087e-05,
+      "loss": 4.708,
+      "step": 450
+    },
+    {
+      "epoch": 0.1472,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.808102345415778e-05,
+      "loss": 4.3924,
+      "step": 460
+    },
+    {
+      "epoch": 0.1504,
+      "grad_norm": 8.861834526062012,
+      "learning_rate": 9.998877161464182e-05,
+      "loss": 4.3606,
+      "step": 470
+    },
+    {
+      "epoch": 0.1536,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.987648776105997e-05,
+      "loss": 4.2736,
+      "step": 480
+    },
+    {
+      "epoch": 0.1568,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.97642039074781e-05,
+      "loss": 4.5541,
+      "step": 490
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 8.72944450378418,
+      "learning_rate": 9.965192005389625e-05,
+      "loss": 4.3459,
+      "step": 500
+    },
+    {
+      "epoch": 0.1632,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.95396362003144e-05,
+      "loss": 4.0927,
+      "step": 510
+    },
+    {
+      "epoch": 0.1664,
+      "grad_norm": 9.396402359008789,
+      "learning_rate": 9.942735234673256e-05,
+      "loss": 4.1877,
+      "step": 520
+    },
+    {
+      "epoch": 0.1696,
+      "grad_norm": 6.486421585083008,
+      "learning_rate": 9.931506849315069e-05,
+      "loss": 3.9887,
+      "step": 530
+    },
+    {
+      "epoch": 0.1728,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.920278463956883e-05,
+      "loss": 4.1643,
+      "step": 540
+    },
+    {
+      "epoch": 0.176,
+      "grad_norm": 8.012758255004883,
+      "learning_rate": 9.909050078598698e-05,
+      "loss": 4.0236,
+      "step": 550
+    },
+    {
+      "epoch": 0.1792,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.897821693240512e-05,
+      "loss": 3.9494,
+      "step": 560
+    },
+    {
+      "epoch": 0.1824,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.886593307882327e-05,
+      "loss": 3.7742,
+      "step": 570
+    },
+    {
+      "epoch": 0.1856,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.875364922524142e-05,
+      "loss": 3.817,
+      "step": 580
+    },
+    {
+      "epoch": 0.1888,
+      "grad_norm": 10.0,
+      "learning_rate": 9.864136537165956e-05,
+      "loss": 3.8002,
+      "step": 590
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.852908151807771e-05,
+      "loss": 3.5841,
+      "step": 600
+    },
+    {
+      "epoch": 0.1952,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.841679766449586e-05,
+      "loss": 3.7045,
+      "step": 610
+    },
+    {
+      "epoch": 0.1984,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.8304513810914e-05,
+      "loss": 3.3623,
+      "step": 620
+    },
+    {
+      "epoch": 0.2016,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.819222995733213e-05,
+      "loss": 3.3953,
+      "step": 630
+    },
+    {
+      "epoch": 0.2048,
+      "grad_norm": 7.71815824508667,
+      "learning_rate": 9.807994610375028e-05,
+      "loss": 3.5513,
+      "step": 640
+    },
+    {
+      "epoch": 0.208,
+      "grad_norm": 7.819033145904541,
+      "learning_rate": 9.796766225016843e-05,
+      "loss": 3.3845,
+      "step": 650
+    },
+    {
+      "epoch": 0.2112,
+      "grad_norm": 9.44756031036377,
+      "learning_rate": 9.785537839658657e-05,
+      "loss": 3.2571,
+      "step": 660
+    },
+    {
+      "epoch": 0.2144,
+      "grad_norm": 5.626587390899658,
+      "learning_rate": 9.774309454300472e-05,
+      "loss": 3.1949,
+      "step": 670
+    },
+    {
+      "epoch": 0.2176,
+      "grad_norm": 9.968693733215332,
+      "learning_rate": 9.763081068942287e-05,
+      "loss": 3.05,
+      "step": 680
+    },
+    {
+      "epoch": 0.2208,
+      "grad_norm": 9.028199195861816,
+      "learning_rate": 9.751852683584101e-05,
+      "loss": 3.378,
+      "step": 690
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.740624298225916e-05,
+      "loss": 3.2383,
+      "step": 700
+    },
+    {
+      "epoch": 0.2272,
+      "grad_norm": 10.0,
+      "learning_rate": 9.729395912867731e-05,
+      "loss": 3.2572,
+      "step": 710
+    },
+    {
+      "epoch": 0.2304,
+      "grad_norm": 9.906242370605469,
+      "learning_rate": 9.718167527509545e-05,
+      "loss": 3.2422,
+      "step": 720
+    },
+    {
+      "epoch": 0.2336,
+      "grad_norm": 7.099668025970459,
+      "learning_rate": 9.706939142151358e-05,
+      "loss": 3.199,
+      "step": 730
+    },
+    {
+      "epoch": 0.2368,
+      "grad_norm": 6.1059136390686035,
+      "learning_rate": 9.695710756793174e-05,
+      "loss": 2.9136,
+      "step": 740
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 8.184073448181152,
+      "learning_rate": 9.684482371434989e-05,
+      "loss": 3.0956,
+      "step": 750
+    },
+    {
+      "epoch": 0.2432,
+      "grad_norm": 7.790270805358887,
+      "learning_rate": 9.673253986076802e-05,
+      "loss": 2.9784,
+      "step": 760
+    },
+    {
+      "epoch": 0.2464,
+      "grad_norm": 9.385482788085938,
+      "learning_rate": 9.662025600718617e-05,
+      "loss": 3.042,
+      "step": 770
+    },
+    {
+      "epoch": 0.2496,
+      "grad_norm": 9.105142593383789,
+      "learning_rate": 9.650797215360432e-05,
+      "loss": 3.0201,
+      "step": 780
+    },
+    {
+      "epoch": 0.2528,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.639568830002246e-05,
+      "loss": 2.8601,
+      "step": 790
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.628340444644061e-05,
+      "loss": 3.0558,
+      "step": 800
+    },
+    {
+      "epoch": 0.2592,
+      "grad_norm": 7.617279052734375,
+      "learning_rate": 9.617112059285875e-05,
+      "loss": 2.9531,
+      "step": 810
+    },
+    {
+      "epoch": 0.2624,
+      "grad_norm": 7.272448539733887,
+      "learning_rate": 9.605883673927689e-05,
+      "loss": 2.9163,
+      "step": 820
+    },
+    {
+      "epoch": 0.2656,
+      "grad_norm": 10.0,
+      "learning_rate": 9.594655288569504e-05,
+      "loss": 3.0416,
+      "step": 830
+    },
+    {
+      "epoch": 0.2688,
+      "grad_norm": 6.786662578582764,
+      "learning_rate": 9.583426903211319e-05,
+      "loss": 3.0136,
+      "step": 840
+    },
+    {
+      "epoch": 0.272,
+      "grad_norm": 6.918086051940918,
+      "learning_rate": 9.572198517853134e-05,
+      "loss": 2.8705,
+      "step": 850
+    },
+    {
+      "epoch": 0.2752,
+      "grad_norm": 9.991483688354492,
+      "learning_rate": 9.560970132494948e-05,
+      "loss": 2.9016,
+      "step": 860
+    },
+    {
+      "epoch": 0.2784,
+      "grad_norm": 6.9515581130981445,
+      "learning_rate": 9.549741747136763e-05,
+      "loss": 2.7265,
+      "step": 870
+    },
+    {
+      "epoch": 0.2816,
+      "grad_norm": 8.731250762939453,
+      "learning_rate": 9.538513361778578e-05,
+      "loss": 2.8442,
+      "step": 880
+    },
+    {
+      "epoch": 0.2848,
+      "grad_norm": 7.565758228302002,
+      "learning_rate": 9.527284976420391e-05,
+      "loss": 2.7234,
+      "step": 890
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 7.732306003570557,
+      "learning_rate": 9.516056591062205e-05,
+      "loss": 3.0467,
+      "step": 900
+    },
+    {
+      "epoch": 0.2912,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.50482820570402e-05,
+      "loss": 2.9938,
+      "step": 910
+    },
+    {
+      "epoch": 0.2944,
+      "grad_norm": 7.414172649383545,
+      "learning_rate": 9.493599820345834e-05,
+      "loss": 2.7438,
+      "step": 920
+    },
+    {
+      "epoch": 0.2976,
+      "grad_norm": 8.1228609085083,
+      "learning_rate": 9.482371434987649e-05,
+      "loss": 2.8815,
+      "step": 930
+    },
+    {
+      "epoch": 0.3008,
+      "grad_norm": 6.716073513031006,
+      "learning_rate": 9.471143049629464e-05,
+      "loss": 2.6688,
+      "step": 940
+    },
+    {
+      "epoch": 0.304,
+      "grad_norm": 5.223897933959961,
+      "learning_rate": 9.459914664271278e-05,
+      "loss": 2.7251,
+      "step": 950
+    },
+    {
+      "epoch": 0.3072,
+      "grad_norm": 7.797822952270508,
+      "learning_rate": 9.448686278913093e-05,
+      "loss": 2.5746,
+      "step": 960
+    },
+    {
+      "epoch": 0.3104,
+      "grad_norm": 6.63670539855957,
+      "learning_rate": 9.437457893554908e-05,
+      "loss": 2.6461,
+      "step": 970
+    },
+    {
+      "epoch": 0.3136,
+      "grad_norm": 10.0,
+      "learning_rate": 9.426229508196722e-05,
+      "loss": 2.7993,
+      "step": 980
+    },
+    {
+      "epoch": 0.3168,
+      "grad_norm": 7.738500595092773,
+      "learning_rate": 9.415001122838537e-05,
+      "loss": 2.7401,
+      "step": 990
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 8.710482597351074,
+      "learning_rate": 9.40377273748035e-05,
+      "loss": 2.6068,
+      "step": 1000
+    },
+    {
+      "epoch": 0.3232,
+      "grad_norm": 6.2432684898376465,
+      "learning_rate": 9.392544352122165e-05,
+      "loss": 2.6246,
+      "step": 1010
+    },
+    {
+      "epoch": 0.3264,
+      "grad_norm": 7.530930519104004,
+      "learning_rate": 9.381315966763979e-05,
+      "loss": 2.7303,
+      "step": 1020
+    },
+    {
+      "epoch": 0.3296,
+      "grad_norm": 9.89444637298584,
+      "learning_rate": 9.370087581405794e-05,
+      "loss": 2.7558,
+      "step": 1030
+    },
+    {
+      "epoch": 0.3328,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.358859196047609e-05,
+      "loss": 2.8216,
+      "step": 1040
+    },
+    {
+      "epoch": 0.336,
+      "grad_norm": 5.699310302734375,
+      "learning_rate": 9.347630810689423e-05,
+      "loss": 2.81,
+      "step": 1050
+    },
+    {
+      "epoch": 0.3392,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.336402425331238e-05,
+      "loss": 2.797,
+      "step": 1060
+    },
+    {
+      "epoch": 0.3424,
+      "grad_norm": 5.669271469116211,
+      "learning_rate": 9.325174039973053e-05,
+      "loss": 2.6244,
+      "step": 1070
+    },
+    {
+      "epoch": 0.3456,
+      "grad_norm": 9.576983451843262,
+      "learning_rate": 9.313945654614867e-05,
+      "loss": 2.7266,
+      "step": 1080
+    },
+    {
+      "epoch": 0.3488,
+      "grad_norm": 5.313910007476807,
+      "learning_rate": 9.30271726925668e-05,
+      "loss": 2.8692,
+      "step": 1090
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 8.139154434204102,
+      "learning_rate": 9.291488883898496e-05,
+      "loss": 2.6439,
+      "step": 1100
+    },
+    {
+      "epoch": 0.3552,
+      "grad_norm": 6.6432600021362305,
+      "learning_rate": 9.280260498540311e-05,
+      "loss": 2.6079,
+      "step": 1110
+    },
+    {
+      "epoch": 0.3584,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.269032113182124e-05,
+      "loss": 2.7581,
+      "step": 1120
+    },
+    {
+      "epoch": 0.3616,
+      "grad_norm": 4.576703071594238,
+      "learning_rate": 9.25780372782394e-05,
+      "loss": 2.6304,
+      "step": 1130
+    },
+    {
+      "epoch": 0.3648,
+      "grad_norm": 5.5572285652160645,
+      "learning_rate": 9.246575342465755e-05,
+      "loss": 2.5335,
+      "step": 1140
+    },
+    {
+      "epoch": 0.368,
+      "grad_norm": 7.863306045532227,
+      "learning_rate": 9.235346957107568e-05,
+      "loss": 2.4998,
+      "step": 1150
+    },
+    {
+      "epoch": 0.3712,
+      "grad_norm": 8.850831031799316,
+      "learning_rate": 9.224118571749383e-05,
+      "loss": 2.6284,
+      "step": 1160
+    },
+    {
+      "epoch": 0.3744,
+      "grad_norm": 6.468632221221924,
+      "learning_rate": 9.212890186391197e-05,
+      "loss": 2.6882,
+      "step": 1170
+    },
+    {
+      "epoch": 0.3776,
+      "grad_norm": 5.677201271057129,
+      "learning_rate": 9.201661801033011e-05,
+      "loss": 2.4775,
+      "step": 1180
+    },
+    {
+      "epoch": 0.3808,
+      "grad_norm": 4.348922252655029,
+      "learning_rate": 9.190433415674826e-05,
+      "loss": 2.7046,
+      "step": 1190
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 7.337803363800049,
+      "learning_rate": 9.179205030316641e-05,
+      "loss": 2.5728,
+      "step": 1200
+    },
+    {
+      "epoch": 0.3872,
+      "grad_norm": 9.881601333618164,
+      "learning_rate": 9.167976644958456e-05,
+      "loss": 2.5674,
+      "step": 1210
+    },
+    {
+      "epoch": 0.3904,
+      "grad_norm": 7.198265552520752,
+      "learning_rate": 9.15674825960027e-05,
+      "loss": 2.545,
+      "step": 1220
+    },
+    {
+      "epoch": 0.3936,
+      "grad_norm": 4.9246907234191895,
+      "learning_rate": 9.145519874242085e-05,
+      "loss": 2.7013,
+      "step": 1230
+    },
+    {
+      "epoch": 0.3968,
+      "grad_norm": 5.395358085632324,
+      "learning_rate": 9.1342914888839e-05,
+      "loss": 2.5742,
+      "step": 1240
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 6.851541042327881,
+      "learning_rate": 9.123063103525713e-05,
+      "loss": 2.6272,
+      "step": 1250
+    },
+    {
+      "epoch": 0.4032,
+      "grad_norm": 6.056896209716797,
+      "learning_rate": 9.111834718167527e-05,
+      "loss": 2.5673,
+      "step": 1260
+    },
+    {
+      "epoch": 0.4064,
+      "grad_norm": 6.445649147033691,
+      "learning_rate": 9.100606332809342e-05,
+      "loss": 2.8856,
+      "step": 1270
+    },
+    {
+      "epoch": 0.4096,
+      "grad_norm": 7.691575527191162,
+      "learning_rate": 9.089377947451156e-05,
+      "loss": 2.6714,
+      "step": 1280
+    },
+    {
+      "epoch": 0.4128,
+      "grad_norm": 5.955436706542969,
+      "learning_rate": 9.078149562092971e-05,
+      "loss": 2.6481,
+      "step": 1290
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 9.62521743774414,
+      "learning_rate": 9.066921176734786e-05,
+      "loss": 2.5761,
+      "step": 1300
+    },
+    {
+      "epoch": 0.4192,
+      "grad_norm": 4.644749641418457,
+      "learning_rate": 9.055692791376601e-05,
+      "loss": 2.5133,
+      "step": 1310
+    },
+    {
+      "epoch": 0.4224,
+      "grad_norm": 4.176416397094727,
+      "learning_rate": 9.044464406018415e-05,
+      "loss": 2.4743,
+      "step": 1320
+    },
+    {
+      "epoch": 0.4256,
+      "grad_norm": 6.192687511444092,
+      "learning_rate": 9.03323602066023e-05,
+      "loss": 2.5556,
+      "step": 1330
+    },
+    {
+      "epoch": 0.4288,
+      "grad_norm": 4.459692001342773,
+      "learning_rate": 9.022007635302045e-05,
+      "loss": 2.4436,
+      "step": 1340
+    },
+    {
+      "epoch": 0.432,
+      "grad_norm": 6.155325412750244,
+      "learning_rate": 9.010779249943859e-05,
+      "loss": 2.4914,
+      "step": 1350
+    },
+    {
+      "epoch": 0.4352,
+      "grad_norm": 4.706230163574219,
+      "learning_rate": 8.999550864585672e-05,
+      "loss": 2.4705,
+      "step": 1360
+    },
+    {
+      "epoch": 0.4384,
+      "grad_norm": 6.316796779632568,
+      "learning_rate": 8.988322479227488e-05,
+      "loss": 2.4537,
+      "step": 1370
+    },
+    {
+      "epoch": 0.4416,
+      "grad_norm": 7.529996395111084,
+      "learning_rate": 8.977094093869301e-05,
+      "loss": 2.3581,
+      "step": 1380
+    },
+    {
+      "epoch": 0.4448,
+      "grad_norm": 7.215747356414795,
+      "learning_rate": 8.965865708511116e-05,
+      "loss": 2.4923,
+      "step": 1390
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 6.232388973236084,
+      "learning_rate": 8.954637323152931e-05,
+      "loss": 2.423,
+      "step": 1400
+    },
+    {
+      "epoch": 0.4512,
+      "grad_norm": 4.451657772064209,
+      "learning_rate": 8.943408937794746e-05,
+      "loss": 2.567,
+      "step": 1410
+    },
+    {
+      "epoch": 0.4544,
+      "grad_norm": 5.643525123596191,
+      "learning_rate": 8.93218055243656e-05,
+      "loss": 2.4042,
+      "step": 1420
+    },
+    {
+      "epoch": 0.4576,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.920952167078375e-05,
+      "loss": 2.4591,
+      "step": 1430
+    },
+    {
+      "epoch": 0.4608,
+      "grad_norm": 7.8325514793396,
+      "learning_rate": 8.909723781720189e-05,
+      "loss": 2.5524,
+      "step": 1440
+    },
+    {
+      "epoch": 0.464,
+      "grad_norm": 7.601844310760498,
+      "learning_rate": 8.898495396362003e-05,
+      "loss": 2.4915,
+      "step": 1450
+    },
+    {
+      "epoch": 0.4672,
+      "grad_norm": 10.0,
+      "learning_rate": 8.887267011003818e-05,
+      "loss": 2.5315,
+      "step": 1460
+    },
+    {
+      "epoch": 0.4704,
+      "grad_norm": 5.353204250335693,
+      "learning_rate": 8.876038625645633e-05,
+      "loss": 2.4169,
+      "step": 1470
+    },
+    {
+      "epoch": 0.4736,
+      "grad_norm": 9.473418235778809,
+      "learning_rate": 8.864810240287447e-05,
+      "loss": 2.4273,
+      "step": 1480
+    },
+    {
+      "epoch": 0.4768,
+      "grad_norm": 9.468120574951172,
+      "learning_rate": 8.853581854929262e-05,
+      "loss": 2.6286,
+      "step": 1490
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 5.575021743774414,
+      "learning_rate": 8.842353469571077e-05,
+      "loss": 2.6011,
+      "step": 1500
+    },
+    {
+      "epoch": 0.4832,
+      "grad_norm": 7.234686374664307,
+      "learning_rate": 8.83112508421289e-05,
+      "loss": 2.3283,
+      "step": 1510
+    },
+    {
+      "epoch": 0.4864,
+      "grad_norm": 4.9356184005737305,
+      "learning_rate": 8.819896698854705e-05,
+      "loss": 2.3356,
+      "step": 1520
+    },
+    {
+      "epoch": 0.4896,
+      "grad_norm": 5.09841775894165,
+      "learning_rate": 8.808668313496519e-05,
+      "loss": 2.3931,
+      "step": 1530
+    },
+    {
+      "epoch": 0.4928,
+      "grad_norm": 5.874563694000244,
+      "learning_rate": 8.797439928138334e-05,
+      "loss": 2.3976,
+      "step": 1540
+    },
+    {
+      "epoch": 0.496,
+      "grad_norm": 6.544667720794678,
+      "learning_rate": 8.786211542780148e-05,
+      "loss": 2.5013,
+      "step": 1550
+    },
+    {
+      "epoch": 0.4992,
+      "grad_norm": 7.496267795562744,
+      "learning_rate": 8.774983157421963e-05,
+      "loss": 2.4145,
+      "step": 1560
+    },
+    {
+      "epoch": 0.5024,
+      "grad_norm": 6.886484146118164,
+      "learning_rate": 8.763754772063778e-05,
+      "loss": 2.5783,
+      "step": 1570
+    },
+    {
+      "epoch": 0.5056,
+      "grad_norm": 5.7106547355651855,
+      "learning_rate": 8.752526386705592e-05,
+      "loss": 2.3237,
+      "step": 1580
+    },
+    {
+      "epoch": 0.5088,
+      "grad_norm": 4.9011125564575195,
+      "learning_rate": 8.741298001347407e-05,
+      "loss": 2.4852,
+      "step": 1590
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 4.599292755126953,
+      "learning_rate": 8.730069615989222e-05,
+      "loss": 2.4273,
+      "step": 1600
+    },
+    {
+      "epoch": 0.5152,
+      "grad_norm": 5.276413440704346,
+      "learning_rate": 8.718841230631036e-05,
+      "loss": 2.3425,
+      "step": 1610
+    },
+    {
+      "epoch": 0.5184,
+      "grad_norm": 9.41219425201416,
+      "learning_rate": 8.70761284527285e-05,
+      "loss": 2.4555,
+      "step": 1620
+    },
+    {
+      "epoch": 0.5216,
+      "grad_norm": 8.96435260772705,
+      "learning_rate": 8.696384459914664e-05,
+      "loss": 2.3372,
+      "step": 1630
+    },
+    {
+      "epoch": 0.5248,
+      "grad_norm": 6.9553985595703125,
+      "learning_rate": 8.68515607455648e-05,
+      "loss": 2.5085,
+      "step": 1640
+    },
+    {
+      "epoch": 0.528,
+      "grad_norm": 5.708377838134766,
+      "learning_rate": 8.673927689198293e-05,
+      "loss": 2.4528,
+      "step": 1650
+    },
+    {
+      "epoch": 0.5312,
+      "grad_norm": 8.115442276000977,
+      "learning_rate": 8.662699303840108e-05,
+      "loss": 2.3779,
+      "step": 1660
+    },
+    {
+      "epoch": 0.5344,
+      "grad_norm": 4.635824680328369,
+      "learning_rate": 8.651470918481923e-05,
+      "loss": 2.2828,
+      "step": 1670
+    },
+    {
+      "epoch": 0.5376,
+      "grad_norm": 3.9847660064697266,
+      "learning_rate": 8.640242533123737e-05,
+      "loss": 2.4229,
+      "step": 1680
+    },
+    {
+      "epoch": 0.5408,
+      "grad_norm": 7.811954975128174,
+      "learning_rate": 8.629014147765552e-05,
+      "loss": 2.2189,
+      "step": 1690
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 6.775632858276367,
+      "learning_rate": 8.617785762407367e-05,
+      "loss": 2.3237,
+      "step": 1700
+    },
+    {
+      "epoch": 0.5472,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.606557377049181e-05,
+      "loss": 2.4138,
+      "step": 1710
+    },
+    {
+      "epoch": 0.5504,
+      "grad_norm": 5.420886039733887,
+      "learning_rate": 8.595328991690995e-05,
+      "loss": 2.4619,
+      "step": 1720
+    },
+    {
+      "epoch": 0.5536,
+      "grad_norm": 5.865475177764893,
+      "learning_rate": 8.58410060633281e-05,
+      "loss": 2.4588,
+      "step": 1730
+    },
+    {
+      "epoch": 0.5568,
+      "grad_norm": 6.2092509269714355,
+      "learning_rate": 8.572872220974623e-05,
+      "loss": 2.4464,
+      "step": 1740
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 7.776535511016846,
+      "learning_rate": 8.561643835616438e-05,
+      "loss": 2.4011,
+      "step": 1750
+    },
+    {
+      "epoch": 0.5632,
+      "grad_norm": 6.631662368774414,
+      "learning_rate": 8.550415450258253e-05,
+      "loss": 2.355,
+      "step": 1760
+    },
+    {
+      "epoch": 0.5664,
+      "grad_norm": 6.007630825042725,
+      "learning_rate": 8.539187064900069e-05,
+      "loss": 2.5012,
+      "step": 1770
+    },
+    {
+      "epoch": 0.5696,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.527958679541882e-05,
+      "loss": 2.5772,
+      "step": 1780
+    },
+    {
+      "epoch": 0.5728,
+      "grad_norm": 5.767541408538818,
+      "learning_rate": 8.516730294183697e-05,
+      "loss": 2.5464,
+      "step": 1790
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 7.417142868041992,
+      "learning_rate": 8.505501908825511e-05,
+      "loss": 2.453,
+      "step": 1800
+    },
+    {
+      "epoch": 0.5792,
+      "grad_norm": 5.6994733810424805,
+      "learning_rate": 8.494273523467325e-05,
+      "loss": 2.5931,
+      "step": 1810
+    },
+    {
+      "epoch": 0.5824,
+      "grad_norm": 7.309426784515381,
+      "learning_rate": 8.48304513810914e-05,
+      "loss": 2.4631,
+      "step": 1820
+    },
+    {
+      "epoch": 0.5856,
+      "grad_norm": 5.572476863861084,
+      "learning_rate": 8.471816752750955e-05,
+      "loss": 2.5555,
+      "step": 1830
+    },
+    {
+      "epoch": 0.5888,
+      "grad_norm": 6.397329330444336,
+      "learning_rate": 8.460588367392769e-05,
+      "loss": 2.2597,
+      "step": 1840
+    },
+    {
+      "epoch": 0.592,
+      "grad_norm": 4.919052600860596,
+      "learning_rate": 8.449359982034584e-05,
+      "loss": 2.3108,
+      "step": 1850
+    },
+    {
+      "epoch": 0.5952,
+      "grad_norm": 8.084444999694824,
+      "learning_rate": 8.438131596676399e-05,
+      "loss": 2.3109,
+      "step": 1860
+    },
+    {
+      "epoch": 0.5984,
+      "grad_norm": 6.542473316192627,
+      "learning_rate": 8.426903211318214e-05,
+      "loss": 2.523,
+      "step": 1870
+    },
+    {
+      "epoch": 0.6016,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.415674825960028e-05,
+      "loss": 2.591,
+      "step": 1880
+    },
+    {
+      "epoch": 0.6048,
+      "grad_norm": 10.0,
+      "learning_rate": 8.404446440601843e-05,
+      "loss": 2.1992,
+      "step": 1890
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 8.659969329833984,
+      "learning_rate": 8.393218055243656e-05,
+      "loss": 2.2673,
+      "step": 1900
+    },
+    {
+      "epoch": 0.6112,
+      "grad_norm": 6.26581335067749,
+      "learning_rate": 8.38198966988547e-05,
+      "loss": 2.2571,
+      "step": 1910
+    },
+    {
+      "epoch": 0.6144,
+      "grad_norm": 4.025020122528076,
+      "learning_rate": 8.370761284527285e-05,
+      "loss": 2.4292,
+      "step": 1920
+    },
+    {
+      "epoch": 0.6176,
+      "grad_norm": 8.125853538513184,
+      "learning_rate": 8.3595328991691e-05,
+      "loss": 2.4198,
+      "step": 1930
+    },
+    {
+      "epoch": 0.6208,
+      "grad_norm": 9.907950401306152,
+      "learning_rate": 8.348304513810914e-05,
+      "loss": 2.3154,
+      "step": 1940
+    },
+    {
+      "epoch": 0.624,
+      "grad_norm": 4.589335918426514,
+      "learning_rate": 8.337076128452729e-05,
+      "loss": 2.2819,
+      "step": 1950
+    },
+    {
+      "epoch": 0.6272,
+      "grad_norm": 3.842802047729492,
+      "learning_rate": 8.325847743094544e-05,
+      "loss": 2.2328,
+      "step": 1960
+    },
+    {
+      "epoch": 0.6304,
+      "grad_norm": 9.061366081237793,
+      "learning_rate": 8.314619357736358e-05,
+      "loss": 2.3952,
+      "step": 1970
+    },
+    {
+      "epoch": 0.6336,
+      "grad_norm": 3.815833330154419,
+      "learning_rate": 8.303390972378173e-05,
+      "loss": 2.4474,
+      "step": 1980
+    },
+    {
+      "epoch": 0.6368,
+      "grad_norm": 8.364211082458496,
+      "learning_rate": 8.292162587019986e-05,
+      "loss": 2.4021,
+      "step": 1990
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 5.37713623046875,
+      "learning_rate": 8.280934201661802e-05,
+      "loss": 2.5195,
+      "step": 2000
+    },
+    {
+      "epoch": 0.6432,
+      "grad_norm": 8.178314208984375,
+      "learning_rate": 8.269705816303615e-05,
+      "loss": 2.3142,
+      "step": 2010
+    },
+    {
+      "epoch": 0.6464,
+      "grad_norm": 7.809556007385254,
+      "learning_rate": 8.25847743094543e-05,
+      "loss": 2.1985,
+      "step": 2020
+    },
+    {
+      "epoch": 0.6496,
+      "grad_norm": 9.732132911682129,
+      "learning_rate": 8.247249045587245e-05,
+      "loss": 2.2352,
+      "step": 2030
+    },
+    {
+      "epoch": 0.6528,
+      "grad_norm": 6.2440571784973145,
+      "learning_rate": 8.236020660229059e-05,
+      "loss": 2.1969,
+      "step": 2040
+    },
+    {
+      "epoch": 0.656,
+      "grad_norm": 5.272789478302002,
+      "learning_rate": 8.224792274870874e-05,
+      "loss": 2.3256,
+      "step": 2050
+    },
+    {
+      "epoch": 0.6592,
+      "grad_norm": 9.74461555480957,
+      "learning_rate": 8.213563889512689e-05,
+      "loss": 2.4389,
+      "step": 2060
+    },
+    {
+      "epoch": 0.6624,
+      "grad_norm": 4.720157146453857,
+      "learning_rate": 8.202335504154503e-05,
+      "loss": 2.4185,
+      "step": 2070
+    },
+    {
+      "epoch": 0.6656,
+      "grad_norm": 6.901200294494629,
+      "learning_rate": 8.191107118796317e-05,
+      "loss": 2.4403,
+      "step": 2080
+    },
+    {
+      "epoch": 0.6688,
+      "grad_norm": 6.6285624504089355,
+      "learning_rate": 8.179878733438132e-05,
+      "loss": 2.3017,
+      "step": 2090
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 5.480203628540039,
+      "learning_rate": 8.168650348079947e-05,
+      "loss": 2.4988,
+      "step": 2100
+    },
+    {
+      "epoch": 0.6752,
+      "grad_norm": 6.834201335906982,
+      "learning_rate": 8.15742196272176e-05,
+      "loss": 2.355,
+      "step": 2110
+    },
+    {
+      "epoch": 0.6784,
+      "grad_norm": 5.478670120239258,
+      "learning_rate": 8.146193577363576e-05,
+      "loss": 2.5405,
+      "step": 2120
+    },
+    {
+      "epoch": 0.6816,
+      "grad_norm": 8.395478248596191,
+      "learning_rate": 8.13496519200539e-05,
+      "loss": 2.2928,
+      "step": 2130
+    },
+    {
+      "epoch": 0.6848,
+      "grad_norm": 7.398046016693115,
+      "learning_rate": 8.123736806647204e-05,
+      "loss": 2.2259,
+      "step": 2140
+    },
+    {
+      "epoch": 0.688,
+      "grad_norm": 8.656116485595703,
+      "learning_rate": 8.11250842128902e-05,
+      "loss": 2.2555,
+      "step": 2150
+    },
+    {
+      "epoch": 0.6912,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.101280035930835e-05,
+      "loss": 2.4798,
+      "step": 2160
+    },
+    {
+      "epoch": 0.6944,
+      "grad_norm": 4.557366847991943,
+      "learning_rate": 8.090051650572648e-05,
+      "loss": 2.5535,
+      "step": 2170
+    },
+    {
+      "epoch": 0.6976,
+      "grad_norm": 4.364936351776123,
+      "learning_rate": 8.078823265214462e-05,
+      "loss": 2.1993,
+      "step": 2180
+    },
+    {
+      "epoch": 0.7008,
+      "grad_norm": 6.322412014007568,
+      "learning_rate": 8.067594879856277e-05,
+      "loss": 2.3593,
+      "step": 2190
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 6.9465789794921875,
+      "learning_rate": 8.056366494498092e-05,
+      "loss": 2.3772,
+      "step": 2200
+    },
+    {
+      "epoch": 0.7072,
+      "grad_norm": 6.708092212677002,
+      "learning_rate": 8.045138109139906e-05,
+      "loss": 2.332,
+      "step": 2210
+    },
+    {
+      "epoch": 0.7104,
+      "grad_norm": 5.4256768226623535,
+      "learning_rate": 8.033909723781721e-05,
+      "loss": 2.3298,
+      "step": 2220
+    },
+    {
+      "epoch": 0.7136,
+      "grad_norm": 4.8214850425720215,
+      "learning_rate": 8.022681338423536e-05,
+      "loss": 2.244,
+      "step": 2230
+    },
+    {
+      "epoch": 0.7168,
+      "grad_norm": 4.757294654846191,
+      "learning_rate": 8.01145295306535e-05,
+      "loss": 2.516,
+      "step": 2240
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 8.969782829284668,
+      "learning_rate": 8.000224567707165e-05,
+      "loss": 2.5106,
+      "step": 2250
+    },
+    {
+      "epoch": 0.7232,
+      "grad_norm": 5.573437690734863,
+      "learning_rate": 7.988996182348978e-05,
+      "loss": 2.5074,
+      "step": 2260
+    },
+    {
+      "epoch": 0.7264,
+      "grad_norm": 6.883465766906738,
+      "learning_rate": 7.977767796990792e-05,
+      "loss": 2.2634,
+      "step": 2270
+    },
+    {
+      "epoch": 0.7296,
+      "grad_norm": 6.035622596740723,
+      "learning_rate": 7.966539411632607e-05,
+      "loss": 2.3059,
+      "step": 2280
+    },
+    {
+      "epoch": 0.7328,
+      "grad_norm": 6.905811309814453,
+      "learning_rate": 7.955311026274422e-05,
+      "loss": 2.3145,
+      "step": 2290
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 6.611968517303467,
+      "learning_rate": 7.944082640916236e-05,
+      "loss": 2.3089,
+      "step": 2300
+    },
+    {
+      "epoch": 0.7392,
+      "grad_norm": 8.46692180633545,
+      "learning_rate": 7.932854255558051e-05,
+      "loss": 2.3344,
+      "step": 2310
+    },
+    {
+      "epoch": 0.7424,
+      "grad_norm": 7.151182174682617,
+      "learning_rate": 7.921625870199866e-05,
+      "loss": 2.2796,
+      "step": 2320
+    },
+    {
+      "epoch": 0.7456,
+      "grad_norm": 3.971759080886841,
+      "learning_rate": 7.910397484841681e-05,
+      "loss": 2.2586,
+      "step": 2330
+    },
+    {
+      "epoch": 0.7488,
+      "grad_norm": 6.936121940612793,
+      "learning_rate": 7.899169099483495e-05,
+      "loss": 2.2539,
+      "step": 2340
+    },
+    {
+      "epoch": 0.752,
+      "grad_norm": 4.994493007659912,
+      "learning_rate": 7.887940714125309e-05,
+      "loss": 2.1336,
+      "step": 2350
+    },
+    {
+      "epoch": 0.7552,
+      "grad_norm": 5.026124000549316,
+      "learning_rate": 7.876712328767124e-05,
+      "loss": 2.2772,
+      "step": 2360
+    },
+    {
+      "epoch": 0.7584,
+      "grad_norm": 8.548295974731445,
+      "learning_rate": 7.865483943408937e-05,
+      "loss": 2.2297,
+      "step": 2370
+    },
+    {
+      "epoch": 0.7616,
+      "grad_norm": 6.2121901512146,
+      "learning_rate": 7.854255558050752e-05,
+      "loss": 2.2928,
+      "step": 2380
+    },
+    {
+      "epoch": 0.7648,
+      "grad_norm": 6.6148810386657715,
+      "learning_rate": 7.843027172692568e-05,
+      "loss": 2.3334,
+      "step": 2390
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 4.6633758544921875,
+      "learning_rate": 7.831798787334381e-05,
+      "loss": 2.4213,
+      "step": 2400
+    },
+    {
+      "epoch": 0.7712,
+      "grad_norm": 6.86946439743042,
+      "learning_rate": 7.820570401976196e-05,
+      "loss": 2.2479,
+      "step": 2410
+    },
+    {
+      "epoch": 0.7744,
+      "grad_norm": 7.258929252624512,
+      "learning_rate": 7.809342016618011e-05,
+      "loss": 2.1996,
+      "step": 2420
+    },
+    {
+      "epoch": 0.7776,
+      "grad_norm": 6.279372215270996,
+      "learning_rate": 7.798113631259825e-05,
+      "loss": 2.2665,
+      "step": 2430
+    },
+    {
+      "epoch": 0.7808,
+      "grad_norm": 8.62110710144043,
+      "learning_rate": 7.78688524590164e-05,
+      "loss": 2.4074,
+      "step": 2440
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 7.775656860543454e-05,
+      "loss": 3.6339,
+      "step": 2450
+    },
+    {
+      "epoch": 0.7872,
+      "grad_norm": 4.590153217315674,
+      "learning_rate": 7.764428475185269e-05,
+      "loss": 2.3271,
+      "step": 2460
+    },
+    {
+      "epoch": 0.7904,
+      "grad_norm": 5.067539691925049,
+      "learning_rate": 7.753200089827083e-05,
+      "loss": 2.2559,
+      "step": 2470
+    },
+    {
+      "epoch": 0.7936,
+      "grad_norm": 4.031103610992432,
+      "learning_rate": 7.741971704468898e-05,
+      "loss": 2.3426,
+      "step": 2480
+    },
+    {
+      "epoch": 0.7968,
+      "grad_norm": 8.196564674377441,
+      "learning_rate": 7.730743319110713e-05,
+      "loss": 2.1718,
+      "step": 2490
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 4.67139196395874,
+      "learning_rate": 7.719514933752526e-05,
+      "loss": 2.5061,
+      "step": 2500
+    },
+    {
+      "epoch": 0.8032,
+      "grad_norm": 6.368116855621338,
+      "learning_rate": 7.708286548394342e-05,
+      "loss": 2.2068,
+      "step": 2510
+    },
+    {
+      "epoch": 0.8064,
+      "grad_norm": 8.849568367004395,
+      "learning_rate": 7.697058163036157e-05,
+      "loss": 2.3294,
+      "step": 2520
+    },
+    {
+      "epoch": 0.8096,
+      "grad_norm": 5.521136283874512,
+      "learning_rate": 7.68582977767797e-05,
+      "loss": 2.3293,
+      "step": 2530
+    },
+    {
+      "epoch": 0.8128,
+      "grad_norm": 7.696922302246094,
+      "learning_rate": 7.674601392319784e-05,
+      "loss": 2.3748,
+      "step": 2540
+    },
+    {
+      "epoch": 0.816,
+      "grad_norm": 7.695867538452148,
+      "learning_rate": 7.663373006961599e-05,
+      "loss": 2.0888,
+      "step": 2550
+    },
+    {
+      "epoch": 0.8192,
+      "grad_norm": 4.007026672363281,
+      "learning_rate": 7.652144621603414e-05,
+      "loss": 2.2417,
+      "step": 2560
+    },
+    {
+      "epoch": 0.8224,
+      "grad_norm": 5.373133659362793,
+      "learning_rate": 7.640916236245228e-05,
+      "loss": 2.397,
+      "step": 2570
+    },
+    {
+      "epoch": 0.8256,
+      "grad_norm": 7.045199871063232,
+      "learning_rate": 7.629687850887043e-05,
+      "loss": 2.3619,
+      "step": 2580
+    },
+    {
+      "epoch": 0.8288,
+      "grad_norm": 6.271018028259277,
+      "learning_rate": 7.618459465528858e-05,
+      "loss": 2.2644,
+      "step": 2590
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 8.043970108032227,
+      "learning_rate": 7.607231080170672e-05,
+      "loss": 2.5374,
+      "step": 2600
+    },
+    {
+      "epoch": 0.8352,
+      "grad_norm": 6.158629894256592,
+      "learning_rate": 7.596002694812487e-05,
+      "loss": 2.2779,
+      "step": 2610
+    },
+    {
+      "epoch": 0.8384,
+      "grad_norm": 3.6692025661468506,
+      "learning_rate": 7.5847743094543e-05,
+      "loss": 2.2478,
+      "step": 2620
+    },
+    {
+      "epoch": 0.8416,
+      "grad_norm": 6.483317852020264,
+      "learning_rate": 7.573545924096114e-05,
+      "loss": 2.152,
+      "step": 2630
+    },
+    {
+      "epoch": 0.8448,
+      "grad_norm": 5.592394828796387,
+      "learning_rate": 7.562317538737929e-05,
+      "loss": 2.4605,
+      "step": 2640
+    },
+    {
+      "epoch": 0.848,
+      "grad_norm": 5.773263931274414,
+      "learning_rate": 7.551089153379744e-05,
+      "loss": 2.4734,
+      "step": 2650
+    },
+    {
+      "epoch": 0.8512,
+      "grad_norm": 5.897200584411621,
+      "learning_rate": 7.53986076802156e-05,
+      "loss": 2.2845,
+      "step": 2660
+    },
+    {
+      "epoch": 0.8544,
+      "grad_norm": 6.476402759552002,
+      "learning_rate": 7.528632382663373e-05,
+      "loss": 2.1883,
+      "step": 2670
+    },
+    {
+      "epoch": 0.8576,
+      "grad_norm": 5.920806884765625,
+      "learning_rate": 7.517403997305188e-05,
+      "loss": 2.4448,
+      "step": 2680
+    },
+    {
+      "epoch": 0.8608,
+      "grad_norm": 4.258580207824707,
+      "learning_rate": 7.506175611947003e-05,
+      "loss": 2.1579,
+      "step": 2690
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 4.463003635406494,
+      "learning_rate": 7.494947226588817e-05,
+      "loss": 2.2095,
+      "step": 2700
+    },
+    {
+      "epoch": 0.8672,
+      "grad_norm": 8.6676607131958,
+      "learning_rate": 7.483718841230631e-05,
+      "loss": 2.3664,
+      "step": 2710
+    },
+    {
+      "epoch": 0.8704,
+      "grad_norm": 8.912688255310059,
+      "learning_rate": 7.472490455872446e-05,
+      "loss": 2.4029,
+      "step": 2720
+    },
+    {
+      "epoch": 0.8736,
+      "grad_norm": 6.20881986618042,
+      "learning_rate": 7.46126207051426e-05,
+      "loss": 2.2843,
+      "step": 2730
+    },
+    {
+      "epoch": 0.8768,
+      "grad_norm": 7.998341083526611,
+      "learning_rate": 7.450033685156075e-05,
+      "loss": 2.2651,
+      "step": 2740
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 9.811906814575195,
+      "learning_rate": 7.43880529979789e-05,
+      "loss": 2.3149,
+      "step": 2750
+    },
+    {
+      "epoch": 0.8832,
+      "grad_norm": 8.817770957946777,
+      "learning_rate": 7.427576914439703e-05,
+      "loss": 2.4565,
+      "step": 2760
+    },
+    {
+      "epoch": 0.8864,
+      "grad_norm": 4.727315902709961,
+      "learning_rate": 7.416348529081518e-05,
+      "loss": 2.2516,
+      "step": 2770
+    },
+    {
+      "epoch": 0.8896,
+      "grad_norm": 6.4811859130859375,
+      "learning_rate": 7.405120143723333e-05,
+      "loss": 2.0967,
+      "step": 2780
+    },
+    {
+      "epoch": 0.8928,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 7.393891758365149e-05,
+      "loss": 2.3187,
+      "step": 2790
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 9.609201431274414,
+      "learning_rate": 7.382663373006962e-05,
+      "loss": 2.2366,
+      "step": 2800
+    },
+    {
+      "epoch": 0.8992,
+      "grad_norm": 6.191689968109131,
+      "learning_rate": 7.371434987648776e-05,
+      "loss": 2.2946,
+      "step": 2810
+    },
+    {
+      "epoch": 0.9024,
+      "grad_norm": 9.036121368408203,
+      "learning_rate": 7.360206602290591e-05,
+      "loss": 2.2624,
+      "step": 2820
+    },
+    {
+      "epoch": 0.9056,
+      "grad_norm": 3.915534257888794,
+      "learning_rate": 7.348978216932405e-05,
+      "loss": 2.4489,
+      "step": 2830
+    },
+    {
+      "epoch": 0.9088,
+      "grad_norm": 7.408352851867676,
+      "learning_rate": 7.33774983157422e-05,
+      "loss": 2.1291,
+      "step": 2840
+    },
+    {
+      "epoch": 0.912,
+      "grad_norm": 8.331685066223145,
+      "learning_rate": 7.326521446216035e-05,
+      "loss": 2.322,
+      "step": 2850
+    },
+    {
+      "epoch": 0.9152,
+      "grad_norm": 5.713507175445557,
+      "learning_rate": 7.315293060857849e-05,
+      "loss": 2.4402,
+      "step": 2860
+    },
+    {
+      "epoch": 0.9184,
+      "grad_norm": 5.220091342926025,
+      "learning_rate": 7.304064675499664e-05,
+      "loss": 2.1427,
+      "step": 2870
+    },
+    {
+      "epoch": 0.9216,
+      "grad_norm": 5.63717794418335,
+      "learning_rate": 7.292836290141479e-05,
+      "loss": 2.2149,
+      "step": 2880
+    },
+    {
+      "epoch": 0.9248,
+      "grad_norm": 8.326608657836914,
+      "learning_rate": 7.281607904783292e-05,
+      "loss": 2.4356,
+      "step": 2890
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 6.667220592498779,
+      "learning_rate": 7.270379519425106e-05,
+      "loss": 2.1894,
+      "step": 2900
+    },
+    {
+      "epoch": 0.9312,
+      "grad_norm": 7.291802883148193,
+      "learning_rate": 7.259151134066921e-05,
+      "loss": 2.2346,
+      "step": 2910
+    },
+    {
+      "epoch": 0.9344,
+      "grad_norm": 5.355696678161621,
+      "learning_rate": 7.247922748708736e-05,
+      "loss": 2.3652,
+      "step": 2920
+    },
+    {
+      "epoch": 0.9376,
+      "grad_norm": 7.320453643798828,
+      "learning_rate": 7.23669436335055e-05,
+      "loss": 2.1491,
+      "step": 2930
+    },
+    {
+      "epoch": 0.9408,
+      "grad_norm": 6.482708930969238,
+      "learning_rate": 7.225465977992365e-05,
+      "loss": 2.1797,
+      "step": 2940
+    },
+    {
+      "epoch": 0.944,
+      "grad_norm": 8.68603515625,
+      "learning_rate": 7.21423759263418e-05,
+      "loss": 2.186,
+      "step": 2950
+    },
+    {
+      "epoch": 0.9472,
+      "grad_norm": 7.641050815582275,
+      "learning_rate": 7.203009207275994e-05,
+      "loss": 2.2185,
+      "step": 2960
+    },
+    {
+      "epoch": 0.9504,
+      "grad_norm": 7.222169876098633,
+      "learning_rate": 7.191780821917809e-05,
+      "loss": 2.3927,
+      "step": 2970
+    },
+    {
+      "epoch": 0.9536,
+      "grad_norm": 4.246700763702393,
+      "learning_rate": 7.180552436559623e-05,
+      "loss": 2.3365,
+      "step": 2980
+    },
+    {
+      "epoch": 0.9568,
+      "grad_norm": 6.096432685852051,
+      "learning_rate": 7.169324051201438e-05,
+      "loss": 2.1232,
+      "step": 2990
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 5.851985454559326,
+      "learning_rate": 7.158095665843251e-05,
+      "loss": 2.2176,
+      "step": 3000
+    },
+    {
+      "epoch": 0.9632,
+      "grad_norm": 7.797232627868652,
+      "learning_rate": 7.146867280485066e-05,
+      "loss": 2.4043,
+      "step": 3010
+    },
+    {
+      "epoch": 0.9664,
+      "grad_norm": 5.3702521324157715,
+      "learning_rate": 7.135638895126882e-05,
+      "loss": 2.2836,
+      "step": 3020
+    },
+    {
+      "epoch": 0.9696,
+      "grad_norm": 8.7493257522583,
+      "learning_rate": 7.124410509768695e-05,
+      "loss": 2.2188,
+      "step": 3030
+    },
+    {
+      "epoch": 0.9728,
+      "grad_norm": 10.0,
+      "learning_rate": 7.11318212441051e-05,
+      "loss": 2.3706,
+      "step": 3040
+    },
+    {
+      "epoch": 0.976,
+      "grad_norm": 8.834953308105469,
+      "learning_rate": 7.101953739052325e-05,
+      "loss": 2.442,
+      "step": 3050
+    },
+    {
+      "epoch": 0.9792,
+      "grad_norm": 7.613494873046875,
+      "learning_rate": 7.090725353694139e-05,
+      "loss": 2.3874,
+      "step": 3060
+    },
+    {
+      "epoch": 0.9824,
+      "grad_norm": 5.66098165512085,
+      "learning_rate": 7.079496968335954e-05,
+      "loss": 2.1254,
+      "step": 3070
+    },
+    {
+      "epoch": 0.9856,
+      "grad_norm": 8.746124267578125,
+      "learning_rate": 7.068268582977768e-05,
+      "loss": 2.2545,
+      "step": 3080
+    },
+    {
+      "epoch": 0.9888,
+      "grad_norm": 9.96347713470459,
+      "learning_rate": 7.057040197619582e-05,
+      "loss": 2.1841,
+      "step": 3090
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 8.428536415100098,
+      "learning_rate": 7.045811812261397e-05,
+      "loss": 2.3754,
+      "step": 3100
+    },
+    {
+      "epoch": 0.9952,
+      "grad_norm": 6.405836582183838,
+      "learning_rate": 7.034583426903212e-05,
+      "loss": 2.2303,
+      "step": 3110
+    },
+    {
+      "epoch": 0.9984,
+      "grad_norm": 7.094132423400879,
+      "learning_rate": 7.023355041545027e-05,
+      "loss": 2.1851,
+      "step": 3120
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 9375,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-3125/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de0dfe1c7042d6c0f1df2bd5079ed1e70733df6289a28a73c7d47650ee78f2bc
+size 5304

checkpoint-6250/README.md

@@ -0,0 +1,202 @@
+---
+base_model: gpt2-medium
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

checkpoint-6250/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GPT2LMHeadModel",
+    "parent_library": "transformers.models.gpt2.modeling_gpt2"
+  },
+  "base_model_name_or_path": "gpt2-medium",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-6250/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6034fd92a3c52a9ec20b5670571bb736284c8ea3eba04613684b80b85276b1f1
+size 25172088

checkpoint-6250/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:243a1376a9402f428f0327a2037909f9f5dda56dfde0c1271b61e5f71ecfbb46
+size 50372538

checkpoint-6250/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c120d48de72cdb78154eb3ff34dcb662bf117dd1d28d6bd0d82dd6f4212161a
+size 14244

checkpoint-6250/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54bf5bc35983da2516a319daa8db116d3e18d36fcecad8a541a6fed01b6c2f20
+size 1064

checkpoint-6250/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-6250/tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-6250/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de0dfe1c7042d6c0f1df2bd5079ed1e70733df6289a28a73c7d47650ee78f2bc
+size 5304

checkpoint-9375/README.md

@@ -0,0 +1,202 @@
+---
+base_model: gpt2-medium
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

checkpoint-9375/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GPT2LMHeadModel",
+    "parent_library": "transformers.models.gpt2.modeling_gpt2"
+  },
+  "base_model_name_or_path": "gpt2-medium",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-9375/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e080b6018d76cf35e380d38c6d986e6242e3d34fb8b0e94622e054c202fb2587
+size 25172088

checkpoint-9375/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5f0e8d012cfc8a35e8edf37ebbd4584056ea26ca8c1da275db85833f14af208
+size 50372538

checkpoint-9375/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45a17cbacf815f7540ba86fc4db7056edcdc3201a531803e9244208a01c54025
+size 14244

checkpoint-9375/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a17c6f8f90b7492d85e3ccb69e5e19d7c97f2aa89c2af5ae7acd1722212a18fd
+size 1064

checkpoint-9375/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-9375/tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-9375/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de0dfe1c7042d6c0f1df2bd5079ed1e70733df6289a28a73c7d47650ee78f2bc
+size 5304

config.json

@@ -0,0 +1,40 @@
+{
+  "_name_or_path": "gpt2-medium",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 1024,
+  "n_head": 16,
+  "n_inner": null,
+  "n_layer": 24,
+  "n_positions": 1024,
+  "n_special": 0,
+  "predict_special_tokens": true,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "transformers_version": "4.45.2",
+  "use_cache": true,
+  "vocab_size": 50257
+}

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_logs.csv

@@ -0,0 +1,939 @@
+step,training_loss,grad_norm
+10,28.3701,10.0
+20,24.5207,10.0
+30,22.6885,10.0
+40,25.2666,10.0
+50,23.0791,10.0
+60,23.3554,9.999999046325684
+70,23.0232,10.000000953674316
+80,21.8077,10.0
+90,20.0271,9.999999046325684
+100,17.0629,9.999999046325684
+110,18.8398,9.999999046325684
+120,15.8197,9.999998092651367
+130,14.2088,9.999998092651367
+140,14.0394,10.0
+150,14.3635,10.0
+160,11.0378,10.0
+170,10.8982,9.999999046325684
+180,9.9566,9.999998092651367
+190,10.2806,9.999999046325684
+200,9.8932,9.999999046325684
+210,8.1511,10.0
+220,8.07,9.999998092651367
+230,8.0589,9.999999046325684
+240,6.8857,9.999998092651367
+250,6.7318,9.999999046325684
+260,7.0872,10.0
+270,6.5861,9.999999046325684
+280,6.3828,10.0
+290,6.0198,10.0
+300,6.2239,9.999999046325684
+310,5.7589,9.999999046325684
+320,5.4196,9.999998092651367
+330,5.6412,9.999999046325684
+340,5.2791,10.0
+350,5.5189,10.0
+360,5.0969,10.0
+370,5.2089,9.999999046325684
+380,6.2735,9.999999046325684
+390,5.1972,9.999999046325684
+400,4.791,9.999998092651367
+410,5.139,10.0
+420,5.1234,9.999998092651367
+430,4.763,9.726828575134277
+440,5.0725,10.0
+450,4.708,10.0
+460,4.3924,9.999999046325684
+470,4.3606,8.861834526062012
+480,4.2736,9.999999046325684
+490,4.5541,9.999999046325684
+500,4.3459,8.72944450378418
+510,4.0927,9.999999046325684
+520,4.1877,9.396402359008789
+530,3.9887,6.486421585083008
+540,4.1643,9.999999046325684
+550,4.0236,8.012758255004883
+560,3.9494,9.999999046325684
+570,3.7742,9.999998092651367
+580,3.817,9.999999046325684
+590,3.8002,10.0
+600,3.5841,9.999998092651367
+610,3.7045,9.999998092651367
+620,3.3623,9.999999046325684
+630,3.3953,9.999998092651367
+640,3.5513,7.71815824508667
+650,3.3845,7.819033145904541
+660,3.2571,9.44756031036377
+670,3.1949,5.626587390899658
+680,3.05,9.968693733215332
+690,3.378,9.028199195861816
+700,3.2383,9.999999046325684
+710,3.2572,10.0
+720,3.2422,9.906242370605469
+730,3.199,7.099668025970459
+740,2.9136,6.1059136390686035
+750,3.0956,8.184073448181152
+760,2.9784,7.790270805358887
+770,3.042,9.385482788085938
+780,3.0201,9.105142593383789
+790,2.8601,9.999999046325684
+800,3.0558,9.999999046325684
+810,2.9531,7.617279052734375
+820,2.9163,7.272448539733887
+830,3.0416,10.0
+840,3.0136,6.786662578582764
+850,2.8705,6.918086051940918
+860,2.9016,9.991483688354492
+870,2.7265,6.9515581130981445
+880,2.8442,8.731250762939453
+890,2.7234,7.565758228302002
+900,3.0467,7.732306003570557
+910,2.9938,9.999999046325684
+920,2.7438,7.414172649383545
+930,2.8815,8.1228609085083
+940,2.6688,6.716073513031006
+950,2.7251,5.223897933959961
+960,2.5746,7.797822952270508
+970,2.6461,6.63670539855957
+980,2.7993,10.0
+990,2.7401,7.738500595092773
+1000,2.6068,8.710482597351074
+1010,2.6246,6.2432684898376465
+1020,2.7303,7.530930519104004
+1030,2.7558,9.89444637298584
+1040,2.8216,9.999998092651367
+1050,2.81,5.699310302734375
+1060,2.797,9.999999046325684
+1070,2.6244,5.669271469116211
+1080,2.7266,9.576983451843262
+1090,2.8692,5.313910007476807
+1100,2.6439,8.139154434204102
+1110,2.6079,6.6432600021362305
+1120,2.7581,9.999999046325684
+1130,2.6304,4.576703071594238
+1140,2.5335,5.5572285652160645
+1150,2.4998,7.863306045532227
+1160,2.6284,8.850831031799316
+1170,2.6882,6.468632221221924
+1180,2.4775,5.677201271057129
+1190,2.7046,4.348922252655029
+1200,2.5728,7.337803363800049
+1210,2.5674,9.881601333618164
+1220,2.545,7.198265552520752
+1230,2.7013,4.9246907234191895
+1240,2.5742,5.395358085632324
+1250,2.6272,6.851541042327881
+1260,2.5673,6.056896209716797
+1270,2.8856,6.445649147033691
+1280,2.6714,7.691575527191162
+1290,2.6481,5.955436706542969
+1300,2.5761,9.62521743774414
+1310,2.5133,4.644749641418457
+1320,2.4743,4.176416397094727
+1330,2.5556,6.192687511444092
+1340,2.4436,4.459692001342773
+1350,2.4914,6.155325412750244
+1360,2.4705,4.706230163574219
+1370,2.4537,6.316796779632568
+1380,2.3581,7.529996395111084
+1390,2.4923,7.215747356414795
+1400,2.423,6.232388973236084
+1410,2.567,4.451657772064209
+1420,2.4042,5.643525123596191
+1430,2.4591,9.999999046325684
+1440,2.5524,7.8325514793396
+1450,2.4915,7.601844310760498
+1460,2.5315,10.0
+1470,2.4169,5.353204250335693
+1480,2.4273,9.473418235778809
+1490,2.6286,9.468120574951172
+1500,2.6011,5.575021743774414
+1510,2.3283,7.234686374664307
+1520,2.3356,4.9356184005737305
+1530,2.3931,5.09841775894165
+1540,2.3976,5.874563694000244
+1550,2.5013,6.544667720794678
+1560,2.4145,7.496267795562744
+1570,2.5783,6.886484146118164
+1580,2.3237,5.7106547355651855
+1590,2.4852,4.9011125564575195
+1600,2.4273,4.599292755126953
+1610,2.3425,5.276413440704346
+1620,2.4555,9.41219425201416
+1630,2.3372,8.96435260772705
+1640,2.5085,6.9553985595703125
+1650,2.4528,5.708377838134766
+1660,2.3779,8.115442276000977
+1670,2.2828,4.635824680328369
+1680,2.4229,3.9847660064697266
+1690,2.2189,7.811954975128174
+1700,2.3237,6.775632858276367
+1710,2.4138,9.999999046325684
+1720,2.4619,5.420886039733887
+1730,2.4588,5.865475177764893
+1740,2.4464,6.2092509269714355
+1750,2.4011,7.776535511016846
+1760,2.355,6.631662368774414
+1770,2.5012,6.007630825042725
+1780,2.5772,9.999998092651367
+1790,2.5464,5.767541408538818
+1800,2.453,7.417142868041992
+1810,2.5931,5.6994733810424805
+1820,2.4631,7.309426784515381
+1830,2.5555,5.572476863861084
+1840,2.2597,6.397329330444336
+1850,2.3108,4.919052600860596
+1860,2.3109,8.084444999694824
+1870,2.523,6.542473316192627
+1880,2.591,9.999998092651367
+1890,2.1992,10.0
+1900,2.2673,8.659969329833984
+1910,2.2571,6.26581335067749
+1920,2.4292,4.025020122528076
+1930,2.4198,8.125853538513184
+1940,2.3154,9.907950401306152
+1950,2.2819,4.589335918426514
+1960,2.2328,3.842802047729492
+1970,2.3952,9.061366081237793
+1980,2.4474,3.815833330154419
+1990,2.4021,8.364211082458496
+2000,2.5195,5.37713623046875
+2010,2.3142,8.178314208984375
+2020,2.1985,7.809556007385254
+2030,2.2352,9.732132911682129
+2040,2.1969,6.2440571784973145
+2050,2.3256,5.272789478302002
+2060,2.4389,9.74461555480957
+2070,2.4185,4.720157146453857
+2080,2.4403,6.901200294494629
+2090,2.3017,6.6285624504089355
+2100,2.4988,5.480203628540039
+2110,2.355,6.834201335906982
+2120,2.5405,5.478670120239258
+2130,2.2928,8.395478248596191
+2140,2.2259,7.398046016693115
+2150,2.2555,8.656116485595703
+2160,2.4798,9.999999046325684
+2170,2.5535,4.557366847991943
+2180,2.1993,4.364936351776123
+2190,2.3593,6.322412014007568
+2200,2.3772,6.9465789794921875
+2210,2.332,6.708092212677002
+2220,2.3298,5.4256768226623535
+2230,2.244,4.8214850425720215
+2240,2.516,4.757294654846191
+2250,2.5106,8.969782829284668
+2260,2.5074,5.573437690734863
+2270,2.2634,6.883465766906738
+2280,2.3059,6.035622596740723
+2290,2.3145,6.905811309814453
+2300,2.3089,6.611968517303467
+2310,2.3344,8.46692180633545
+2320,2.2796,7.151182174682617
+2330,2.2586,3.971759080886841
+2340,2.2539,6.936121940612793
+2350,2.1336,4.994493007659912
+2360,2.2772,5.026124000549316
+2370,2.2297,8.548295974731445
+2380,2.2928,6.2121901512146
+2390,2.3334,6.6148810386657715
+2400,2.4213,4.6633758544921875
+2410,2.2479,6.86946439743042
+2420,2.1996,7.258929252624512
+2430,2.2665,6.279372215270996
+2440,2.4074,8.62110710144043
+2450,3.6339,10.000000953674316
+2460,2.3271,4.590153217315674
+2470,2.2559,5.067539691925049
+2480,2.3426,4.031103610992432
+2490,2.1718,8.196564674377441
+2500,2.5061,4.67139196395874
+2510,2.2068,6.368116855621338
+2520,2.3294,8.849568367004395
+2530,2.3293,5.521136283874512
+2540,2.3748,7.696922302246094
+2550,2.0888,7.695867538452148
+2560,2.2417,4.007026672363281
+2570,2.397,5.373133659362793
+2580,2.3619,7.045199871063232
+2590,2.2644,6.271018028259277
+2600,2.5374,8.043970108032227
+2610,2.2779,6.158629894256592
+2620,2.2478,3.6692025661468506
+2630,2.152,6.483317852020264
+2640,2.4605,5.592394828796387
+2650,2.4734,5.773263931274414
+2660,2.2845,5.897200584411621
+2670,2.1883,6.476402759552002
+2680,2.4448,5.920806884765625
+2690,2.1579,4.258580207824707
+2700,2.2095,4.463003635406494
+2710,2.3664,8.6676607131958
+2720,2.4029,8.912688255310059
+2730,2.2843,6.20881986618042
+2740,2.2651,7.998341083526611
+2750,2.3149,9.811906814575195
+2760,2.4565,8.817770957946777
+2770,2.2516,4.727315902709961
+2780,2.0967,6.4811859130859375
+2790,2.3187,9.999999046325684
+2800,2.2366,9.609201431274414
+2810,2.2946,6.191689968109131
+2820,2.2624,9.036121368408203
+2830,2.4489,3.915534257888794
+2840,2.1291,7.408352851867676
+2850,2.322,8.331685066223145
+2860,2.4402,5.713507175445557
+2870,2.1427,5.220091342926025
+2880,2.2149,5.63717794418335
+2890,2.4356,8.326608657836914
+2900,2.1894,6.667220592498779
+2910,2.2346,7.291802883148193
+2920,2.3652,5.355696678161621
+2930,2.1491,7.320453643798828
+2940,2.1797,6.482708930969238
+2950,2.186,8.68603515625
+2960,2.2185,7.641050815582275
+2970,2.3927,7.222169876098633
+2980,2.3365,4.246700763702393
+2990,2.1232,6.096432685852051
+3000,2.2176,5.851985454559326
+3010,2.4043,7.797232627868652
+3020,2.2836,5.3702521324157715
+3030,2.2188,8.7493257522583
+3040,2.3706,10.0
+3050,2.442,8.834953308105469
+3060,2.3874,7.613494873046875
+3070,2.1254,5.66098165512085
+3080,2.2545,8.746124267578125
+3090,2.1841,9.96347713470459
+3100,2.3754,8.428536415100098
+3110,2.2303,6.405836582183838
+3120,2.1851,7.094132423400879
+3130,2.2898,5.770329475402832
+3140,2.3988,4.412771701812744
+3150,2.35,9.633313179016113
+3160,2.1658,7.720678806304932
+3170,2.1582,7.196460723876953
+3180,2.3164,3.8286609649658203
+3190,2.2672,7.099523067474365
+3200,2.2812,6.283836364746094
+3210,2.1724,4.959673881530762
+3220,2.159,9.999998092651367
+3230,2.3596,4.327045440673828
+3240,2.3693,7.3624653816223145
+3250,2.3983,8.922405242919922
+3260,2.1969,6.19892692565918
+3270,2.2934,6.3894524574279785
+3280,2.2858,7.240560054779053
+3290,2.2498,6.6486992835998535
+3300,2.2005,7.461024761199951
+3310,2.2171,9.999999046325684
+3320,2.0921,6.346460342407227
+3330,2.2411,6.42893648147583
+3340,2.0989,5.788027286529541
+3350,2.3168,9.999999046325684
+3360,2.2532,5.730334758758545
+3370,2.1161,7.052060127258301
+3380,2.2371,4.694394588470459
+3390,2.5154,6.626163482666016
+3400,2.1627,7.122054576873779
+3410,2.072,6.109677791595459
+3420,2.354,7.214818000793457
+3430,2.3704,9.6759672164917
+3440,2.3059,4.210315227508545
+3450,2.3811,6.809922218322754
+3460,2.4183,5.466648578643799
+3470,2.2508,8.649759292602539
+3480,2.1897,4.219826698303223
+3490,2.1246,6.436397552490234
+3500,2.5233,9.999999046325684
+3510,2.3533,10.0
+3520,2.2338,3.651031732559204
+3530,2.2797,5.527792453765869
+3540,2.1877,4.227158546447754
+3550,2.1618,6.878480434417725
+3560,2.1245,5.831515789031982
+3570,2.1778,3.441094160079956
+3580,1.9831,7.330443382263184
+3590,2.1796,8.98618221282959
+3600,2.3529,4.757387638092041
+3610,2.3361,3.5186734199523926
+3620,2.1439,6.991964340209961
+3630,2.1227,6.133205413818359
+3640,2.2649,4.490849018096924
+3650,2.2646,6.143466949462891
+3660,2.2631,4.09475040435791
+3670,2.2013,9.874448776245117
+3680,2.2698,7.732521057128906
+3690,2.1787,5.697376251220703
+3700,2.1337,6.074872970581055
+3710,2.1082,7.495859622955322
+3720,2.2119,5.88323450088501
+3730,2.1251,4.20490837097168
+3740,2.3263,7.4095540046691895
+3750,2.0824,5.438231468200684
+3760,2.37,7.119965553283691
+3770,2.3092,4.426470756530762
+3780,2.1568,5.602468490600586
+3790,2.0438,4.94999361038208
+3800,2.2386,6.560863494873047
+3810,3.1694,9.445927619934082
+3820,2.2535,6.8221564292907715
+3830,2.1744,6.22130012512207
+3840,2.5386,5.471611499786377
+3850,2.1352,3.5220141410827637
+3860,2.2617,6.965565204620361
+3870,2.1913,5.471559047698975
+3880,2.2588,7.852587699890137
+3890,2.1421,3.5346343517303467
+3900,2.3086,8.021695137023926
+3910,2.1955,5.215954780578613
+3920,2.0465,6.547055244445801
+3930,2.2372,8.973469734191895
+3940,2.2969,8.85340404510498
+3950,2.1088,8.8905029296875
+3960,2.2229,3.574281930923462
+3970,2.0212,6.460089683532715
+3980,2.2653,6.969429969787598
+3990,2.2088,6.06675386428833
+4000,2.1276,9.980952262878418
+4010,2.2661,5.178830146789551
+4020,2.1305,9.019587516784668
+4030,2.1621,6.0943074226379395
+4040,2.0759,3.6441543102264404
+4050,2.1728,4.56536865234375
+4060,2.1825,4.460301876068115
+4070,2.2135,4.983593940734863
+4080,2.3566,6.907173156738281
+4090,2.2541,5.130218982696533
+4100,1.9409,4.747898578643799
+4110,2.2585,4.745423793792725
+4120,2.0822,6.524007320404053
+4130,2.0697,8.787076950073242
+4140,2.1111,3.5528957843780518
+4150,2.1446,5.02815580368042
+4160,2.0923,3.4902503490448
+4170,2.291,4.021839141845703
+4180,2.1877,5.555225372314453
+4190,2.0687,6.962709426879883
+4200,2.3351,10.0
+4210,2.2501,4.848272800445557
+4220,2.1061,5.333714962005615
+4230,2.1438,6.211556434631348
+4240,2.1886,8.62783145904541
+4250,2.2546,7.506083965301514
+4260,2.3712,5.266780376434326
+4270,2.1088,9.855100631713867
+4280,2.347,3.677947759628296
+4290,2.2568,7.6731276512146
+4300,2.0683,5.236011505126953
+4310,2.1171,6.686437606811523
+4320,2.2386,10.0
+4330,2.1134,4.5010986328125
+4340,2.4407,3.935786724090576
+4350,2.2234,6.398930072784424
+4360,2.1862,8.995603561401367
+4370,2.1446,8.50732421875
+4380,2.1897,8.399739265441895
+4390,2.1621,9.738068580627441
+4400,2.2949,7.87959098815918
+4410,2.0071,6.368355751037598
+4420,2.2448,5.002744197845459
+4430,2.3409,4.510824203491211
+4440,2.1176,6.529369831085205
+4450,2.0862,9.94469165802002
+4460,2.4251,5.885842323303223
+4470,1.9864,6.890707492828369
+4480,2.1435,4.47102689743042
+4490,2.0326,4.225493431091309
+4500,2.1564,4.37125825881958
+4510,2.0743,8.290303230285645
+4520,2.2523,5.776940822601318
+4530,2.1078,4.822579383850098
+4540,2.0797,5.790931224822998
+4550,2.1237,5.75389289855957
+4560,2.2125,7.504802227020264
+4570,2.1611,4.340479850769043
+4580,2.2023,8.178547859191895
+4590,2.1268,9.241583824157715
+4600,2.1976,4.36149263381958
+4610,2.1717,9.999999046325684
+4620,1.9805,5.395240306854248
+4630,2.2378,5.555797576904297
+4640,2.1956,5.219169616699219
+4650,2.0657,9.0369234085083
+4660,1.9879,7.064931392669678
+4670,2.2089,6.1723809242248535
+4680,2.1378,9.340553283691406
+4690,2.0892,7.710911750793457
+4700,2.421,8.593657493591309
+4710,2.0054,3.763519763946533
+4720,2.2029,5.541579723358154
+4730,2.0555,4.48792839050293
+4740,2.1252,4.694746494293213
+4750,2.2292,3.626134157180786
+4760,2.138,7.184067249298096
+4770,2.2403,5.250986576080322
+4780,2.1862,6.30639123916626
+4790,2.1111,5.524565696716309
+4800,2.1711,4.2406697273254395
+4810,2.057,4.823990821838379
+4820,2.0207,6.957513809204102
+4830,2.222,5.449457168579102
+4840,2.2214,4.320427894592285
+4850,2.2,4.92500638961792
+4860,2.0129,4.991371154785156
+4870,2.2163,6.521137237548828
+4880,2.0385,7.385705947875977
+4890,2.1222,8.725979804992676
+4900,2.0741,4.52073335647583
+4910,2.1543,6.499835014343262
+4920,2.2229,7.410144329071045
+4930,2.1517,4.372899055480957
+4940,2.1163,3.7120919227600098
+4950,2.1247,7.357861518859863
+4960,2.1781,6.083400726318359
+4970,1.9974,6.298897743225098
+4980,2.1545,6.194458484649658
+4990,2.1107,5.1978278160095215
+5000,2.3348,4.76450777053833
+5010,2.0502,4.51431131362915
+5020,2.1568,6.624210834503174
+5030,2.0227,5.511273384094238
+5040,2.1213,3.3496336936950684
+5050,2.1931,7.776872158050537
+5060,2.1976,6.206504821777344
+5070,2.2875,9.433921813964844
+5080,2.2109,5.77447509765625
+5090,2.1561,3.713836193084717
+5100,2.1125,3.5686769485473633
+5110,2.2041,7.770430564880371
+5120,2.1188,5.841362953186035
+5130,2.0692,3.36885142326355
+5140,2.0279,7.515949249267578
+5150,2.0905,4.221615314483643
+5160,2.1089,6.048422336578369
+5170,2.2271,5.11595344543457
+5180,2.1604,5.2255449295043945
+5190,2.0856,5.745364189147949
+5200,2.1981,4.490666389465332
+5210,2.0797,5.564126968383789
+5220,2.163,5.2047247886657715
+5230,2.2305,8.44140625
+5240,2.1215,8.154293060302734
+5250,2.1661,5.150942802429199
+5260,2.2111,6.649692058563232
+5270,2.0773,8.254748344421387
+5280,2.101,2.958098888397217
+5290,2.4461,7.841224670410156
+5300,2.1184,3.867351531982422
+5310,2.2102,9.999999046325684
+5320,2.1324,5.966880798339844
+5330,2.2293,5.331724643707275
+5340,2.044,5.222733497619629
+5350,2.1925,3.4903883934020996
+5360,2.1628,9.891901969909668
+5370,2.2044,3.6791141033172607
+5380,2.0421,3.284372091293335
+5390,2.0564,2.7818946838378906
+5400,2.1152,9.54232406616211
+5410,2.1527,6.284071445465088
+5420,2.0515,4.312674045562744
+5430,2.2114,4.142785549163818
+5440,2.2083,7.880631446838379
+5450,2.2029,4.2559709548950195
+5460,2.2198,6.414773941040039
+5470,1.9202,4.784211158752441
+5480,2.2925,8.340254783630371
+5490,2.1257,3.7721922397613525
+5500,2.1088,5.6239776611328125
+5510,2.1422,6.1071295738220215
+5520,2.0904,8.268501281738281
+5530,2.2511,8.720447540283203
+5540,2.1065,9.763360023498535
+5550,2.0478,9.336665153503418
+5560,2.07,9.999999046325684
+5570,2.0205,5.811784744262695
+5580,2.1649,8.127388000488281
+5590,2.1256,5.085277080535889
+5600,2.1498,5.166441440582275
+5610,2.1661,6.641541481018066
+5620,2.2342,3.5791046619415283
+5630,2.136,4.0631279945373535
+5640,2.4784,4.964014530181885
+5650,2.3117,7.552999019622803
+5660,2.1387,4.7818284034729
+5670,2.1771,7.117303848266602
+5680,1.9511,6.260792255401611
+5690,2.3494,6.069239139556885
+5700,2.037,3.6275076866149902
+5710,2.1981,5.257174491882324
+5720,2.1808,9.999999046325684
+5730,2.1297,6.373122215270996
+5740,2.1768,6.5584940910339355
+5750,2.0283,8.723784446716309
+5760,2.2787,5.2506537437438965
+5770,2.0738,6.952129364013672
+5780,1.9776,7.82886266708374
+5790,2.137,8.772919654846191
+5800,2.3822,4.685011386871338
+5810,2.2003,6.074718952178955
+5820,1.978,6.719532489776611
+5830,2.0413,6.288917064666748
+5840,2.0542,3.5031559467315674
+5850,2.1693,7.236006259918213
+5860,2.2759,5.247088432312012
+5870,2.1932,4.443241596221924
+5880,2.2292,3.587862968444824
+5890,1.9304,2.6182422637939453
+5900,2.1348,3.457829713821411
+5910,2.0977,4.102409839630127
+5920,2.171,6.1073994636535645
+5930,2.1105,4.545835494995117
+5940,2.0686,6.679617881774902
+5950,2.1866,8.60380744934082
+5960,2.1823,5.819077014923096
+5970,2.1523,3.7871108055114746
+5980,2.2493,5.970479965209961
+5990,4.3472,10.0
+6000,2.2363,8.660836219787598
+6010,2.1017,7.8099894523620605
+6020,2.1996,5.9949445724487305
+6030,2.1646,6.623493671417236
+6040,2.1039,7.280769348144531
+6050,2.1895,5.2272257804870605
+6060,2.1616,6.899860858917236
+6070,2.1634,7.497065544128418
+6080,2.0569,4.0999956130981445
+6090,2.1319,3.794092893600464
+6100,2.1621,5.509084224700928
+6110,2.2731,6.076055526733398
+6120,1.9689,3.3105826377868652
+6130,2.1141,5.338077068328857
+6140,1.9314,5.527437210083008
+6150,2.0693,3.655165195465088
+6160,2.0442,9.39285659790039
+6170,2.3283,8.24338150024414
+6180,2.1328,4.415241718292236
+6190,2.0253,5.860321044921875
+6200,2.0622,5.472553253173828
+6210,1.9528,4.431563854217529
+6220,2.0531,7.67194938659668
+6230,2.0059,3.6711692810058594
+6240,2.032,4.1530890464782715
+6250,2.0153,4.8354926109313965
+6260,2.0774,4.306488037109375
+6270,2.0359,4.352275371551514
+6280,2.029,7.978499889373779
+6290,2.0506,6.480661869049072
+6300,2.0744,3.58092999458313
+6310,2.1674,5.924591064453125
+6320,2.1415,6.986868381500244
+6330,2.119,4.46516752243042
+6340,2.0115,5.372344493865967
+6350,2.0578,5.572484970092773
+6360,2.2153,4.962610244750977
+6370,2.0666,4.404867649078369
+6380,2.2205,4.287213325500488
+6390,2.0955,7.050772666931152
+6400,2.1016,6.098924160003662
+6410,2.2017,6.557415962219238
+6420,2.0959,6.623813152313232
+6430,2.16,6.275115966796875
+6440,2.063,5.501457214355469
+6450,1.9966,3.797224760055542
+6460,2.1951,9.272015571594238
+6470,1.997,3.943631172180176
+6480,2.2323,7.709688663482666
+6490,2.2165,8.620623588562012
+6500,2.177,4.471000671386719
+6510,2.1923,4.813187122344971
+6520,2.2597,4.149843692779541
+6530,2.0405,4.905725002288818
+6540,2.2226,6.384262561798096
+6550,2.0731,7.396153450012207
+6560,2.049,4.137185096740723
+6570,2.1183,5.075475215911865
+6580,2.1061,5.995041370391846
+6590,1.9933,8.12357234954834
+6600,2.1183,4.065415382385254
+6610,1.9599,3.1915132999420166
+6620,2.0938,6.77773380279541
+6630,2.079,5.450401306152344
+6640,2.1767,4.6854963302612305
+6650,2.0163,6.384552955627441
+6660,2.1001,4.87563419342041
+6670,2.0417,3.775580406188965
+6680,2.1564,4.963315486907959
+6690,2.1124,7.423795223236084
+6700,2.0786,5.808844089508057
+6710,2.1347,4.398837089538574
+6720,1.9574,9.142531394958496
+6730,2.0017,5.84624719619751
+6740,1.959,7.081725597381592
+6750,2.0554,4.674961566925049
+6760,2.0301,6.768802642822266
+6770,2.092,3.5873348712921143
+6780,1.9953,4.120325088500977
+6790,2.0688,7.765623569488525
+6800,2.219,4.869895935058594
+6810,2.1398,5.694615840911865
+6820,2.0682,7.104328632354736
+6830,2.052,5.0297932624816895
+6840,2.211,4.520491123199463
+6850,1.9884,3.5239853858947754
+6860,2.1121,7.16238260269165
+6870,2.1276,4.042484760284424
+6880,2.0987,4.16971492767334
+6890,2.3154,7.618093013763428
+6900,2.1601,5.255606174468994
+6910,1.9644,4.716766834259033
+6920,1.9962,4.157872676849365
+6930,2.1439,6.823659896850586
+6940,1.9356,5.478399753570557
+6950,2.1821,4.689790725708008
+6960,2.0843,5.204911231994629
+6970,2.0767,6.285693168640137
+6980,2.1958,8.708101272583008
+6990,2.1266,6.604062080383301
+7000,2.2054,5.105891704559326
+7010,1.9358,5.714686393737793
+7020,2.076,3.433338165283203
+7030,2.0635,4.48581600189209
+7040,2.1363,5.338774681091309
+7050,1.9187,7.590831279754639
+7060,2.1299,5.141582489013672
+7070,2.1819,5.08326530456543
+7080,2.1374,5.986049175262451
+7090,1.9291,4.5929999351501465
+7100,2.0773,3.4489879608154297
+7110,2.0115,4.485833168029785
+7120,2.0676,6.017434120178223
+7130,2.0369,6.2123332023620605
+7140,2.0942,3.7782444953918457
+7150,2.1302,7.148087024688721
+7160,2.1838,5.7953362464904785
+7170,2.0159,3.9319803714752197
+7180,1.9249,4.671140193939209
+7190,2.0409,4.661734104156494
+7200,2.1667,7.39704704284668
+7210,2.1305,8.355154037475586
+7220,2.1391,5.1650285720825195
+7230,2.0748,6.877825736999512
+7240,2.0812,4.656685829162598
+7250,2.0358,5.901729106903076
+7260,2.1514,6.996543884277344
+7270,2.0404,4.831353187561035
+7280,1.9482,7.972875595092773
+7290,2.2954,7.147056579589844
+7300,2.1596,5.505180835723877
+7310,2.1602,7.347818374633789
+7320,2.0548,3.6011338233947754
+7330,2.0395,3.5904648303985596
+7340,2.0023,5.380495071411133
+7350,1.9918,4.518589973449707
+7360,1.9901,5.9018235206604
+7370,2.0245,3.6925086975097656
+7380,2.2619,9.259016990661621
+7390,2.1797,5.5859808921813965
+7400,2.0357,4.7031683921813965
+7410,2.1377,6.260189056396484
+7420,1.9451,4.784355163574219
+7430,2.0388,8.991378784179688
+7440,2.379,6.376221656799316
+7450,2.1915,5.871402740478516
+7460,2.0817,5.426553249359131
+7470,1.924,5.839360237121582
+7480,2.0216,2.95749568939209
+7490,1.905,8.544522285461426
+7500,1.9288,5.736949443817139
+7510,1.9417,4.557583808898926
+7520,2.3085,5.55007266998291
+7530,1.9185,4.35857629776001
+7540,2.1785,6.802826404571533
+7550,1.9825,7.604827880859375
+7560,1.9379,3.024289846420288
+7570,2.0376,4.845406532287598
+7580,2.0193,7.506366729736328
+7590,2.0873,7.756492614746094
+7600,2.1243,6.46793270111084
+7610,2.1875,6.6785359382629395
+7620,2.1792,3.648634433746338
+7630,2.0782,5.907644271850586
+7640,2.2235,7.055563926696777
+7650,2.0072,6.187650203704834
+7660,2.1009,7.704046726226807
+7670,1.9614,5.42090368270874
+7680,2.1542,4.029887676239014
+7690,2.0916,6.325004577636719
+7700,2.0036,3.8288917541503906
+7710,2.1405,5.6230645179748535
+7720,2.1522,6.5077667236328125
+7730,2.1962,4.403681755065918
+7740,2.1528,9.999998092651367
+7750,2.0474,3.427657127380371
+7760,2.0905,8.58780288696289
+7770,2.1772,7.026243686676025
+7780,2.0321,5.6110405921936035
+7790,1.8835,7.481403350830078
+7800,1.9508,4.553055286407471
+7810,2.2059,5.52471923828125
+7820,3.6667,3.0748660564422607
+7830,2.188,4.8406982421875
+7840,1.9868,6.943801403045654
+7850,1.9901,6.901176452636719
+7860,1.9197,5.6528520584106445
+7870,2.2289,9.42558765411377
+7880,2.1969,6.620875835418701
+7890,2.1826,5.273202896118164
+7900,2.2718,6.013164043426514
+7910,1.9115,5.780605792999268
+7920,2.2051,5.694150924682617
+7930,2.0739,6.350783824920654
+7940,2.008,3.847805976867676
+7950,1.9905,5.354320526123047
+7960,2.1381,5.156245231628418
+7970,2.259,5.384928226470947
+7980,2.206,6.737875938415527
+7990,2.1322,3.4885711669921875
+8000,2.2369,7.307969093322754
+8010,1.9901,4.461708068847656
+8020,2.1096,4.739271640777588
+8030,2.0258,4.999455451965332
+8040,2.0223,3.893523931503296
+8050,2.1654,3.8808035850524902
+8060,2.0741,3.7033116817474365
+8070,2.1801,5.134984493255615
+8080,2.1353,5.389072895050049
+8090,2.0169,3.2037174701690674
+8100,2.0105,6.975593090057373
+8110,2.07,4.0726423263549805
+8120,2.1497,9.999998092651367
+8130,1.9266,3.413611888885498
+8140,2.2622,9.220563888549805
+8150,2.0675,4.455341339111328
+8160,2.171,5.176009178161621
+8170,2.1158,3.342127561569214
+8180,2.1136,3.67814564704895
+8190,2.1383,7.2132391929626465
+8200,2.0992,5.090510845184326
+8210,2.0596,4.983635902404785
+8220,2.053,4.354836463928223
+8230,1.9845,4.732203006744385
+8240,1.9879,3.1118311882019043
+8250,2.0729,4.70589542388916
+8260,2.1209,5.572944164276123
+8270,2.2191,9.762155532836914
+8280,2.1546,4.850767612457275
+8290,2.096,4.146419525146484
+8300,2.1326,4.993535995483398
+8310,2.1873,6.308148384094238
+8320,2.1019,7.768478870391846
+8330,2.06,3.5128209590911865
+8340,2.0348,8.1862211227417
+8350,2.1624,4.3322625160217285
+8360,2.0585,6.597864627838135
+8370,2.0265,4.030665874481201
+8380,2.0382,5.775618553161621
+8390,2.0573,3.6797897815704346
+8400,2.0572,4.699132919311523
+8410,2.0878,3.724128246307373
+8420,2.3014,6.061574459075928
+8430,2.0776,7.603193759918213
+8440,2.075,4.737987041473389
+8450,1.9306,3.1090354919433594
+8460,1.9749,4.949204921722412
+8470,2.1108,6.295539855957031
+8480,2.0563,5.153507709503174
+8490,2.1859,4.624126434326172
+8500,2.1191,4.7635884284973145
+8510,2.0735,4.604827404022217
+8520,2.1324,6.650067329406738
+8530,1.9815,4.415541648864746
+8540,1.9876,4.163292407989502
+8550,2.001,2.965481996536255
+8560,1.9329,3.5702438354492188
+8570,2.0726,6.828572750091553
+8580,1.9933,5.8427300453186035
+8590,2.2388,7.576897621154785
+8600,1.9418,3.542816162109375
+8610,2.1486,8.087502479553223
+8620,2.06,4.78680419921875
+8630,2.1584,3.113011598587036
+8640,2.014,3.982513666152954
+8650,1.811,5.133899211883545
+8660,2.0902,6.8349609375
+8670,2.1602,6.302601337432861
+8680,2.1091,3.433237075805664
+8690,2.098,6.184892654418945
+8700,2.1538,5.453632354736328
+8710,1.9261,7.197730541229248
+8720,1.9436,8.1513032913208
+8730,2.0882,5.156404495239258
+8740,1.9373,6.841371059417725
+8750,2.07,5.5138373374938965
+8760,2.2179,4.273646831512451
+8770,1.9038,6.17716121673584
+8780,2.0042,5.650930881500244
+8790,2.018,7.444324970245361
+8800,2.2024,5.6876068115234375
+8810,2.1015,2.757317543029785
+8820,2.0039,5.419109344482422
+8830,2.0869,4.2704877853393555
+8840,2.0181,4.7045392990112305
+8850,2.0192,5.270239353179932
+8860,2.0618,4.050564765930176
+8870,2.0668,5.236793041229248
+8880,2.0369,6.486008644104004
+8890,1.9495,4.113170146942139
+8900,1.9701,6.932297229766846
+8910,2.1513,5.153751373291016
+8920,2.1062,3.6570873260498047
+8930,2.0764,6.329987525939941
+8940,2.0468,5.174140930175781
+8950,1.9971,3.7998836040496826
+8960,2.1847,8.548787117004395
+8970,2.0329,3.677898645401001
+8980,2.1371,6.614884853363037
+8990,1.9745,6.62887716293335
+9000,2.27,5.070212364196777
+9010,2.0103,6.215460300445557
+9020,2.1006,5.103015899658203
+9030,1.9379,4.409531593322754
+9040,2.1983,6.719906806945801
+9050,2.1848,6.215670108795166
+9060,2.2883,6.721705436706543
+9070,2.0083,3.897223472595215
+9080,2.0583,5.6442461013793945
+9090,2.1312,6.394291877746582
+9100,1.9678,2.766772985458374
+9110,2.1135,3.434525966644287
+9120,2.127,4.80876350402832
+9130,1.9927,5.191465377807617
+9140,2.0743,6.8004631996154785
+9150,2.0535,3.939542531967163
+9160,2.2046,3.9124600887298584
+9170,2.0143,6.297423362731934
+9180,2.0839,5.202453136444092
+9190,1.985,6.202855587005615
+9200,2.1004,4.195956230163574
+9210,1.9329,4.236852645874023
+9220,2.2291,5.220149040222168
+9230,2.057,9.999998092651367
+9240,1.9977,6.199564456939697
+9250,2.0441,5.6598734855651855
+9260,2.1307,3.8681833744049072
+9270,2.0349,4.755743503570557
+9280,2.0914,5.985792636871338
+9290,1.8999,4.568302154541016
+9300,2.0736,3.314087390899658
+9310,1.9899,4.549111366271973
+9320,2.0542,3.963134288787842
+9330,2.1633,4.205166816711426
+9340,2.135,4.5056657791137695
+9350,2.0926,4.240407466888428
+9360,1.8936,3.3892221450805664
+9370,1.9597,5.6661481857299805
+9375,nan,nan