Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+base_model: gpt2-medium
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GPT2LMHeadModel",
+    "parent_library": "transformers.models.gpt2.modeling_gpt2"
+  },
+  "base_model_name_or_path": "gpt2-medium",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d1652c7a003dd8627cd4a9648e12d63c6d4c984d8c61b0fca89d4f8de2daeaf
+size 25172088

checkpoint-3125/README.md

@@ -0,0 +1,202 @@
+---
+base_model: gpt2-medium
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

checkpoint-3125/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GPT2LMHeadModel",
+    "parent_library": "transformers.models.gpt2.modeling_gpt2"
+  },
+  "base_model_name_or_path": "gpt2-medium",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-3125/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6272464f0c1bda6ffa9cdadfef39d61b8b78421f8fcbb6ad2806ad028346120
+size 25172088

checkpoint-3125/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbfa0492104e0d14ab252606a95a1182673ac69bc120fe2d5d982be7311ea6da
+size 50372538

checkpoint-3125/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:feded292b067a53a2aeb0e2a23dd8fd5fd080c27efc2767fe4b430da7e5f7d6f
+size 14244

checkpoint-3125/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:392855cc9cbe029377262097ef598767921e2a3bc6937822c989a7603ee182c3
+size 1064

checkpoint-3125/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-3125/tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-3125/trainer_state.json

@@ -0,0 +1,2217 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 3125,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0032,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 2.132196162046908e-06,
+      "loss": 25.0099,
+      "step": 10
+    },
+    {
+      "epoch": 0.0064,
+      "grad_norm": 10.0,
+      "learning_rate": 4.264392324093816e-06,
+      "loss": 21.1499,
+      "step": 20
+    },
+    {
+      "epoch": 0.0096,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 6.396588486140726e-06,
+      "loss": 19.6111,
+      "step": 30
+    },
+    {
+      "epoch": 0.0128,
+      "grad_norm": 10.0,
+      "learning_rate": 8.528784648187633e-06,
+      "loss": 22.3722,
+      "step": 40
+    },
+    {
+      "epoch": 0.016,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 1.0660980810234541e-05,
+      "loss": 20.1462,
+      "step": 50
+    },
+    {
+      "epoch": 0.0192,
+      "grad_norm": 10.0,
+      "learning_rate": 1.2793176972281452e-05,
+      "loss": 20.4235,
+      "step": 60
+    },
+    {
+      "epoch": 0.0224,
+      "grad_norm": 10.0,
+      "learning_rate": 1.4925373134328357e-05,
+      "loss": 20.7853,
+      "step": 70
+    },
+    {
+      "epoch": 0.0256,
+      "grad_norm": 10.0,
+      "learning_rate": 1.7057569296375266e-05,
+      "loss": 19.5459,
+      "step": 80
+    },
+    {
+      "epoch": 0.0288,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 1.9189765458422178e-05,
+      "loss": 18.4778,
+      "step": 90
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 2.1321961620469083e-05,
+      "loss": 16.2191,
+      "step": 100
+    },
+    {
+      "epoch": 0.0352,
+      "grad_norm": 10.0,
+      "learning_rate": 2.345415778251599e-05,
+      "loss": 18.5773,
+      "step": 110
+    },
+    {
+      "epoch": 0.0384,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 2.5586353944562904e-05,
+      "loss": 15.9389,
+      "step": 120
+    },
+    {
+      "epoch": 0.0416,
+      "grad_norm": 10.0,
+      "learning_rate": 2.771855010660981e-05,
+      "loss": 15.6064,
+      "step": 130
+    },
+    {
+      "epoch": 0.0448,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 2.9850746268656714e-05,
+      "loss": 15.6366,
+      "step": 140
+    },
+    {
+      "epoch": 0.048,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 3.1982942430703626e-05,
+      "loss": 16.7232,
+      "step": 150
+    },
+    {
+      "epoch": 0.0512,
+      "grad_norm": 10.0,
+      "learning_rate": 3.411513859275053e-05,
+      "loss": 13.2957,
+      "step": 160
+    },
+    {
+      "epoch": 0.0544,
+      "grad_norm": 10.0,
+      "learning_rate": 3.624733475479744e-05,
+      "loss": 13.3429,
+      "step": 170
+    },
+    {
+      "epoch": 0.0576,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 3.8379530916844355e-05,
+      "loss": 13.3301,
+      "step": 180
+    },
+    {
+      "epoch": 0.0608,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 4.051172707889126e-05,
+      "loss": 15.1708,
+      "step": 190
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 10.0,
+      "learning_rate": 4.2643923240938166e-05,
+      "loss": 14.8533,
+      "step": 200
+    },
+    {
+      "epoch": 0.0672,
+      "grad_norm": 10.0,
+      "learning_rate": 4.477611940298508e-05,
+      "loss": 11.6389,
+      "step": 210
+    },
+    {
+      "epoch": 0.0704,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 4.690831556503198e-05,
+      "loss": 12.2515,
+      "step": 220
+    },
+    {
+      "epoch": 0.0736,
+      "grad_norm": 10.0,
+      "learning_rate": 4.904051172707889e-05,
+      "loss": 12.6637,
+      "step": 230
+    },
+    {
+      "epoch": 0.0768,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 5.117270788912581e-05,
+      "loss": 10.6091,
+      "step": 240
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 5.330490405117271e-05,
+      "loss": 10.9362,
+      "step": 250
+    },
+    {
+      "epoch": 0.0832,
+      "grad_norm": 10.0,
+      "learning_rate": 5.543710021321962e-05,
+      "loss": 12.4545,
+      "step": 260
+    },
+    {
+      "epoch": 0.0864,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 5.756929637526652e-05,
+      "loss": 11.5238,
+      "step": 270
+    },
+    {
+      "epoch": 0.0896,
+      "grad_norm": 10.0,
+      "learning_rate": 5.970149253731343e-05,
+      "loss": 10.8966,
+      "step": 280
+    },
+    {
+      "epoch": 0.0928,
+      "grad_norm": 10.0,
+      "learning_rate": 6.183368869936035e-05,
+      "loss": 10.5903,
+      "step": 290
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 10.0,
+      "learning_rate": 6.396588486140725e-05,
+      "loss": 10.4872,
+      "step": 300
+    },
+    {
+      "epoch": 0.0992,
+      "grad_norm": 10.0,
+      "learning_rate": 6.609808102345416e-05,
+      "loss": 9.9663,
+      "step": 310
+    },
+    {
+      "epoch": 0.1024,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 6.823027718550106e-05,
+      "loss": 9.4655,
+      "step": 320
+    },
+    {
+      "epoch": 0.1056,
+      "grad_norm": 10.0,
+      "learning_rate": 7.036247334754798e-05,
+      "loss": 10.487,
+      "step": 330
+    },
+    {
+      "epoch": 0.1088,
+      "grad_norm": 10.000001907348633,
+      "learning_rate": 7.249466950959489e-05,
+      "loss": 9.3626,
+      "step": 340
+    },
+    {
+      "epoch": 0.112,
+      "grad_norm": 10.0,
+      "learning_rate": 7.46268656716418e-05,
+      "loss": 9.864,
+      "step": 350
+    },
+    {
+      "epoch": 0.1152,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 7.675906183368871e-05,
+      "loss": 8.8008,
+      "step": 360
+    },
+    {
+      "epoch": 0.1184,
+      "grad_norm": 10.0,
+      "learning_rate": 7.889125799573562e-05,
+      "loss": 9.7772,
+      "step": 370
+    },
+    {
+      "epoch": 0.1216,
+      "grad_norm": 10.0,
+      "learning_rate": 8.102345415778252e-05,
+      "loss": 8.7418,
+      "step": 380
+    },
+    {
+      "epoch": 0.1248,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.315565031982943e-05,
+      "loss": 9.1157,
+      "step": 390
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 8.528784648187633e-05,
+      "loss": 8.1395,
+      "step": 400
+    },
+    {
+      "epoch": 0.1312,
+      "grad_norm": 10.0,
+      "learning_rate": 8.742004264392325e-05,
+      "loss": 8.8305,
+      "step": 410
+    },
+    {
+      "epoch": 0.1344,
+      "grad_norm": 10.0,
+      "learning_rate": 8.955223880597016e-05,
+      "loss": 9.2089,
+      "step": 420
+    },
+    {
+      "epoch": 0.1376,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.168443496801706e-05,
+      "loss": 8.5206,
+      "step": 430
+    },
+    {
+      "epoch": 0.1408,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.381663113006397e-05,
+      "loss": 8.8993,
+      "step": 440
+    },
+    {
+      "epoch": 0.144,
+      "grad_norm": 10.0,
+      "learning_rate": 9.594882729211087e-05,
+      "loss": 8.8173,
+      "step": 450
+    },
+    {
+      "epoch": 0.1472,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.808102345415778e-05,
+      "loss": 8.3408,
+      "step": 460
+    },
+    {
+      "epoch": 0.1504,
+      "grad_norm": 10.0,
+      "learning_rate": 9.998877161464182e-05,
+      "loss": 7.5853,
+      "step": 470
+    },
+    {
+      "epoch": 0.1536,
+      "grad_norm": 10.0,
+      "learning_rate": 9.987648776105997e-05,
+      "loss": 7.4544,
+      "step": 480
+    },
+    {
+      "epoch": 0.1568,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.97642039074781e-05,
+      "loss": 8.0499,
+      "step": 490
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 10.0,
+      "learning_rate": 9.965192005389625e-05,
+      "loss": 8.2335,
+      "step": 500
+    },
+    {
+      "epoch": 0.1632,
+      "grad_norm": 10.0,
+      "learning_rate": 9.95396362003144e-05,
+      "loss": 7.0574,
+      "step": 510
+    },
+    {
+      "epoch": 0.1664,
+      "grad_norm": 10.0,
+      "learning_rate": 9.942735234673256e-05,
+      "loss": 6.8412,
+      "step": 520
+    },
+    {
+      "epoch": 0.1696,
+      "grad_norm": 10.0,
+      "learning_rate": 9.931506849315069e-05,
+      "loss": 7.1418,
+      "step": 530
+    },
+    {
+      "epoch": 0.1728,
+      "grad_norm": 10.0,
+      "learning_rate": 9.920278463956883e-05,
+      "loss": 7.1722,
+      "step": 540
+    },
+    {
+      "epoch": 0.176,
+      "grad_norm": 10.0,
+      "learning_rate": 9.909050078598698e-05,
+      "loss": 6.7641,
+      "step": 550
+    },
+    {
+      "epoch": 0.1792,
+      "grad_norm": 10.0,
+      "learning_rate": 9.897821693240512e-05,
+      "loss": 6.3646,
+      "step": 560
+    },
+    {
+      "epoch": 0.1824,
+      "grad_norm": 10.0,
+      "learning_rate": 9.886593307882327e-05,
+      "loss": 6.3469,
+      "step": 570
+    },
+    {
+      "epoch": 0.1856,
+      "grad_norm": 10.0,
+      "learning_rate": 9.875364922524142e-05,
+      "loss": 6.38,
+      "step": 580
+    },
+    {
+      "epoch": 0.1888,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.864136537165956e-05,
+      "loss": 5.8154,
+      "step": 590
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.852908151807771e-05,
+      "loss": 5.6794,
+      "step": 600
+    },
+    {
+      "epoch": 0.1952,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.841679766449586e-05,
+      "loss": 5.4415,
+      "step": 610
+    },
+    {
+      "epoch": 0.1984,
+      "grad_norm": 10.0,
+      "learning_rate": 9.8304513810914e-05,
+      "loss": 4.7814,
+      "step": 620
+    },
+    {
+      "epoch": 0.2016,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.819222995733213e-05,
+      "loss": 4.8304,
+      "step": 630
+    },
+    {
+      "epoch": 0.2048,
+      "grad_norm": 10.0,
+      "learning_rate": 9.807994610375028e-05,
+      "loss": 4.9017,
+      "step": 640
+    },
+    {
+      "epoch": 0.208,
+      "grad_norm": 10.0,
+      "learning_rate": 9.796766225016843e-05,
+      "loss": 4.5826,
+      "step": 650
+    },
+    {
+      "epoch": 0.2112,
+      "grad_norm": 10.0,
+      "learning_rate": 9.785537839658657e-05,
+      "loss": 4.4032,
+      "step": 660
+    },
+    {
+      "epoch": 0.2144,
+      "grad_norm": 10.0,
+      "learning_rate": 9.774309454300472e-05,
+      "loss": 4.4789,
+      "step": 670
+    },
+    {
+      "epoch": 0.2176,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.763081068942287e-05,
+      "loss": 3.5318,
+      "step": 680
+    },
+    {
+      "epoch": 0.2208,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.751852683584101e-05,
+      "loss": 3.9803,
+      "step": 690
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 10.0,
+      "learning_rate": 9.740624298225916e-05,
+      "loss": 3.3411,
+      "step": 700
+    },
+    {
+      "epoch": 0.2272,
+      "grad_norm": 10.0,
+      "learning_rate": 9.729395912867731e-05,
+      "loss": 3.8421,
+      "step": 710
+    },
+    {
+      "epoch": 0.2304,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.718167527509545e-05,
+      "loss": 3.8598,
+      "step": 720
+    },
+    {
+      "epoch": 0.2336,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.706939142151358e-05,
+      "loss": 3.6175,
+      "step": 730
+    },
+    {
+      "epoch": 0.2368,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.695710756793174e-05,
+      "loss": 3.3833,
+      "step": 740
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 10.0,
+      "learning_rate": 9.684482371434989e-05,
+      "loss": 3.1501,
+      "step": 750
+    },
+    {
+      "epoch": 0.2432,
+      "grad_norm": 10.0,
+      "learning_rate": 9.673253986076802e-05,
+      "loss": 3.2144,
+      "step": 760
+    },
+    {
+      "epoch": 0.2464,
+      "grad_norm": 10.0,
+      "learning_rate": 9.662025600718617e-05,
+      "loss": 3.1372,
+      "step": 770
+    },
+    {
+      "epoch": 0.2496,
+      "grad_norm": 10.0,
+      "learning_rate": 9.650797215360432e-05,
+      "loss": 2.728,
+      "step": 780
+    },
+    {
+      "epoch": 0.2528,
+      "grad_norm": 10.0,
+      "learning_rate": 9.639568830002246e-05,
+      "loss": 2.7485,
+      "step": 790
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.628340444644061e-05,
+      "loss": 2.7431,
+      "step": 800
+    },
+    {
+      "epoch": 0.2592,
+      "grad_norm": 10.0,
+      "learning_rate": 9.617112059285875e-05,
+      "loss": 2.534,
+      "step": 810
+    },
+    {
+      "epoch": 0.2624,
+      "grad_norm": 10.0,
+      "learning_rate": 9.605883673927689e-05,
+      "loss": 2.6129,
+      "step": 820
+    },
+    {
+      "epoch": 0.2656,
+      "grad_norm": 10.0,
+      "learning_rate": 9.594655288569504e-05,
+      "loss": 2.6607,
+      "step": 830
+    },
+    {
+      "epoch": 0.2688,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.583426903211319e-05,
+      "loss": 2.5158,
+      "step": 840
+    },
+    {
+      "epoch": 0.272,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.572198517853134e-05,
+      "loss": 2.2633,
+      "step": 850
+    },
+    {
+      "epoch": 0.2752,
+      "grad_norm": 10.0,
+      "learning_rate": 9.560970132494948e-05,
+      "loss": 2.4407,
+      "step": 860
+    },
+    {
+      "epoch": 0.2784,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.549741747136763e-05,
+      "loss": 2.1584,
+      "step": 870
+    },
+    {
+      "epoch": 0.2816,
+      "grad_norm": 10.0,
+      "learning_rate": 9.538513361778578e-05,
+      "loss": 2.1552,
+      "step": 880
+    },
+    {
+      "epoch": 0.2848,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.527284976420391e-05,
+      "loss": 1.8968,
+      "step": 890
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.516056591062205e-05,
+      "loss": 1.9182,
+      "step": 900
+    },
+    {
+      "epoch": 0.2912,
+      "grad_norm": 10.0,
+      "learning_rate": 9.50482820570402e-05,
+      "loss": 1.9413,
+      "step": 910
+    },
+    {
+      "epoch": 0.2944,
+      "grad_norm": 10.0,
+      "learning_rate": 9.493599820345834e-05,
+      "loss": 1.8959,
+      "step": 920
+    },
+    {
+      "epoch": 0.2976,
+      "grad_norm": 10.0,
+      "learning_rate": 9.482371434987649e-05,
+      "loss": 1.9225,
+      "step": 930
+    },
+    {
+      "epoch": 0.3008,
+      "grad_norm": 10.0,
+      "learning_rate": 9.471143049629464e-05,
+      "loss": 1.8802,
+      "step": 940
+    },
+    {
+      "epoch": 0.304,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.459914664271278e-05,
+      "loss": 1.8896,
+      "step": 950
+    },
+    {
+      "epoch": 0.3072,
+      "grad_norm": 10.0,
+      "learning_rate": 9.448686278913093e-05,
+      "loss": 1.7142,
+      "step": 960
+    },
+    {
+      "epoch": 0.3104,
+      "grad_norm": 10.0,
+      "learning_rate": 9.437457893554908e-05,
+      "loss": 1.6888,
+      "step": 970
+    },
+    {
+      "epoch": 0.3136,
+      "grad_norm": 10.0,
+      "learning_rate": 9.426229508196722e-05,
+      "loss": 1.6177,
+      "step": 980
+    },
+    {
+      "epoch": 0.3168,
+      "grad_norm": 10.0,
+      "learning_rate": 9.415001122838537e-05,
+      "loss": 1.645,
+      "step": 990
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 10.0,
+      "learning_rate": 9.40377273748035e-05,
+      "loss": 1.6792,
+      "step": 1000
+    },
+    {
+      "epoch": 0.3232,
+      "grad_norm": 10.0,
+      "learning_rate": 9.392544352122165e-05,
+      "loss": 1.8599,
+      "step": 1010
+    },
+    {
+      "epoch": 0.3264,
+      "grad_norm": 10.0,
+      "learning_rate": 9.381315966763979e-05,
+      "loss": 1.6779,
+      "step": 1020
+    },
+    {
+      "epoch": 0.3296,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.370087581405794e-05,
+      "loss": 1.4669,
+      "step": 1030
+    },
+    {
+      "epoch": 0.3328,
+      "grad_norm": 10.0,
+      "learning_rate": 9.358859196047609e-05,
+      "loss": 1.3649,
+      "step": 1040
+    },
+    {
+      "epoch": 0.336,
+      "grad_norm": 10.0,
+      "learning_rate": 9.347630810689423e-05,
+      "loss": 1.3987,
+      "step": 1050
+    },
+    {
+      "epoch": 0.3392,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.336402425331238e-05,
+      "loss": 1.2421,
+      "step": 1060
+    },
+    {
+      "epoch": 0.3424,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.325174039973053e-05,
+      "loss": 1.3011,
+      "step": 1070
+    },
+    {
+      "epoch": 0.3456,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.313945654614867e-05,
+      "loss": 1.2287,
+      "step": 1080
+    },
+    {
+      "epoch": 0.3488,
+      "grad_norm": 10.0,
+      "learning_rate": 9.30271726925668e-05,
+      "loss": 1.1587,
+      "step": 1090
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 10.0,
+      "learning_rate": 9.291488883898496e-05,
+      "loss": 1.2415,
+      "step": 1100
+    },
+    {
+      "epoch": 0.3552,
+      "grad_norm": 10.0,
+      "learning_rate": 9.280260498540311e-05,
+      "loss": 1.1415,
+      "step": 1110
+    },
+    {
+      "epoch": 0.3584,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.269032113182124e-05,
+      "loss": 1.1941,
+      "step": 1120
+    },
+    {
+      "epoch": 0.3616,
+      "grad_norm": 10.0,
+      "learning_rate": 9.25780372782394e-05,
+      "loss": 1.1668,
+      "step": 1130
+    },
+    {
+      "epoch": 0.3648,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.246575342465755e-05,
+      "loss": 1.0887,
+      "step": 1140
+    },
+    {
+      "epoch": 0.368,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.235346957107568e-05,
+      "loss": 1.0604,
+      "step": 1150
+    },
+    {
+      "epoch": 0.3712,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.224118571749383e-05,
+      "loss": 0.9446,
+      "step": 1160
+    },
+    {
+      "epoch": 0.3744,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.212890186391197e-05,
+      "loss": 0.9737,
+      "step": 1170
+    },
+    {
+      "epoch": 0.3776,
+      "grad_norm": 10.0,
+      "learning_rate": 9.201661801033011e-05,
+      "loss": 0.9654,
+      "step": 1180
+    },
+    {
+      "epoch": 0.3808,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.190433415674826e-05,
+      "loss": 1.0253,
+      "step": 1190
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.179205030316641e-05,
+      "loss": 0.8597,
+      "step": 1200
+    },
+    {
+      "epoch": 0.3872,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 9.167976644958456e-05,
+      "loss": 0.8202,
+      "step": 1210
+    },
+    {
+      "epoch": 0.3904,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.15674825960027e-05,
+      "loss": 0.8386,
+      "step": 1220
+    },
+    {
+      "epoch": 0.3936,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.145519874242085e-05,
+      "loss": 0.8318,
+      "step": 1230
+    },
+    {
+      "epoch": 0.3968,
+      "grad_norm": 10.0,
+      "learning_rate": 9.1342914888839e-05,
+      "loss": 0.8571,
+      "step": 1240
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.123063103525713e-05,
+      "loss": 0.8326,
+      "step": 1250
+    },
+    {
+      "epoch": 0.4032,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.111834718167527e-05,
+      "loss": 0.7655,
+      "step": 1260
+    },
+    {
+      "epoch": 0.4064,
+      "grad_norm": 10.0,
+      "learning_rate": 9.100606332809342e-05,
+      "loss": 0.747,
+      "step": 1270
+    },
+    {
+      "epoch": 0.4096,
+      "grad_norm": 10.0,
+      "learning_rate": 9.089377947451156e-05,
+      "loss": 0.8671,
+      "step": 1280
+    },
+    {
+      "epoch": 0.4128,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.078149562092971e-05,
+      "loss": 0.8758,
+      "step": 1290
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 10.000000953674316,
+      "learning_rate": 9.066921176734786e-05,
+      "loss": 0.8714,
+      "step": 1300
+    },
+    {
+      "epoch": 0.4192,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.055692791376601e-05,
+      "loss": 0.8022,
+      "step": 1310
+    },
+    {
+      "epoch": 0.4224,
+      "grad_norm": 8.615378379821777,
+      "learning_rate": 9.044464406018415e-05,
+      "loss": 0.7234,
+      "step": 1320
+    },
+    {
+      "epoch": 0.4256,
+      "grad_norm": 10.0,
+      "learning_rate": 9.03323602066023e-05,
+      "loss": 0.7416,
+      "step": 1330
+    },
+    {
+      "epoch": 0.4288,
+      "grad_norm": 10.0,
+      "learning_rate": 9.022007635302045e-05,
+      "loss": 0.6898,
+      "step": 1340
+    },
+    {
+      "epoch": 0.432,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 9.010779249943859e-05,
+      "loss": 0.6261,
+      "step": 1350
+    },
+    {
+      "epoch": 0.4352,
+      "grad_norm": 10.0,
+      "learning_rate": 8.999550864585672e-05,
+      "loss": 0.6267,
+      "step": 1360
+    },
+    {
+      "epoch": 0.4384,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.988322479227488e-05,
+      "loss": 0.6171,
+      "step": 1370
+    },
+    {
+      "epoch": 0.4416,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.977094093869301e-05,
+      "loss": 0.5677,
+      "step": 1380
+    },
+    {
+      "epoch": 0.4448,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.965865708511116e-05,
+      "loss": 0.5227,
+      "step": 1390
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.954637323152931e-05,
+      "loss": 0.5775,
+      "step": 1400
+    },
+    {
+      "epoch": 0.4512,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.943408937794746e-05,
+      "loss": 0.6166,
+      "step": 1410
+    },
+    {
+      "epoch": 0.4544,
+      "grad_norm": 10.0,
+      "learning_rate": 8.93218055243656e-05,
+      "loss": 0.6165,
+      "step": 1420
+    },
+    {
+      "epoch": 0.4576,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.920952167078375e-05,
+      "loss": 0.5849,
+      "step": 1430
+    },
+    {
+      "epoch": 0.4608,
+      "grad_norm": 10.0,
+      "learning_rate": 8.909723781720189e-05,
+      "loss": 0.6439,
+      "step": 1440
+    },
+    {
+      "epoch": 0.464,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.898495396362003e-05,
+      "loss": 0.5886,
+      "step": 1450
+    },
+    {
+      "epoch": 0.4672,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.887267011003818e-05,
+      "loss": 0.5935,
+      "step": 1460
+    },
+    {
+      "epoch": 0.4704,
+      "grad_norm": 10.0,
+      "learning_rate": 8.876038625645633e-05,
+      "loss": 0.5868,
+      "step": 1470
+    },
+    {
+      "epoch": 0.4736,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.864810240287447e-05,
+      "loss": 0.5489,
+      "step": 1480
+    },
+    {
+      "epoch": 0.4768,
+      "grad_norm": 10.0,
+      "learning_rate": 8.853581854929262e-05,
+      "loss": 0.5106,
+      "step": 1490
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.842353469571077e-05,
+      "loss": 0.4885,
+      "step": 1500
+    },
+    {
+      "epoch": 0.4832,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.83112508421289e-05,
+      "loss": 0.5323,
+      "step": 1510
+    },
+    {
+      "epoch": 0.4864,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.819896698854705e-05,
+      "loss": 0.4359,
+      "step": 1520
+    },
+    {
+      "epoch": 0.4896,
+      "grad_norm": 7.423900604248047,
+      "learning_rate": 8.808668313496519e-05,
+      "loss": 0.4789,
+      "step": 1530
+    },
+    {
+      "epoch": 0.4928,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.797439928138334e-05,
+      "loss": 0.4367,
+      "step": 1540
+    },
+    {
+      "epoch": 0.496,
+      "grad_norm": 7.3594489097595215,
+      "learning_rate": 8.786211542780148e-05,
+      "loss": 0.465,
+      "step": 1550
+    },
+    {
+      "epoch": 0.4992,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.774983157421963e-05,
+      "loss": 0.4244,
+      "step": 1560
+    },
+    {
+      "epoch": 0.5024,
+      "grad_norm": 10.0,
+      "learning_rate": 8.763754772063778e-05,
+      "loss": 0.4311,
+      "step": 1570
+    },
+    {
+      "epoch": 0.5056,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.752526386705592e-05,
+      "loss": 0.4787,
+      "step": 1580
+    },
+    {
+      "epoch": 0.5088,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.741298001347407e-05,
+      "loss": 0.4777,
+      "step": 1590
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 10.0,
+      "learning_rate": 8.730069615989222e-05,
+      "loss": 0.469,
+      "step": 1600
+    },
+    {
+      "epoch": 0.5152,
+      "grad_norm": 10.0,
+      "learning_rate": 8.718841230631036e-05,
+      "loss": 0.4519,
+      "step": 1610
+    },
+    {
+      "epoch": 0.5184,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.70761284527285e-05,
+      "loss": 0.4084,
+      "step": 1620
+    },
+    {
+      "epoch": 0.5216,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.696384459914664e-05,
+      "loss": 0.3627,
+      "step": 1630
+    },
+    {
+      "epoch": 0.5248,
+      "grad_norm": 8.502776145935059,
+      "learning_rate": 8.68515607455648e-05,
+      "loss": 0.3742,
+      "step": 1640
+    },
+    {
+      "epoch": 0.528,
+      "grad_norm": 10.0,
+      "learning_rate": 8.673927689198293e-05,
+      "loss": 0.3525,
+      "step": 1650
+    },
+    {
+      "epoch": 0.5312,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.662699303840108e-05,
+      "loss": 0.3889,
+      "step": 1660
+    },
+    {
+      "epoch": 0.5344,
+      "grad_norm": 10.0,
+      "learning_rate": 8.651470918481923e-05,
+      "loss": 0.5584,
+      "step": 1670
+    },
+    {
+      "epoch": 0.5376,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.640242533123737e-05,
+      "loss": 0.3988,
+      "step": 1680
+    },
+    {
+      "epoch": 0.5408,
+      "grad_norm": 8.782861709594727,
+      "learning_rate": 8.629014147765552e-05,
+      "loss": 0.3439,
+      "step": 1690
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 8.174032211303711,
+      "learning_rate": 8.617785762407367e-05,
+      "loss": 0.3265,
+      "step": 1700
+    },
+    {
+      "epoch": 0.5472,
+      "grad_norm": 4.737403392791748,
+      "learning_rate": 8.606557377049181e-05,
+      "loss": 0.329,
+      "step": 1710
+    },
+    {
+      "epoch": 0.5504,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.595328991690995e-05,
+      "loss": 0.3093,
+      "step": 1720
+    },
+    {
+      "epoch": 0.5536,
+      "grad_norm": 10.0,
+      "learning_rate": 8.58410060633281e-05,
+      "loss": 0.3119,
+      "step": 1730
+    },
+    {
+      "epoch": 0.5568,
+      "grad_norm": 10.0,
+      "learning_rate": 8.572872220974623e-05,
+      "loss": 0.3227,
+      "step": 1740
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 9.882418632507324,
+      "learning_rate": 8.561643835616438e-05,
+      "loss": 0.2979,
+      "step": 1750
+    },
+    {
+      "epoch": 0.5632,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.550415450258253e-05,
+      "loss": 0.2693,
+      "step": 1760
+    },
+    {
+      "epoch": 0.5664,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.539187064900069e-05,
+      "loss": 0.3187,
+      "step": 1770
+    },
+    {
+      "epoch": 0.5696,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.527958679541882e-05,
+      "loss": 0.291,
+      "step": 1780
+    },
+    {
+      "epoch": 0.5728,
+      "grad_norm": 7.839264869689941,
+      "learning_rate": 8.516730294183697e-05,
+      "loss": 0.2833,
+      "step": 1790
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 4.611258029937744,
+      "learning_rate": 8.505501908825511e-05,
+      "loss": 0.2525,
+      "step": 1800
+    },
+    {
+      "epoch": 0.5792,
+      "grad_norm": 4.125186443328857,
+      "learning_rate": 8.494273523467325e-05,
+      "loss": 0.2727,
+      "step": 1810
+    },
+    {
+      "epoch": 0.5824,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.48304513810914e-05,
+      "loss": 0.2718,
+      "step": 1820
+    },
+    {
+      "epoch": 0.5856,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.471816752750955e-05,
+      "loss": 0.2787,
+      "step": 1830
+    },
+    {
+      "epoch": 0.5888,
+      "grad_norm": 4.507540225982666,
+      "learning_rate": 8.460588367392769e-05,
+      "loss": 0.2903,
+      "step": 1840
+    },
+    {
+      "epoch": 0.592,
+      "grad_norm": 8.763723373413086,
+      "learning_rate": 8.449359982034584e-05,
+      "loss": 0.2821,
+      "step": 1850
+    },
+    {
+      "epoch": 0.5952,
+      "grad_norm": 6.04974365234375,
+      "learning_rate": 8.438131596676399e-05,
+      "loss": 0.2469,
+      "step": 1860
+    },
+    {
+      "epoch": 0.5984,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.426903211318214e-05,
+      "loss": 0.2389,
+      "step": 1870
+    },
+    {
+      "epoch": 0.6016,
+      "grad_norm": 10.0,
+      "learning_rate": 8.415674825960028e-05,
+      "loss": 0.2749,
+      "step": 1880
+    },
+    {
+      "epoch": 0.6048,
+      "grad_norm": 10.0,
+      "learning_rate": 8.404446440601843e-05,
+      "loss": 0.2829,
+      "step": 1890
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 9.99999713897705,
+      "learning_rate": 8.393218055243656e-05,
+      "loss": 0.2725,
+      "step": 1900
+    },
+    {
+      "epoch": 0.6112,
+      "grad_norm": 10.000001907348633,
+      "learning_rate": 8.38198966988547e-05,
+      "loss": 0.2698,
+      "step": 1910
+    },
+    {
+      "epoch": 0.6144,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.370761284527285e-05,
+      "loss": 0.2678,
+      "step": 1920
+    },
+    {
+      "epoch": 0.6176,
+      "grad_norm": 10.0,
+      "learning_rate": 8.3595328991691e-05,
+      "loss": 0.2741,
+      "step": 1930
+    },
+    {
+      "epoch": 0.6208,
+      "grad_norm": 10.0,
+      "learning_rate": 8.348304513810914e-05,
+      "loss": 0.2475,
+      "step": 1940
+    },
+    {
+      "epoch": 0.624,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.337076128452729e-05,
+      "loss": 0.2711,
+      "step": 1950
+    },
+    {
+      "epoch": 0.6272,
+      "grad_norm": 10.0,
+      "learning_rate": 8.325847743094544e-05,
+      "loss": 0.2592,
+      "step": 1960
+    },
+    {
+      "epoch": 0.6304,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.314619357736358e-05,
+      "loss": 0.2348,
+      "step": 1970
+    },
+    {
+      "epoch": 0.6336,
+      "grad_norm": 7.087158679962158,
+      "learning_rate": 8.303390972378173e-05,
+      "loss": 0.2185,
+      "step": 1980
+    },
+    {
+      "epoch": 0.6368,
+      "grad_norm": 6.557620048522949,
+      "learning_rate": 8.292162587019986e-05,
+      "loss": 0.2443,
+      "step": 1990
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.280934201661802e-05,
+      "loss": 0.2012,
+      "step": 2000
+    },
+    {
+      "epoch": 0.6432,
+      "grad_norm": 8.828990936279297,
+      "learning_rate": 8.269705816303615e-05,
+      "loss": 0.1992,
+      "step": 2010
+    },
+    {
+      "epoch": 0.6464,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.25847743094543e-05,
+      "loss": 0.2005,
+      "step": 2020
+    },
+    {
+      "epoch": 0.6496,
+      "grad_norm": 10.0,
+      "learning_rate": 8.247249045587245e-05,
+      "loss": 0.2244,
+      "step": 2030
+    },
+    {
+      "epoch": 0.6528,
+      "grad_norm": 10.0,
+      "learning_rate": 8.236020660229059e-05,
+      "loss": 0.3351,
+      "step": 2040
+    },
+    {
+      "epoch": 0.656,
+      "grad_norm": 10.0,
+      "learning_rate": 8.224792274870874e-05,
+      "loss": 0.3391,
+      "step": 2050
+    },
+    {
+      "epoch": 0.6592,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.213563889512689e-05,
+      "loss": 0.2319,
+      "step": 2060
+    },
+    {
+      "epoch": 0.6624,
+      "grad_norm": 10.0,
+      "learning_rate": 8.202335504154503e-05,
+      "loss": 0.2222,
+      "step": 2070
+    },
+    {
+      "epoch": 0.6656,
+      "grad_norm": 8.043660163879395,
+      "learning_rate": 8.191107118796317e-05,
+      "loss": 0.1829,
+      "step": 2080
+    },
+    {
+      "epoch": 0.6688,
+      "grad_norm": 5.343007564544678,
+      "learning_rate": 8.179878733438132e-05,
+      "loss": 0.1756,
+      "step": 2090
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 3.530430316925049,
+      "learning_rate": 8.168650348079947e-05,
+      "loss": 0.1959,
+      "step": 2100
+    },
+    {
+      "epoch": 0.6752,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.15742196272176e-05,
+      "loss": 0.178,
+      "step": 2110
+    },
+    {
+      "epoch": 0.6784,
+      "grad_norm": 10.0,
+      "learning_rate": 8.146193577363576e-05,
+      "loss": 0.1867,
+      "step": 2120
+    },
+    {
+      "epoch": 0.6816,
+      "grad_norm": 8.584357261657715,
+      "learning_rate": 8.13496519200539e-05,
+      "loss": 0.1832,
+      "step": 2130
+    },
+    {
+      "epoch": 0.6848,
+      "grad_norm": 4.891357421875,
+      "learning_rate": 8.123736806647204e-05,
+      "loss": 0.1771,
+      "step": 2140
+    },
+    {
+      "epoch": 0.688,
+      "grad_norm": 5.522593021392822,
+      "learning_rate": 8.11250842128902e-05,
+      "loss": 0.1527,
+      "step": 2150
+    },
+    {
+      "epoch": 0.6912,
+      "grad_norm": 9.386947631835938,
+      "learning_rate": 8.101280035930835e-05,
+      "loss": 0.1769,
+      "step": 2160
+    },
+    {
+      "epoch": 0.6944,
+      "grad_norm": 10.0,
+      "learning_rate": 8.090051650572648e-05,
+      "loss": 0.1889,
+      "step": 2170
+    },
+    {
+      "epoch": 0.6976,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.078823265214462e-05,
+      "loss": 0.1827,
+      "step": 2180
+    },
+    {
+      "epoch": 0.7008,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.067594879856277e-05,
+      "loss": 0.2055,
+      "step": 2190
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 9.999998092651367,
+      "learning_rate": 8.056366494498092e-05,
+      "loss": 0.1802,
+      "step": 2200
+    },
+    {
+      "epoch": 0.7072,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 8.045138109139906e-05,
+      "loss": 0.1706,
+      "step": 2210
+    },
+    {
+      "epoch": 0.7104,
+      "grad_norm": 3.970720052719116,
+      "learning_rate": 8.033909723781721e-05,
+      "loss": 0.1604,
+      "step": 2220
+    },
+    {
+      "epoch": 0.7136,
+      "grad_norm": 3.6069769859313965,
+      "learning_rate": 8.022681338423536e-05,
+      "loss": 0.1453,
+      "step": 2230
+    },
+    {
+      "epoch": 0.7168,
+      "grad_norm": 3.9357783794403076,
+      "learning_rate": 8.01145295306535e-05,
+      "loss": 0.1516,
+      "step": 2240
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 5.958951950073242,
+      "learning_rate": 8.000224567707165e-05,
+      "loss": 0.1512,
+      "step": 2250
+    },
+    {
+      "epoch": 0.7232,
+      "grad_norm": 9.320231437683105,
+      "learning_rate": 7.988996182348978e-05,
+      "loss": 0.1642,
+      "step": 2260
+    },
+    {
+      "epoch": 0.7264,
+      "grad_norm": 4.681884765625,
+      "learning_rate": 7.977767796990792e-05,
+      "loss": 0.1613,
+      "step": 2270
+    },
+    {
+      "epoch": 0.7296,
+      "grad_norm": 7.769372940063477,
+      "learning_rate": 7.966539411632607e-05,
+      "loss": 0.1544,
+      "step": 2280
+    },
+    {
+      "epoch": 0.7328,
+      "grad_norm": 4.547751426696777,
+      "learning_rate": 7.955311026274422e-05,
+      "loss": 0.1636,
+      "step": 2290
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 8.214717864990234,
+      "learning_rate": 7.944082640916236e-05,
+      "loss": 0.1372,
+      "step": 2300
+    },
+    {
+      "epoch": 0.7392,
+      "grad_norm": 7.887718677520752,
+      "learning_rate": 7.932854255558051e-05,
+      "loss": 0.158,
+      "step": 2310
+    },
+    {
+      "epoch": 0.7424,
+      "grad_norm": 9.999999046325684,
+      "learning_rate": 7.921625870199866e-05,
+      "loss": 0.1611,
+      "step": 2320
+    },
+    {
+      "epoch": 0.7456,
+      "grad_norm": 3.0139806270599365,
+      "learning_rate": 7.910397484841681e-05,
+      "loss": 0.1584,
+      "step": 2330
+    },
+    {
+      "epoch": 0.7488,
+      "grad_norm": 5.308823585510254,
+      "learning_rate": 7.899169099483495e-05,
+      "loss": 0.1489,
+      "step": 2340
+    },
+    {
+      "epoch": 0.752,
+      "grad_norm": 1.894464373588562,
+      "learning_rate": 7.887940714125309e-05,
+      "loss": 0.1335,
+      "step": 2350
+    },
+    {
+      "epoch": 0.7552,
+      "grad_norm": 8.55656909942627,
+      "learning_rate": 7.876712328767124e-05,
+      "loss": 0.1363,
+      "step": 2360
+    },
+    {
+      "epoch": 0.7584,
+      "grad_norm": 7.630896091461182,
+      "learning_rate": 7.865483943408937e-05,
+      "loss": 0.1398,
+      "step": 2370
+    },
+    {
+      "epoch": 0.7616,
+      "grad_norm": 9.361071586608887,
+      "learning_rate": 7.854255558050752e-05,
+      "loss": 0.1307,
+      "step": 2380
+    },
+    {
+      "epoch": 0.7648,
+      "grad_norm": 3.613555431365967,
+      "learning_rate": 7.843027172692568e-05,
+      "loss": 0.1461,
+      "step": 2390
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 4.020608901977539,
+      "learning_rate": 7.831798787334381e-05,
+      "loss": 0.1432,
+      "step": 2400
+    },
+    {
+      "epoch": 0.7712,
+      "grad_norm": 6.401773452758789,
+      "learning_rate": 7.820570401976196e-05,
+      "loss": 0.1396,
+      "step": 2410
+    },
+    {
+      "epoch": 0.7744,
+      "grad_norm": 6.332232475280762,
+      "learning_rate": 7.809342016618011e-05,
+      "loss": 0.1355,
+      "step": 2420
+    },
+    {
+      "epoch": 0.7776,
+      "grad_norm": 2.399341344833374,
+      "learning_rate": 7.798113631259825e-05,
+      "loss": 0.1416,
+      "step": 2430
+    },
+    {
+      "epoch": 0.7808,
+      "grad_norm": 5.8365654945373535,
+      "learning_rate": 7.78688524590164e-05,
+      "loss": 0.1265,
+      "step": 2440
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 10.0,
+      "learning_rate": 7.775656860543454e-05,
+      "loss": 0.6647,
+      "step": 2450
+    },
+    {
+      "epoch": 0.7872,
+      "grad_norm": 5.435879230499268,
+      "learning_rate": 7.764428475185269e-05,
+      "loss": 0.1334,
+      "step": 2460
+    },
+    {
+      "epoch": 0.7904,
+      "grad_norm": 10.0,
+      "learning_rate": 7.753200089827083e-05,
+      "loss": 0.1284,
+      "step": 2470
+    },
+    {
+      "epoch": 0.7936,
+      "grad_norm": 7.202799320220947,
+      "learning_rate": 7.741971704468898e-05,
+      "loss": 0.1369,
+      "step": 2480
+    },
+    {
+      "epoch": 0.7968,
+      "grad_norm": 2.0989317893981934,
+      "learning_rate": 7.730743319110713e-05,
+      "loss": 0.1105,
+      "step": 2490
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 4.924466609954834,
+      "learning_rate": 7.719514933752526e-05,
+      "loss": 0.1226,
+      "step": 2500
+    },
+    {
+      "epoch": 0.8032,
+      "grad_norm": 7.978616714477539,
+      "learning_rate": 7.708286548394342e-05,
+      "loss": 0.1089,
+      "step": 2510
+    },
+    {
+      "epoch": 0.8064,
+      "grad_norm": 4.150684356689453,
+      "learning_rate": 7.697058163036157e-05,
+      "loss": 0.1234,
+      "step": 2520
+    },
+    {
+      "epoch": 0.8096,
+      "grad_norm": 4.818097114562988,
+      "learning_rate": 7.68582977767797e-05,
+      "loss": 0.1188,
+      "step": 2530
+    },
+    {
+      "epoch": 0.8128,
+      "grad_norm": 2.5419697761535645,
+      "learning_rate": 7.674601392319784e-05,
+      "loss": 0.113,
+      "step": 2540
+    },
+    {
+      "epoch": 0.816,
+      "grad_norm": 5.721562385559082,
+      "learning_rate": 7.663373006961599e-05,
+      "loss": 0.109,
+      "step": 2550
+    },
+    {
+      "epoch": 0.8192,
+      "grad_norm": 2.7478086948394775,
+      "learning_rate": 7.652144621603414e-05,
+      "loss": 0.101,
+      "step": 2560
+    },
+    {
+      "epoch": 0.8224,
+      "grad_norm": 9.692163467407227,
+      "learning_rate": 7.640916236245228e-05,
+      "loss": 0.1156,
+      "step": 2570
+    },
+    {
+      "epoch": 0.8256,
+      "grad_norm": 6.791487693786621,
+      "learning_rate": 7.629687850887043e-05,
+      "loss": 0.12,
+      "step": 2580
+    },
+    {
+      "epoch": 0.8288,
+      "grad_norm": 4.655740261077881,
+      "learning_rate": 7.618459465528858e-05,
+      "loss": 0.114,
+      "step": 2590
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 2.558154344558716,
+      "learning_rate": 7.607231080170672e-05,
+      "loss": 0.102,
+      "step": 2600
+    },
+    {
+      "epoch": 0.8352,
+      "grad_norm": 5.348006725311279,
+      "learning_rate": 7.596002694812487e-05,
+      "loss": 0.156,
+      "step": 2610
+    },
+    {
+      "epoch": 0.8384,
+      "grad_norm": 4.152245044708252,
+      "learning_rate": 7.5847743094543e-05,
+      "loss": 0.1256,
+      "step": 2620
+    },
+    {
+      "epoch": 0.8416,
+      "grad_norm": 6.711794853210449,
+      "learning_rate": 7.573545924096114e-05,
+      "loss": 0.1184,
+      "step": 2630
+    },
+    {
+      "epoch": 0.8448,
+      "grad_norm": 6.55480432510376,
+      "learning_rate": 7.562317538737929e-05,
+      "loss": 0.1253,
+      "step": 2640
+    },
+    {
+      "epoch": 0.848,
+      "grad_norm": 6.035834312438965,
+      "learning_rate": 7.551089153379744e-05,
+      "loss": 0.125,
+      "step": 2650
+    },
+    {
+      "epoch": 0.8512,
+      "grad_norm": 7.503954887390137,
+      "learning_rate": 7.53986076802156e-05,
+      "loss": 0.1068,
+      "step": 2660
+    },
+    {
+      "epoch": 0.8544,
+      "grad_norm": 4.4410271644592285,
+      "learning_rate": 7.528632382663373e-05,
+      "loss": 0.1137,
+      "step": 2670
+    },
+    {
+      "epoch": 0.8576,
+      "grad_norm": 4.127198696136475,
+      "learning_rate": 7.517403997305188e-05,
+      "loss": 0.1127,
+      "step": 2680
+    },
+    {
+      "epoch": 0.8608,
+      "grad_norm": 3.0780539512634277,
+      "learning_rate": 7.506175611947003e-05,
+      "loss": 0.1009,
+      "step": 2690
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 8.409754753112793,
+      "learning_rate": 7.494947226588817e-05,
+      "loss": 0.1017,
+      "step": 2700
+    },
+    {
+      "epoch": 0.8672,
+      "grad_norm": 4.118874549865723,
+      "learning_rate": 7.483718841230631e-05,
+      "loss": 0.1,
+      "step": 2710
+    },
+    {
+      "epoch": 0.8704,
+      "grad_norm": 2.367293357849121,
+      "learning_rate": 7.472490455872446e-05,
+      "loss": 0.1019,
+      "step": 2720
+    },
+    {
+      "epoch": 0.8736,
+      "grad_norm": 5.165258884429932,
+      "learning_rate": 7.46126207051426e-05,
+      "loss": 0.0891,
+      "step": 2730
+    },
+    {
+      "epoch": 0.8768,
+      "grad_norm": 1.9673935174942017,
+      "learning_rate": 7.450033685156075e-05,
+      "loss": 0.0869,
+      "step": 2740
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 4.972908020019531,
+      "learning_rate": 7.43880529979789e-05,
+      "loss": 0.0894,
+      "step": 2750
+    },
+    {
+      "epoch": 0.8832,
+      "grad_norm": 4.128866195678711,
+      "learning_rate": 7.427576914439703e-05,
+      "loss": 0.0888,
+      "step": 2760
+    },
+    {
+      "epoch": 0.8864,
+      "grad_norm": 1.7406138181686401,
+      "learning_rate": 7.416348529081518e-05,
+      "loss": 0.0876,
+      "step": 2770
+    },
+    {
+      "epoch": 0.8896,
+      "grad_norm": 1.7218992710113525,
+      "learning_rate": 7.405120143723333e-05,
+      "loss": 0.0925,
+      "step": 2780
+    },
+    {
+      "epoch": 0.8928,
+      "grad_norm": 2.668508768081665,
+      "learning_rate": 7.393891758365149e-05,
+      "loss": 0.0872,
+      "step": 2790
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 5.266975402832031,
+      "learning_rate": 7.382663373006962e-05,
+      "loss": 0.083,
+      "step": 2800
+    },
+    {
+      "epoch": 0.8992,
+      "grad_norm": 2.1134185791015625,
+      "learning_rate": 7.371434987648776e-05,
+      "loss": 0.0871,
+      "step": 2810
+    },
+    {
+      "epoch": 0.9024,
+      "grad_norm": 4.195724010467529,
+      "learning_rate": 7.360206602290591e-05,
+      "loss": 0.0941,
+      "step": 2820
+    },
+    {
+      "epoch": 0.9056,
+      "grad_norm": 2.4259421825408936,
+      "learning_rate": 7.348978216932405e-05,
+      "loss": 0.0937,
+      "step": 2830
+    },
+    {
+      "epoch": 0.9088,
+      "grad_norm": 2.7671356201171875,
+      "learning_rate": 7.33774983157422e-05,
+      "loss": 0.095,
+      "step": 2840
+    },
+    {
+      "epoch": 0.912,
+      "grad_norm": 6.370889663696289,
+      "learning_rate": 7.326521446216035e-05,
+      "loss": 0.0826,
+      "step": 2850
+    },
+    {
+      "epoch": 0.9152,
+      "grad_norm": 6.082813739776611,
+      "learning_rate": 7.315293060857849e-05,
+      "loss": 0.0891,
+      "step": 2860
+    },
+    {
+      "epoch": 0.9184,
+      "grad_norm": 4.761292934417725,
+      "learning_rate": 7.304064675499664e-05,
+      "loss": 0.0905,
+      "step": 2870
+    },
+    {
+      "epoch": 0.9216,
+      "grad_norm": 3.1539108753204346,
+      "learning_rate": 7.292836290141479e-05,
+      "loss": 0.0836,
+      "step": 2880
+    },
+    {
+      "epoch": 0.9248,
+      "grad_norm": 3.095581531524658,
+      "learning_rate": 7.281607904783292e-05,
+      "loss": 0.0782,
+      "step": 2890
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 2.6302435398101807,
+      "learning_rate": 7.270379519425106e-05,
+      "loss": 0.0844,
+      "step": 2900
+    },
+    {
+      "epoch": 0.9312,
+      "grad_norm": 4.609825134277344,
+      "learning_rate": 7.259151134066921e-05,
+      "loss": 0.0773,
+      "step": 2910
+    },
+    {
+      "epoch": 0.9344,
+      "grad_norm": 6.823300361633301,
+      "learning_rate": 7.247922748708736e-05,
+      "loss": 0.0811,
+      "step": 2920
+    },
+    {
+      "epoch": 0.9376,
+      "grad_norm": 2.5662624835968018,
+      "learning_rate": 7.23669436335055e-05,
+      "loss": 0.0751,
+      "step": 2930
+    },
+    {
+      "epoch": 0.9408,
+      "grad_norm": 1.9695119857788086,
+      "learning_rate": 7.225465977992365e-05,
+      "loss": 0.087,
+      "step": 2940
+    },
+    {
+      "epoch": 0.944,
+      "grad_norm": 3.3225550651550293,
+      "learning_rate": 7.21423759263418e-05,
+      "loss": 0.0825,
+      "step": 2950
+    },
+    {
+      "epoch": 0.9472,
+      "grad_norm": 3.849437713623047,
+      "learning_rate": 7.203009207275994e-05,
+      "loss": 0.0767,
+      "step": 2960
+    },
+    {
+      "epoch": 0.9504,
+      "grad_norm": 1.750457525253296,
+      "learning_rate": 7.191780821917809e-05,
+      "loss": 0.0882,
+      "step": 2970
+    },
+    {
+      "epoch": 0.9536,
+      "grad_norm": 6.477813243865967,
+      "learning_rate": 7.180552436559623e-05,
+      "loss": 0.0919,
+      "step": 2980
+    },
+    {
+      "epoch": 0.9568,
+      "grad_norm": 1.4271601438522339,
+      "learning_rate": 7.169324051201438e-05,
+      "loss": 0.0804,
+      "step": 2990
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 7.061021327972412,
+      "learning_rate": 7.158095665843251e-05,
+      "loss": 0.0832,
+      "step": 3000
+    },
+    {
+      "epoch": 0.9632,
+      "grad_norm": 2.1753170490264893,
+      "learning_rate": 7.146867280485066e-05,
+      "loss": 0.0741,
+      "step": 3010
+    },
+    {
+      "epoch": 0.9664,
+      "grad_norm": 7.894692897796631,
+      "learning_rate": 7.135638895126882e-05,
+      "loss": 0.0723,
+      "step": 3020
+    },
+    {
+      "epoch": 0.9696,
+      "grad_norm": 2.4842376708984375,
+      "learning_rate": 7.124410509768695e-05,
+      "loss": 0.0746,
+      "step": 3030
+    },
+    {
+      "epoch": 0.9728,
+      "grad_norm": 4.693914413452148,
+      "learning_rate": 7.11318212441051e-05,
+      "loss": 0.0818,
+      "step": 3040
+    },
+    {
+      "epoch": 0.976,
+      "grad_norm": 3.4993011951446533,
+      "learning_rate": 7.101953739052325e-05,
+      "loss": 0.0743,
+      "step": 3050
+    },
+    {
+      "epoch": 0.9792,
+      "grad_norm": 2.4109108448028564,
+      "learning_rate": 7.090725353694139e-05,
+      "loss": 0.0698,
+      "step": 3060
+    },
+    {
+      "epoch": 0.9824,
+      "grad_norm": 3.363265037536621,
+      "learning_rate": 7.079496968335954e-05,
+      "loss": 0.076,
+      "step": 3070
+    },
+    {
+      "epoch": 0.9856,
+      "grad_norm": 4.7013959884643555,
+      "learning_rate": 7.068268582977768e-05,
+      "loss": 0.0801,
+      "step": 3080
+    },
+    {
+      "epoch": 0.9888,
+      "grad_norm": 1.858520269393921,
+      "learning_rate": 7.057040197619582e-05,
+      "loss": 0.0691,
+      "step": 3090
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 2.71395206451416,
+      "learning_rate": 7.045811812261397e-05,
+      "loss": 0.0811,
+      "step": 3100
+    },
+    {
+      "epoch": 0.9952,
+      "grad_norm": 3.254763603210449,
+      "learning_rate": 7.034583426903212e-05,
+      "loss": 0.0772,
+      "step": 3110
+    },
+    {
+      "epoch": 0.9984,
+      "grad_norm": 6.519776821136475,
+      "learning_rate": 7.023355041545027e-05,
+      "loss": 0.0718,
+      "step": 3120
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 9375,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-3125/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97555f8ce3aa7913ee05adca8d58969542dca6efa4c6dcd6a6ea25e10d7fbbd0
+size 5304

checkpoint-6250/README.md

@@ -0,0 +1,202 @@
+---
+base_model: gpt2-medium
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

checkpoint-6250/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GPT2LMHeadModel",
+    "parent_library": "transformers.models.gpt2.modeling_gpt2"
+  },
+  "base_model_name_or_path": "gpt2-medium",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-6250/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:689c64ef188fa74751e49b80968470e493c5588180844da5342020e2265c2c8a
+size 25172088

checkpoint-6250/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40b6d520a4dfbd55f460e5a668a9110172c1747f47a389e7f0b4e99302f3d2cf
+size 50372538

checkpoint-6250/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52fefebcb3a72d7a7a7ad505177d152a6c69b3050bea80f92945b4d318c2414f
+size 14244

checkpoint-6250/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54bf5bc35983da2516a319daa8db116d3e18d36fcecad8a541a6fed01b6c2f20
+size 1064

checkpoint-6250/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-6250/tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-6250/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97555f8ce3aa7913ee05adca8d58969542dca6efa4c6dcd6a6ea25e10d7fbbd0
+size 5304

checkpoint-9375/README.md

@@ -0,0 +1,202 @@
+---
+base_model: gpt2-medium
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

checkpoint-9375/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GPT2LMHeadModel",
+    "parent_library": "transformers.models.gpt2.modeling_gpt2"
+  },
+  "base_model_name_or_path": "gpt2-medium",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-9375/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d1652c7a003dd8627cd4a9648e12d63c6d4c984d8c61b0fca89d4f8de2daeaf
+size 25172088

checkpoint-9375/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdbcab328afafac35d79fcaa42038bc7602d341850862f4f9908d9d0fa0e6547
+size 50372538

checkpoint-9375/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50fdfeef75a872ba1bc02c850c42b1db8abc27c7b8844c939df6c63695af50cd
+size 14244

checkpoint-9375/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a17c6f8f90b7492d85e3ccb69e5e19d7c97f2aa89c2af5ae7acd1722212a18fd
+size 1064

checkpoint-9375/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-9375/tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-9375/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97555f8ce3aa7913ee05adca8d58969542dca6efa4c6dcd6a6ea25e10d7fbbd0
+size 5304

config.json

@@ -0,0 +1,40 @@
+{
+  "_name_or_path": "gpt2-medium",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 1024,
+  "n_head": 16,
+  "n_inner": null,
+  "n_layer": 24,
+  "n_positions": 1024,
+  "n_special": 0,
+  "predict_special_tokens": true,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "transformers_version": "4.45.2",
+  "use_cache": true,
+  "vocab_size": 50257
+}

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_logs.csv

@@ -0,0 +1,939 @@
+step,training_loss,grad_norm
+10,25.0099,10.000000953674316
+20,21.1499,10.0
+30,19.6111,9.999999046325684
+40,22.3722,10.0
+50,20.1462,9.999999046325684
+60,20.4235,10.0
+70,20.7853,10.0
+80,19.5459,10.0
+90,18.4778,9.999999046325684
+100,16.2191,9.999999046325684
+110,18.5773,10.0
+120,15.9389,9.999999046325684
+130,15.6064,10.0
+140,15.6366,10.000000953674316
+150,16.7232,10.000000953674316
+160,13.2957,10.0
+170,13.3429,10.0
+180,13.3301,9.999999046325684
+190,15.1708,10.000000953674316
+200,14.8533,10.0
+210,11.6389,10.0
+220,12.2515,9.999999046325684
+230,12.6637,10.0
+240,10.6091,9.999999046325684
+250,10.9362,9.999999046325684
+260,12.4545,10.0
+270,11.5238,10.000000953674316
+280,10.8966,10.0
+290,10.5903,10.0
+300,10.4872,10.0
+310,9.9663,10.0
+320,9.4655,9.999999046325684
+330,10.487,10.0
+340,9.3626,10.000001907348633
+350,9.864,10.0
+360,8.8008,9.999999046325684
+370,9.7772,10.0
+380,8.7418,10.0
+390,9.1157,9.999999046325684
+400,8.1395,10.000000953674316
+410,8.8305,10.0
+420,9.2089,10.0
+430,8.5206,10.000000953674316
+440,8.8993,9.999999046325684
+450,8.8173,10.0
+460,8.3408,10.000000953674316
+470,7.5853,10.0
+480,7.4544,10.0
+490,8.0499,10.000000953674316
+500,8.2335,10.0
+510,7.0574,10.0
+520,6.8412,10.0
+530,7.1418,10.0
+540,7.1722,10.0
+550,6.7641,10.0
+560,6.3646,10.0
+570,6.3469,10.0
+580,6.38,10.0
+590,5.8154,10.000000953674316
+600,5.6794,10.000000953674316
+610,5.4415,9.999999046325684
+620,4.7814,10.0
+630,4.8304,10.000000953674316
+640,4.9017,10.0
+650,4.5826,10.0
+660,4.4032,10.0
+670,4.4789,10.0
+680,3.5318,10.000000953674316
+690,3.9803,10.000000953674316
+700,3.3411,10.0
+710,3.8421,10.0
+720,3.8598,10.000000953674316
+730,3.6175,9.999999046325684
+740,3.3833,9.999999046325684
+750,3.1501,10.0
+760,3.2144,10.0
+770,3.1372,10.0
+780,2.728,10.0
+790,2.7485,10.0
+800,2.7431,10.000000953674316
+810,2.534,10.0
+820,2.6129,10.0
+830,2.6607,10.0
+840,2.5158,9.999999046325684
+850,2.2633,9.999999046325684
+860,2.4407,10.0
+870,2.1584,9.999999046325684
+880,2.1552,10.0
+890,1.8968,10.000000953674316
+900,1.9182,9.999999046325684
+910,1.9413,10.0
+920,1.8959,10.0
+930,1.9225,10.0
+940,1.8802,10.0
+950,1.8896,9.999999046325684
+960,1.7142,10.0
+970,1.6888,10.0
+980,1.6177,10.0
+990,1.645,10.0
+1000,1.6792,10.0
+1010,1.8599,10.0
+1020,1.6779,10.0
+1030,1.4669,9.999999046325684
+1040,1.3649,10.0
+1050,1.3987,10.0
+1060,1.2421,9.999998092651367
+1070,1.3011,9.999998092651367
+1080,1.2287,9.999999046325684
+1090,1.1587,10.0
+1100,1.2415,10.0
+1110,1.1415,10.0
+1120,1.1941,9.999998092651367
+1130,1.1668,10.0
+1140,1.0887,10.000000953674316
+1150,1.0604,9.999998092651367
+1160,0.9446,9.999999046325684
+1170,0.9737,9.999999046325684
+1180,0.9654,10.0
+1190,1.0253,10.000000953674316
+1200,0.8597,9.999999046325684
+1210,0.8202,9.999998092651367
+1220,0.8386,9.999999046325684
+1230,0.8318,9.999999046325684
+1240,0.8571,10.0
+1250,0.8326,9.999999046325684
+1260,0.7655,9.999999046325684
+1270,0.747,10.0
+1280,0.8671,10.0
+1290,0.8758,9.999999046325684
+1300,0.8714,10.000000953674316
+1310,0.8022,9.999999046325684
+1320,0.7234,8.615378379821777
+1330,0.7416,10.0
+1340,0.6898,10.0
+1350,0.6261,9.999999046325684
+1360,0.6267,10.0
+1370,0.6171,9.999999046325684
+1380,0.5677,9.999998092651367
+1390,0.5227,9.999999046325684
+1400,0.5775,9.999999046325684
+1410,0.6166,9.999998092651367
+1420,0.6165,10.0
+1430,0.5849,9.999999046325684
+1440,0.6439,10.0
+1450,0.5886,9.999998092651367
+1460,0.5935,9.999999046325684
+1470,0.5868,10.0
+1480,0.5489,9.999998092651367
+1490,0.5106,10.0
+1500,0.4885,9.999999046325684
+1510,0.5323,9.999999046325684
+1520,0.4359,9.999999046325684
+1530,0.4789,7.423900604248047
+1540,0.4367,9.999998092651367
+1550,0.465,7.3594489097595215
+1560,0.4244,9.999998092651367
+1570,0.4311,10.0
+1580,0.4787,9.999999046325684
+1590,0.4777,9.999999046325684
+1600,0.469,10.0
+1610,0.4519,10.0
+1620,0.4084,9.999998092651367
+1630,0.3627,9.999998092651367
+1640,0.3742,8.502776145935059
+1650,0.3525,10.0
+1660,0.3889,9.999999046325684
+1670,0.5584,10.0
+1680,0.3988,9.999999046325684
+1690,0.3439,8.782861709594727
+1700,0.3265,8.174032211303711
+1710,0.329,4.737403392791748
+1720,0.3093,9.999999046325684
+1730,0.3119,10.0
+1740,0.3227,10.0
+1750,0.2979,9.882418632507324
+1760,0.2693,9.999998092651367
+1770,0.3187,9.999999046325684
+1780,0.291,9.999999046325684
+1790,0.2833,7.839264869689941
+1800,0.2525,4.611258029937744
+1810,0.2727,4.125186443328857
+1820,0.2718,9.999999046325684
+1830,0.2787,9.999999046325684
+1840,0.2903,4.507540225982666
+1850,0.2821,8.763723373413086
+1860,0.2469,6.04974365234375
+1870,0.2389,9.999998092651367
+1880,0.2749,10.0
+1890,0.2829,10.0
+1900,0.2725,9.99999713897705
+1910,0.2698,10.000001907348633
+1920,0.2678,9.999999046325684
+1930,0.2741,10.0
+1940,0.2475,10.0
+1950,0.2711,9.999999046325684
+1960,0.2592,10.0
+1970,0.2348,9.999999046325684
+1980,0.2185,7.087158679962158
+1990,0.2443,6.557620048522949
+2000,0.2012,9.999999046325684
+2010,0.1992,8.828990936279297
+2020,0.2005,9.999998092651367
+2030,0.2244,10.0
+2040,0.3351,10.0
+2050,0.3391,10.0
+2060,0.2319,9.999998092651367
+2070,0.2222,10.0
+2080,0.1829,8.043660163879395
+2090,0.1756,5.343007564544678
+2100,0.1959,3.530430316925049
+2110,0.178,9.999999046325684
+2120,0.1867,10.0
+2130,0.1832,8.584357261657715
+2140,0.1771,4.891357421875
+2150,0.1527,5.522593021392822
+2160,0.1769,9.386947631835938
+2170,0.1889,10.0
+2180,0.1827,9.999999046325684
+2190,0.2055,9.999999046325684
+2200,0.1802,9.999998092651367
+2210,0.1706,9.999999046325684
+2220,0.1604,3.970720052719116
+2230,0.1453,3.6069769859313965
+2240,0.1516,3.9357783794403076
+2250,0.1512,5.958951950073242
+2260,0.1642,9.320231437683105
+2270,0.1613,4.681884765625
+2280,0.1544,7.769372940063477
+2290,0.1636,4.547751426696777
+2300,0.1372,8.214717864990234
+2310,0.158,7.887718677520752
+2320,0.1611,9.999999046325684
+2330,0.1584,3.0139806270599365
+2340,0.1489,5.308823585510254
+2350,0.1335,1.894464373588562
+2360,0.1363,8.55656909942627
+2370,0.1398,7.630896091461182
+2380,0.1307,9.361071586608887
+2390,0.1461,3.613555431365967
+2400,0.1432,4.020608901977539
+2410,0.1396,6.401773452758789
+2420,0.1355,6.332232475280762
+2430,0.1416,2.399341344833374
+2440,0.1265,5.8365654945373535
+2450,0.6647,10.0
+2460,0.1334,5.435879230499268
+2470,0.1284,10.0
+2480,0.1369,7.202799320220947
+2490,0.1105,2.0989317893981934
+2500,0.1226,4.924466609954834
+2510,0.1089,7.978616714477539
+2520,0.1234,4.150684356689453
+2530,0.1188,4.818097114562988
+2540,0.113,2.5419697761535645
+2550,0.109,5.721562385559082
+2560,0.101,2.7478086948394775
+2570,0.1156,9.692163467407227
+2580,0.12,6.791487693786621
+2590,0.114,4.655740261077881
+2600,0.102,2.558154344558716
+2610,0.156,5.348006725311279
+2620,0.1256,4.152245044708252
+2630,0.1184,6.711794853210449
+2640,0.1253,6.55480432510376
+2650,0.125,6.035834312438965
+2660,0.1068,7.503954887390137
+2670,0.1137,4.4410271644592285
+2680,0.1127,4.127198696136475
+2690,0.1009,3.0780539512634277
+2700,0.1017,8.409754753112793
+2710,0.1,4.118874549865723
+2720,0.1019,2.367293357849121
+2730,0.0891,5.165258884429932
+2740,0.0869,1.9673935174942017
+2750,0.0894,4.972908020019531
+2760,0.0888,4.128866195678711
+2770,0.0876,1.7406138181686401
+2780,0.0925,1.7218992710113525
+2790,0.0872,2.668508768081665
+2800,0.083,5.266975402832031
+2810,0.0871,2.1134185791015625
+2820,0.0941,4.195724010467529
+2830,0.0937,2.4259421825408936
+2840,0.095,2.7671356201171875
+2850,0.0826,6.370889663696289
+2860,0.0891,6.082813739776611
+2870,0.0905,4.761292934417725
+2880,0.0836,3.1539108753204346
+2890,0.0782,3.095581531524658
+2900,0.0844,2.6302435398101807
+2910,0.0773,4.609825134277344
+2920,0.0811,6.823300361633301
+2930,0.0751,2.5662624835968018
+2940,0.087,1.9695119857788086
+2950,0.0825,3.3225550651550293
+2960,0.0767,3.849437713623047
+2970,0.0882,1.750457525253296
+2980,0.0919,6.477813243865967
+2990,0.0804,1.4271601438522339
+3000,0.0832,7.061021327972412
+3010,0.0741,2.1753170490264893
+3020,0.0723,7.894692897796631
+3030,0.0746,2.4842376708984375
+3040,0.0818,4.693914413452148
+3050,0.0743,3.4993011951446533
+3060,0.0698,2.4109108448028564
+3070,0.076,3.363265037536621
+3080,0.0801,4.7013959884643555
+3090,0.0691,1.858520269393921
+3100,0.0811,2.71395206451416
+3110,0.0772,3.254763603210449
+3120,0.0718,6.519776821136475
+3130,0.0746,1.91741144657135
+3140,0.071,1.5471233129501343
+3150,0.0646,3.8411548137664795
+3160,0.068,4.6282267570495605
+3170,0.0706,3.1158607006073
+3180,0.0711,4.9777913093566895
+3190,0.0649,2.6763927936553955
+3200,0.066,4.165836334228516
+3210,0.068,9.641294479370117
+3220,0.0695,2.3524587154388428
+3230,0.0869,3.930358648300171
+3240,0.0734,3.403099298477173
+3250,0.0729,4.471584796905518
+3260,0.0649,2.041577100753784
+3270,0.068,4.053502559661865
+3280,0.0693,1.712865948677063
+3290,0.073,8.817410469055176
+3300,0.0665,1.9689174890518188
+3310,0.0683,4.147378921508789
+3320,0.0635,3.4082727432250977
+3330,0.0587,1.8768893480300903
+3340,0.0628,1.7891249656677246
+3350,0.059,1.5056793689727783
+3360,0.0642,2.3811066150665283
+3370,0.0573,4.5819902420043945
+3380,0.0926,3.9442105293273926
+3390,0.0765,4.184739112854004
+3400,0.0657,2.208756446838379
+3410,0.0584,1.9726155996322632
+3420,0.0663,2.3560941219329834
+3430,0.0652,4.31378173828125
+3440,0.0619,5.586067199707031
+3450,0.0653,4.057238578796387
+3460,0.0647,2.5588834285736084
+3470,0.0604,3.098489761352539
+3480,0.0665,3.6659464836120605
+3490,0.059,2.530979633331299
+3500,0.062,4.6189374923706055
+3510,0.0573,2.125894784927368
+3520,0.0588,3.884368896484375
+3530,0.0648,4.213693141937256
+3540,0.0585,4.863856315612793
+3550,0.0584,2.9554522037506104
+3560,0.0611,1.8812214136123657
+3570,0.0591,2.52107572555542
+3580,0.0622,3.270277500152588
+3590,0.0586,2.0023467540740967
+3600,0.0517,2.0342307090759277
+3610,0.0568,3.3358206748962402
+3620,0.0509,1.158366322517395
+3630,0.0642,1.7512037754058838
+3640,0.0559,1.3442176580429077
+3650,0.0565,4.689185619354248
+3660,0.0575,3.3459503650665283
+3670,0.0534,2.5210814476013184
+3680,0.0562,1.4273737668991089
+3690,0.0531,3.007507562637329
+3700,0.0521,2.328012228012085
+3710,0.0586,3.815755844116211
+3720,0.0559,2.570431709289551
+3730,0.0555,2.9417104721069336
+3740,0.0515,1.5327764749526978
+3750,0.054,3.548718214035034
+3760,0.0552,1.7067848443984985
+3770,0.0459,1.4624521732330322
+3780,0.0459,4.673206329345703
+3790,0.0515,2.1773905754089355
+3800,0.0623,4.247639179229736
+3810,17.18,2.148364543914795
+3820,0.0535,3.752838134765625
+3830,0.0552,3.2757108211517334
+3840,0.0495,2.085975408554077
+3850,0.0468,1.4556792974472046
+3860,0.0583,1.6175802946090698
+3870,0.0486,1.6184957027435303
+3880,0.0778,9.999999046325684
+3890,0.0771,7.507689952850342
+3900,0.1067,4.089766502380371
+3910,0.069,2.4886679649353027
+3920,0.0643,3.568296432495117
+3930,0.0568,2.1102867126464844
+3940,0.0591,1.3482815027236938
+3950,0.0576,2.092593193054199
+3960,0.0484,2.106621503829956
+3970,0.0433,2.036658525466919
+3980,0.0449,1.402249813079834
+3990,0.0482,2.1640737056732178
+4000,0.0463,2.2044899463653564
+4010,0.0549,7.269384384155273
+4020,0.0522,3.0673646926879883
+4030,0.0471,5.095019340515137
+4040,0.0483,3.6901588439941406
+4050,0.0456,1.2914890050888062
+4060,0.0475,2.4384307861328125
+4070,0.0456,1.596550703048706
+4080,0.0517,1.6937170028686523
+4090,0.0437,1.587768316268921
+4100,0.0457,4.193778991699219
+4110,0.0479,2.494598865509033
+4120,0.0433,2.869279623031616
+4130,0.0519,4.543278217315674
+4140,0.0492,1.5083669424057007
+4150,0.0426,2.1934690475463867
+4160,0.0441,2.10366153717041
+4170,0.0437,1.6721271276474
+4180,0.0432,4.587328910827637
+4190,0.053,2.2152934074401855
+4200,0.05,3.132293224334717
+4210,0.0408,3.02851939201355
+4220,0.046,2.1754748821258545
+4230,0.0428,2.0088369846343994
+4240,0.0424,2.7489728927612305
+4250,0.0448,1.6788060665130615
+4260,0.041,2.4949731826782227
+4270,0.0441,1.9527579545974731
+4280,0.043,2.471604108810425
+4290,0.0562,4.944948196411133
+4300,0.0469,2.4510498046875
+4310,0.0428,1.1143766641616821
+4320,0.0448,1.719165325164795
+4330,0.0417,2.277461528778076
+4340,0.038,3.449477195739746
+4350,0.0396,2.189485549926758
+4360,0.0398,1.732558012008667
+4370,0.0433,2.732851266860962
+4380,0.0361,1.4708316326141357
+4390,0.0438,2.114934206008911
+4400,0.0421,4.473976135253906
+4410,0.0437,1.1676849126815796
+4420,0.0351,2.1100847721099854
+4430,0.0456,2.426988124847412
+4440,0.0381,3.200514078140259
+4450,0.0414,1.525909423828125
+4460,0.0407,1.2169736623764038
+4470,0.0476,4.590202808380127
+4480,0.052,1.4722408056259155
+4490,0.0459,1.351094126701355
+4500,0.0446,1.2844514846801758
+4510,0.0392,0.783967912197113
+4520,0.0407,2.0353965759277344
+4530,0.0381,1.3156319856643677
+4540,0.0353,1.4378070831298828
+4550,0.0348,3.0744736194610596
+4560,0.0402,1.6298600435256958
+4570,0.0365,2.6494648456573486
+4580,0.0363,1.653573751449585
+4590,0.0393,1.7645295858383179
+4600,0.0406,4.306888580322266
+4610,0.0358,3.1965126991271973
+4620,0.0396,1.2502505779266357
+4630,0.035,2.091470718383789
+4640,0.0428,1.1398119926452637
+4650,0.0353,2.7160258293151855
+4660,0.0415,0.9645196199417114
+4670,0.036,4.158140182495117
+4680,0.0376,1.1706950664520264
+4690,0.0374,1.1529676914215088
+4700,0.037,1.944616675376892
+4710,0.0376,1.7939577102661133
+4720,0.0345,2.588820695877075
+4730,0.0355,1.099802017211914
+4740,0.036,1.9600342512130737
+4750,0.0413,3.2066643238067627
+4760,0.036,4.0401811599731445
+4770,0.0366,1.673994541168213
+4780,0.0375,4.127200126647949
+4790,0.0364,2.1488308906555176
+4800,0.0366,3.2764744758605957
+4810,0.0387,4.183980941772461
+4820,0.0376,2.350937604904175
+4830,0.0378,1.4431155920028687
+4840,0.0352,2.455113172531128
+4850,0.0388,2.4375059604644775
+4860,0.0352,1.8249945640563965
+4870,0.0366,2.562242031097412
+4880,0.0381,1.7993731498718262
+4890,0.0342,1.6024712324142456
+4900,0.0355,1.0818573236465454
+4910,0.0381,0.9753634929656982
+4920,0.0341,1.9120144844055176
+4930,0.0377,2.293750762939453
+4940,0.0343,3.1085097789764404
+4950,0.036,1.3106160163879395
+4960,0.0341,2.5809476375579834
+4970,0.0306,1.423450231552124
+4980,0.0339,3.155231475830078
+4990,0.0369,1.6214268207550049
+5000,0.0398,3.8274025917053223
+5010,0.0347,3.3754544258117676
+5020,0.0335,2.3379604816436768
+5030,0.0327,1.3889174461364746
+5040,0.0362,2.985076427459717
+5050,0.0365,1.5555957555770874
+5060,0.0384,2.723813533782959
+5070,0.0336,1.3659981489181519
+5080,0.0307,3.0281941890716553
+5090,0.0326,1.122520089149475
+5100,0.0331,2.31559681892395
+5110,0.0375,1.262046217918396
+5120,0.0344,2.673083782196045
+5130,0.0434,5.857226371765137
+5140,0.0477,2.3795230388641357
+5150,0.0524,2.1502673625946045
+5160,0.0429,1.4233574867248535
+5170,0.0345,2.0955042839050293
+5180,0.0351,2.341566562652588
+5190,0.0344,3.160850763320923
+5200,0.0331,2.272794485092163
+5210,0.0335,1.2717911005020142
+5220,0.0319,2.4570579528808594
+5230,0.0343,1.8033403158187866
+5240,0.032,1.5098129510879517
+5250,0.0327,1.48603355884552
+5260,0.0353,1.4422051906585693
+5270,0.0321,1.18756103515625
+5280,0.0352,1.1758790016174316
+5290,0.03,1.2466790676116943
+5300,0.0314,1.9264726638793945
+5310,0.0328,1.9639238119125366
+5320,0.0347,2.070943832397461
+5330,0.0333,2.0837361812591553
+5340,0.0323,1.9007681608200073
+5350,0.0341,4.6805830001831055
+5360,0.0288,1.4064592123031616
+5370,0.0303,3.4108219146728516
+5380,0.0314,0.9329103827476501
+5390,0.0284,1.681719183921814
+5400,0.0299,0.932102382183075
+5410,0.0329,2.7038533687591553
+5420,0.0303,1.4268467426300049
+5430,0.0326,1.9577819108963013
+5440,0.0325,2.5221760272979736
+5450,0.0295,3.2790379524230957
+5460,0.0307,0.8850557208061218
+5470,0.0289,1.8326717615127563
+5480,0.0292,1.3088713884353638
+5490,0.0311,1.0117987394332886
+5500,0.0278,2.7869484424591064
+5510,0.0278,1.3128660917282104
+5520,0.03,1.1051273345947266
+5530,0.0306,1.3793776035308838
+5540,0.0299,2.992269515991211
+5550,0.0356,1.6625593900680542
+5560,0.0349,1.0821179151535034
+5570,0.0313,1.0884099006652832
+5580,0.0306,1.5417124032974243
+5590,0.0329,2.5081472396850586
+5600,0.0331,0.9784805178642273
+5610,0.0295,2.7062466144561768
+5620,0.0294,2.028097629547119
+5630,0.0282,0.9252501130104065
+5640,0.029,1.6614617109298706
+5650,0.0286,1.5624871253967285
+5660,0.0299,1.7454490661621094
+5670,0.0296,1.7463639974594116
+5680,0.0317,1.4736864566802979
+5690,0.0293,1.21503484249115
+5700,0.0305,3.489445209503174
+5710,0.03,1.21157705783844
+5720,0.0288,2.4874978065490723
+5730,0.0354,1.4586814641952515
+5740,0.0345,1.2466408014297485
+5750,0.0281,1.0634334087371826
+5760,0.0311,1.6378570795059204
+5770,0.0298,1.5089457035064697
+5780,0.029,2.608259677886963
+5790,0.0268,1.7564635276794434
+5800,0.0274,1.1659057140350342
+5810,0.0272,2.8701725006103516
+5820,0.0279,1.034717321395874
+5830,0.0307,1.3043538331985474
+5840,0.0258,2.0375423431396484
+5850,0.0308,2.1126816272735596
+5860,0.0282,1.7014706134796143
+5870,0.0284,2.7476634979248047
+5880,0.0242,1.6362510919570923
+5890,0.0361,1.3239845037460327
+5900,0.0277,1.4442349672317505
+5910,0.0258,1.9677876234054565
+5920,0.0298,2.842235803604126
+5930,0.0292,1.5647774934768677
+5940,0.0357,1.7668209075927734
+5950,0.0302,2.651283025741577
+5960,0.0304,5.3611040115356445
+5970,0.0284,5.584418773651123
+5980,0.0297,3.7711610794067383
+5990,0.0289,0.7803456783294678
+6000,0.0247,1.1393859386444092
+6010,0.0276,1.7076109647750854
+6020,0.0277,1.2361043691635132
+6030,0.0279,1.1116836071014404
+6040,0.0284,2.2229862213134766
+6050,0.0306,1.5508464574813843
+6060,0.0308,1.7325248718261719
+6070,0.0261,1.0399812459945679
+6080,0.0297,2.994081974029541
+6090,0.0276,2.2742340564727783
+6100,0.0277,1.2096333503723145
+6110,0.0289,1.3712992668151855
+6120,0.0273,1.6062151193618774
+6130,0.0297,1.5088088512420654
+6140,0.026,0.9999478459358215
+6150,0.0285,1.0416607856750488
+6160,0.0259,1.9191815853118896
+6170,0.0259,1.457220196723938
+6180,0.0264,1.0753467082977295
+6190,0.0258,2.804377317428589
+6200,0.0282,1.9763532876968384
+6210,0.0244,2.1442463397979736
+6220,0.0251,1.6756515502929688
+6230,0.0276,2.3364181518554688
+6240,0.0266,3.023191452026367
+6250,0.0273,2.4868950843811035
+6260,0.027,2.500319480895996
+6270,0.0248,1.7919338941574097
+6280,0.0342,0.9290552139282227
+6290,0.0262,1.057898759841919
+6300,0.0262,1.664808750152588
+6310,0.0265,2.4442942142486572
+6320,0.0255,2.3966939449310303
+6330,0.0263,2.9697370529174805
+6340,0.0256,2.686727285385132
+6350,0.0238,1.7658582925796509
+6360,0.0227,0.8644446730613708
+6370,0.0441,10.0
+6380,0.0367,2.123744249343872
+6390,0.0335,5.936791896820068
+6400,0.0298,2.3005146980285645
+6410,0.0288,1.058663010597229
+6420,0.028,2.2832794189453125
+6430,0.024,1.0518248081207275
+6440,0.0253,0.7201146483421326
+6450,0.0246,1.5872597694396973
+6460,0.0267,1.0472866296768188
+6470,0.0244,1.5414655208587646
+6480,0.0232,1.91981840133667
+6490,0.0247,0.8029799461364746
+6500,0.0254,1.7385778427124023
+6510,0.0257,1.1203155517578125
+6520,0.0223,0.8405721783638
+6530,0.0241,0.8406912088394165
+6540,0.0228,1.0132052898406982
+6550,0.0266,1.5743025541305542
+6560,0.0221,2.5406606197357178
+6570,0.0241,1.8731350898742676
+6580,0.0235,1.8953135013580322
+6590,0.03,1.2279338836669922
+6600,0.0238,2.3483939170837402
+6610,0.0284,1.5353964567184448
+6620,0.0254,0.7940895557403564
+6630,0.0268,0.8867064118385315
+6640,0.0229,0.6823979020118713
+6650,0.0221,0.9054269790649414
+6660,0.0242,0.844618558883667
+6670,0.0229,1.4453538656234741
+6680,0.0223,1.9151674509048462
+6690,0.0245,0.8978673219680786
+6700,0.0237,1.192948341369629
+6710,0.0233,0.9310412406921387
+6720,0.0236,3.681347608566284
+6730,0.0209,1.9511719942092896
+6740,0.0233,0.8533463478088379
+6750,0.024,1.8927245140075684
+6760,0.0254,1.9887471199035645
+6770,0.0233,0.9910705089569092
+6780,0.0276,0.9697316884994507
+6790,0.0221,1.3073784112930298
+6800,0.0258,2.332897663116455
+6810,0.024,2.5246870517730713
+6820,0.0225,0.9514437317848206
+6830,0.0265,0.9734804630279541
+6840,0.0225,0.7412808537483215
+6850,0.0218,0.6654082536697388
+6860,0.0239,1.154433250427246
+6870,0.0227,2.431039571762085
+6880,0.0203,0.7000795602798462
+6890,0.0235,0.9966208338737488
+6900,0.0224,0.5826752185821533
+6910,0.0196,1.576804280281067
+6920,0.0204,0.846698522567749
+6930,0.022,1.1780568361282349
+6940,0.0227,1.391042709350586
+6950,0.0218,0.8042160868644714
+6960,0.02,1.000321388244629
+6970,0.0216,0.9171059131622314
+6980,0.0213,1.0661230087280273
+6990,0.0204,1.4091787338256836
+7000,0.0225,1.5097310543060303
+7010,0.0217,2.1942641735076904
+7020,0.0225,2.1799325942993164
+7030,0.0235,0.8118078708648682
+7040,0.022,0.6434078216552734
+7050,0.0215,2.233530282974243
+7060,0.0234,2.489169120788574
+7070,0.022,1.5902799367904663
+7080,0.0255,2.1637775897979736
+7090,0.0235,0.90614914894104
+7100,0.0233,2.2505555152893066
+7110,0.022,1.0614933967590332
+7120,0.0189,1.5377411842346191
+7130,0.0217,2.9292163848876953
+7140,0.0194,0.55870121717453
+7150,0.025,2.509446859359741
+7160,0.0215,2.2066006660461426
+7170,0.0224,1.0167932510375977
+7180,0.0227,1.1303454637527466
+7190,0.0264,1.1977907419204712
+7200,0.0232,1.1619302034378052
+7210,0.0213,0.7383434772491455
+7220,0.0229,1.3148552179336548
+7230,0.0244,3.0743839740753174
+7240,0.0209,2.406111240386963
+7250,0.0283,1.536865472793579
+7260,0.0226,0.5282655954360962
+7270,0.021,1.2603291273117065
+7280,0.0207,0.9980093240737915
+7290,0.0238,1.0919678211212158
+7300,0.0202,0.9208861589431763
+7310,0.0204,1.6525688171386719
+7320,0.0192,1.4176781177520752
+7330,0.0206,1.4673593044281006
+7340,0.0218,0.8493990302085876
+7350,0.0191,1.3118574619293213
+7360,0.0194,0.7866161465644836
+7370,0.0228,0.8892452120780945
+7380,0.0219,2.4955861568450928
+7390,0.0204,1.814651608467102
+7400,0.0228,1.1132270097732544
+7410,0.0208,0.7917268872261047
+7420,0.0225,1.779470443725586
+7430,0.0209,1.540031909942627
+7440,0.0201,0.795350193977356
+7450,0.0203,1.4017839431762695
+7460,0.0223,1.2247642278671265
+7470,0.0218,1.8473584651947021
+7480,0.0228,1.563549518585205
+7490,0.0184,0.9918416142463684
+7500,0.02,0.7983741164207458
+7510,0.0189,1.003884196281433
+7520,0.0187,2.06343150138855
+7530,0.0209,0.8900611400604248
+7540,0.0209,1.086491346359253
+7550,0.025,1.247149109840393
+7560,0.0189,1.2688690423965454
+7570,0.0267,0.8303091526031494
+7580,0.0205,1.7538371086120605
+7590,0.0223,0.9375876188278198
+7600,0.0203,1.3351136445999146
+7610,0.0193,0.670791506767273
+7620,0.0207,0.6955625414848328
+7630,0.0197,0.8708732724189758
+7640,0.1552,2.0503456592559814
+7650,0.0256,2.0993776321411133
+7660,0.0259,1.001733660697937
+7670,0.0271,1.1097873449325562
+7680,0.0234,1.4637205600738525
+7690,0.0224,0.7054375410079956
+7700,0.0198,1.0400899648666382
+7710,0.0239,1.1329540014266968
+7720,0.0225,1.9706392288208008
+7730,0.0227,0.9307739734649658
+7740,0.0197,1.021101951599121
+7750,0.022,0.6976405382156372
+7760,0.0209,1.2085492610931396
+7770,0.0202,0.9504523277282715
+7780,0.0191,1.4022413492202759
+7790,0.0195,0.7028554677963257
+7800,0.024,0.8556867241859436
+7810,0.0206,1.331692099571228
+7820,0.3014,3.2675046920776367
+7830,0.0234,1.7758318185806274
+7840,0.0225,1.8547520637512207
+7850,0.0208,0.6967349648475647
+7860,0.0198,1.194005012512207
+7870,0.0201,1.7068527936935425
+7880,0.0239,1.449600100517273
+7890,0.0225,1.0100475549697876
+7900,0.0215,0.9704424142837524
+7910,0.0192,0.7141832113265991
+7920,0.02,1.8738646507263184
+7930,0.0205,0.9470159411430359
+7940,0.0193,1.2336801290512085
+7950,0.0202,0.9015800952911377
+7960,0.0234,0.7350621223449707
+7970,0.0207,1.2060322761535645
+7980,0.0188,0.6335831880569458
+7990,0.0195,1.0557260513305664
+8000,0.0182,0.5716488361358643
+8010,0.0204,0.75430828332901
+8020,0.0226,0.7589021921157837
+8030,0.0197,0.9131786227226257
+8040,0.02,0.7432297468185425
+8050,0.0182,0.7287900447845459
+8060,0.0201,0.7430145740509033
+8070,0.0188,0.5040020942687988
+8080,0.0171,0.6938813328742981
+8090,0.0201,0.7587171792984009
+8100,0.0196,1.0006415843963623
+8110,0.0179,0.6382442116737366
+8120,0.0177,0.9802420735359192
+8130,0.0201,1.5089691877365112
+8140,0.0205,0.8414797782897949
+8150,0.0209,0.9343305230140686
+8160,0.0194,0.7130104899406433
+8170,0.0187,1.5550020933151245
+8180,0.0208,1.134194254875183
+8190,0.0194,0.9144425392150879
+8200,0.0177,1.6265108585357666
+8210,0.0183,1.8325061798095703
+8220,0.0185,1.106653094291687
+8230,0.0186,1.0227482318878174
+8240,0.0182,0.8500052094459534
+8250,0.0182,0.670935332775116
+8260,0.0205,1.164656400680542
+8270,0.0182,1.3024622201919556
+8280,0.0232,0.7322014570236206
+8290,0.0183,0.7198416590690613
+8300,0.0184,0.5351381897926331
+8310,0.0224,0.6597144603729248
+8320,0.0183,0.9235522747039795
+8330,0.0176,0.5945689082145691
+8340,0.0165,0.6714538335800171
+8350,0.0184,0.5681155323982239
+8360,0.0176,1.2865463495254517
+8370,0.0205,0.675859808921814
+8380,0.0183,0.7957953810691833
+8390,0.0188,0.9204811453819275
+8400,0.0175,1.300011157989502
+8410,0.0199,1.3987044095993042
+8420,0.0213,0.7248627543449402
+8430,0.0173,0.7257712483406067
+8440,0.0188,0.7517357468605042
+8450,0.0172,1.4544386863708496
+8460,0.0184,0.8707684278488159
+8470,0.0193,0.6012953519821167
+8480,0.0198,0.9415525197982788
+8490,0.0179,1.2781232595443726
+8500,0.0165,0.6411517262458801
+8510,0.0214,1.3048300743103027
+8520,0.0164,1.0317035913467407
+8530,0.0191,1.280391812324524
+8540,0.0194,1.2745990753173828
+8550,0.0187,0.811211884021759
+8560,0.0189,1.224231481552124
+8570,0.018,1.0096673965454102
+8580,0.0198,0.8405938148498535
+8590,0.0199,1.4348673820495605
+8600,0.0189,1.1237190961837769
+8610,0.0176,0.7278895974159241
+8620,0.0192,1.0273234844207764
+8630,0.0207,0.7905086278915405
+8640,0.0188,1.2564754486083984
+8650,0.0193,1.3913291692733765
+8660,0.0185,1.0669301748275757
+8670,0.0197,0.7301857471466064
+8680,0.0188,1.1367110013961792
+8690,0.0181,0.8318895101547241
+8700,0.0185,0.9521366953849792
+8710,0.0174,0.5941877961158752
+8720,0.0178,0.6861777305603027
+8730,0.0178,0.40648457407951355
+8740,0.0188,1.452647089958191
+8750,0.019,1.1174592971801758
+8760,0.023,0.76162189245224
+8770,0.019,0.9453418850898743
+8780,0.017,0.779726505279541
+8790,0.0184,2.2095513343811035
+8800,0.0179,0.6634120941162109
+8810,0.0163,1.0645198822021484
+8820,0.0175,0.751386284828186
+8830,0.0186,1.3877284526824951
+8840,0.0171,0.6783541440963745
+8850,0.0188,1.2080283164978027
+8860,0.0192,1.290414810180664
+8870,0.016,1.0223037004470825
+8880,0.0172,0.7657661437988281
+8890,0.0166,1.068636178970337
+8900,0.0176,0.7494480013847351
+8910,0.0167,0.9242642521858215
+8920,0.0167,0.5937004089355469
+8930,0.0176,1.159949779510498
+8940,0.0167,0.9256805777549744
+8950,0.0163,0.529428243637085
+8960,0.0182,1.145539402961731
+8970,0.0167,0.6877376437187195
+8980,0.0189,0.8787598013877869
+8990,0.016,0.8161001205444336
+9000,0.0178,0.7280420660972595
+9010,0.0176,0.5738248229026794
+9020,0.0174,0.648565411567688
+9030,0.0173,0.8187539577484131
+9040,0.0186,0.7371270656585693
+9050,0.0179,0.6220325827598572
+9060,0.0163,0.532849907875061
+9070,0.0167,1.0468082427978516
+9080,0.0172,0.8014238476753235
+9090,0.0204,0.7104499936103821
+9100,0.0177,1.26918363571167
+9110,0.0167,0.806438684463501
+9120,0.0256,1.2076534032821655
+9130,0.0205,0.5213536024093628
+9140,0.0166,0.7805724740028381
+9150,0.019,0.7784114480018616
+9160,0.0175,0.6235218644142151
+9170,0.0154,0.7562257647514343
+9180,0.018,0.5506068468093872
+9190,0.0151,0.8118301630020142
+9200,0.0159,0.9710285067558289
+9210,0.0175,0.6396836042404175
+9220,0.0181,0.6802005171775818
+9230,0.0164,0.8104036450386047
+9240,0.0157,0.545315146446228
+9250,0.0179,0.5534526705741882
+9260,0.0173,0.5620812177658081
+9270,0.0212,0.6944151520729065
+9280,0.0185,0.7129794359207153
+9290,0.0171,0.5501587390899658
+9300,0.0172,0.4670201241970062
+9310,0.0176,0.6461374759674072
+9320,0.0177,0.7871281504631042
+9330,0.0162,0.6447378396987915
+9340,0.0168,0.6140932440757751
+9350,0.0186,0.5023877024650574
+9360,0.0182,0.5248384475708008
+9370,0.0176,0.608130156993866
+9375,nan,nan