KOREAson
/

KO-REAson-G3-4B-0831

@@ -3,197 +3,228 @@ library_name: transformers
 tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 tags: []
 ---
+# KO-REAson
+**KO-REAson** is a series of Korean-centric reasoning language models developed in collaboration with OneLineAI, KISTI, HAE-RAE and ORACLE.
+We use the **Language-Mixed Chain-of-Thought (CoT)** approach, which allows the model to alternate between English and Korean during the “Think” stage of reasoning, preserving key Korean terms while leveraging English for logical scaffolding.
+Top-performing models of our series [KO-REAson-AX3_1-7B-0831](https://huggingface.co/KoReason/KO-REASon-AX3_1-7B-0831) and [KO-REAson-7B-Q2_5-0831](https://huggingface.co/KoReason/KO-REASon-7B-Q2_5-0831) show performance comparable to models trained on closed-source datasets such as Exaone-Deep-7.8B.
+<p align="left">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/60d3e619b8448e1785bbda2a/CaHtUra_lWZmg04d-QNtm.png"
+       alt="Model Comparison" width="750"/>
+  <br>
+  <em style="display:inline-block; max-width:750px; text-align:cener; white-space:normal; word-wrap:break-word; line-height:1.5;">
+    <b>Left:</b> Average performance (Held-out-Ko) of open models trained on closed or open data;
+    our models are highlighted in green.
+  </em>
+</p>
+## Model Details
+The **KO-REAson-0831** family comes in six variants based on the base model used.
+| Model (link)                                                                                 | Base           | Notes                       |
+| -------------------------------------------------------------------------------------------- | -------------------- | --------------------------- |
+| [KO-REAson-L3_1-8B-0831](https://huggingface.co/KoReason/KO-REASon-L3_1-8B-0831)   | [Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)      | `L3_1` → Llama-3.1-8B       |
+| [KO-REAson-KL3_1-8B-0831](https://huggingface.co/KoReason/KO-REASon-KL3_1-8B-0831) | [Koni-Llama-3.1-8B](https://huggingface.co/KISTI-KONI/KONI-Llama3.1-8B-Instruct-20241024)    | `KL3_1` → Koni-Llama-3.1-8B |
+| [KO-REAson-G3-4B-0831](https://huggingface.co/KoReason/KO-REASon-G3-4B-0831)        | [Gemma-3 4B](https://huggingface.co/google/gemma-3-4b-it)           | `G3` → Gemma-3-4B           |
+| [KO-REAson-AX3_1-7B-0831](https://huggingface.co/KoReason/KO-REASon-AX3_1-7B-0831) | [A.X.-3.1-Light (≈7B)](https://huggingface.co/skt/A.X-3.1-Light) | `AX3_1` → A.X.-3.1-Light    |
+| [KO-REAson-K2505_8B-0831](https://huggingface.co/KoReason/KO-REASon-K2505_8B-0831) | [Kanana-2505 (8B)](https://huggingface.co/kakaocorp/kanana-1.5-8b-instruct-2505)     | `K2505` → Kanana-2505       |
+| [KO-REAson-7B-Q2_5-0831](https://huggingface.co/KoReason/KO-REASon-7B-Q2_5-0831)   | [Qwen-2.5 (7B)](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)        | `Q2_5` → Qwen-2.5           |
+# Performance
+**Evaluation Datasets**
+The model's performance was evaluated across a total of 11 benchmarks, and the evaluation suite is divided into two parts: (You can check these benchmarks in [HAERAE-HUB/KoSimpleEval](https://huggingface.co/datasets/HAERAE-HUB/KoSimpleEval))
+- **Held-in**: This set of benchmarks is used for routine monitoring of the model's performance during the training and ablation study phases.
+- **Held-out**: This set is used only once to evaluate the final model after all training and ablations are complete.
+This separation is designed to prevent inadvertent overfitting to the benchmarks during the iterative training process and to provide a more accurate measure of the model's generalization capabilities.
+|**Category**|**Held-in**|**Held-out**|
+|---|---|---|
+|**General Knowledge**|KMMLU-Redux|KMMLU-HARD, KMMLU-Pro|
+|**Reasoning**|MCLM|KSM, GPQA, AIME2024, AIME2025|
+|**Korean-specific**|HAE-RAE Bench|CLIcK, KoBALT-700|
+**Comparison with models trained on public datasets**
+<table>
+  <thead>
+    <tr>
+      <th>Models</th>
+      <th># Instances</th>
+      <th>Methodology</th>
+      <th>Held-Out (Ko)</th>
+      <th>Held-Out (En)</th>
+      <th>Total</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <th>KO-REASon-AX3_1-7B-0831(Ours)</th>
+      <td>260k</td>
+      <td>SFT</td>
+      <td><b>44.6</b></td>
+      <td>41.2</td>
+      <td><u>43.3</u></td>
+    </tr>
+    <tr>
+      <th>KO-REASon-7B-Q2_5-0831(Ours)</th>
+      <td>260k</td>
+      <td>SFT</td>
+      <td><b>45.10</b></td>
+      <td>38.75</td>
+      <td><u>49.95</u></td>
+    </tr>
+    <tr>
+      <td colspan="6" style="text-align:center; font-weight:bold;">Open Recipe (En)</td>
+    </tr>
+    <tr>
+      <th>OpenThinker3-7B</th>
+      <td>1.2M</td>
+      <td>SFT</td>
+      <td>33.6</td>
+      <td><b>55.5</b></td>
+      <td>41.8</td>
+    </tr>
+    <tr>
+      <th>s1.1-7B</th>
+      <td>1k</td>
+      <td>SFT</td>
+      <td>35.6</td>
+      <td>23.4</td>
+      <td>31.1</td>
+    </tr>
+    <tr>
+      <th>Llama-3.1-Nemotron-Nano-8B-v1</th>
+      <td>&gt;3M</td>
+      <td>SFT &amp; RL</td>
+      <td>27.0</td>
+      <td>44.1</td>
+      <td>33.4</td>
+    </tr>
+    <tr>
+      <td colspan="6" style="text-align:center; font-weight:bold;">Open Recipe (Ko)</td>
+    </tr>
+    <tr>
+      <th>Ko-R1-14B</th>
+      <td>45k</td>
+      <td>SFT</td>
+      <td><u>43.7</u></td>
+      <td><u>46.3</u></td>
+      <td><b>44.7</b></td>
+    </tr>
+    <tr>
+      <th>Ko-R1-7B</th>
+      <td>45k</td>
+      <td>SFT</td>
+      <td>27.3</td>
+      <td>36.1</td>
+      <td>30.6</td>
+    </tr>
+    <tr>
+      <th>LLaMa-3.1-Ko-Reasoning-8B</th>
+      <td>63k</td>
+      <td>SFT</td>
+      <td>17.7</td>
+      <td>7.7</td>
+      <td>14.0</td>
+    </tr>
+  </tbody>
+</table>
+**Held-out benchmark performance**
+<table border="1" cellspacing="0" cellpadding="6">
+  <thead>
+    <tr>
+      <th rowspan="2">Model</th>
+      <th rowspan="2">Model Size</th>
+      <th colspan="2">General</th>
+      <th colspan="4">Reasoning</th>
+      <th colspan="2">Korean-Specific</th>
+      <th rowspan="2">Average<br>(Held-out)</th>
+      <th rowspan="2">Average<br>(Held-out-Ko)</th>
+    </tr>
+    <tr>
+      <th>KMMLU-HARD</th>
+      <th>KMMLU-Pro</th>
+      <th>KSM</th>
+      <th>AIME 2024</th>
+      <th>AIME 2025</th>
+      <th>GPQA</th>
+      <th>CLiCK</th>
+      <th>KoBALT-700</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><b>Llama-3.1-Nemotron-Nano-8B</b></td>
+      <td>8.03</td><td>21.47</td><td>22.89</td><td>47.06</td><td>56.67</td><td>43.33</td><td>32.32</td><td>34.54</td><td>9.29</td><td>33.45</td><td>27.05</td>
+    </tr>
+    <tr>
+      <td><b>Llama-3.1-Korean-Reasoning-8B-Instruct</b></td>
+      <td>8.03</td><td>14.91</td><td>21.72</td><td>6.09</td><td>0.00</td><td>0.00</td><td>23.23</td><td>39.65</td><td>6.14</td><td>13.97</td><td>17.70</td>
+    </tr>
+    <tr>
+      <td><b>EXAONE-Deep-7.8B</b></td>
+      <td>7.82</td><td><u>40.96</u></td><td>37.35</td><td><b>70.80</b></td><td><b>70.00</b></td><td><b>63.33</b></td><td><b>64.65</b></td><td>54.24</td><td>18.86</td><td><b>52.52</b></td><td>44.44</td>
+    </tr>
+    <tr>
+      <td><b>DeepSeek-R1-Distill-Qwen-7B</b></td>
+      <td>7.62</td><td>0.00</td><td>23.00</td><td>56.09</td><td>60.00</td><td>40.00</td><td>43.43</td><td>0.00</td><td>8.29</td><td>28.85</td><td>17.48</td>
+    </tr>
+    <tr>
+      <td><b>DeepSeek-R1-Distill-Llama-8B</b></td>
+      <td>8.03</td><td>23.22</td><td>26.26</td><td>29.97</td><td>33.33</td><td>20.00</td><td><U>46.46</u></td><td>39.05</td><td>13.29</td><td>28.95</td><td>26.36</td>
+    </tr>
+    <tr>
+      <td><b>s1.1-7B</b></td>
+      <td>7.62</td><td>31.16</td><td><u>37.70</u></td><td>30.60</td><td>16.67</td><td>23.33</td><td>30.30</td><td><u>56.84</u></td><td><u>21.86</u></td><td>31.06</td><td>35.63</td>
+    </tr>
+    <tr>
+      <td><b>OpenThinker3-7B</b></td>
+      <td>7.62</td><td>30.31</td><td>26.26</td><td><u>63.59</u></td><td><u>66.67</u></td><td><u>53.33</u></td><td><u>46.46</u></td><td>47.69</td><td>10.14</td><td>35.63</td><td>30.60</td>
+    </tr>
+    <tr>
+      <td><b>Ko-R1-7B</b></td>
+      <td>7.61</td><td>28.46</td><td>19.31</td><td>51.61</td><td>46.67</td><td>33.33</td><td>28.28</td><td>32.48</td><td>4.71</td><td>30.61</td><td>27.31</td>
+    </tr>
+    <tr>
+      <td><b>KO-REASon-AX3_1-7B-0831</b></td>
+      <td>7.26</td><td>45.57</td><td>38.13</td><td>52.80</td><td>53.33</td><td>33.33</td><td>36.87</td><td><b>62.86</b></td><td>23.43</td><td><u>43.29</u></td><td><u>44.56</u></td>
+    </tr>
+    <tr>
+      <td><b>KO-REASon-7B-Q2_5-0831</b></td>
+      <td>7.26</td><td><b>46.81</b></td><td><b>44.93</b></td><td>48.11</td><td>43.33</td><td>30.00</td><td>42.93</td><td>60.65</td><td><b>25.00</b></td><td>42.72</td><td><b>45.10</b></td>
+    </tr>
+  </tbody>
+</table>
+## Citation
+```
+The paper will be released soon!
+```
+## Contact
+For any questions contact us via the following email :)
+```
+[email protected]
+```
+## Acknowlegments
+This research was supported by the Korea Institute of Science and Technology Information (KISTI) (No.(KISTI) K25L1M1C1), aimed at developing KONI (KISTI Open Neural Intelligence), a large language model specialized in science and technology.