Blame for README.md

---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
base_model: AlgoDriveAI/Inferno-Exp-1.5
datasets:
- openai/gsm8k
tags:
- math
- gsm8k
- reasoning
- small-language-model
- gqa
- custom_code
- reinforcement-learning
- grpo
- experimental
model-index:
- name: Inferno-Exp-1.5-GSM8K
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8K
      type: openai/gsm8k
      config: main
      split: test
    metrics:
    - type: accuracy
      value: 8.87
      name: accuracy (zero-shot, greedy)
    - type: accuracy
      value: 9.86
      name: maj@8 (zero-shot)
---

# Inferno-Exp-1.5-GSM8K

> ⚠️ **Experimental.** This is a research checkpoint from AlgoDriveAI's Inferno series.

**Inferno-Exp-1.5-GSM8K** is a **76M-parameter math and general reasoning model** trained from scratch. It solves
**8.87% of the GSM8K test set** (117 of 1,319 problems) zero-shot, with one greedy answer per problem,
reasoning step by step in plain English: no code interpreter, no calculator, no verifier.

Only **about 6% of its pretraining text** (800 MB of 13.5 GB of raw text) was synthetic
grade-school-math-style material; the rest is general data. After pretraining, it received a light
GRPO policy-tuning pass on GSM8K *training* questions.

> To our knowledge, Inferno-Exp-1.5-GSM8K is the smallest openly released model to solve about 1 in 11 GSM8K
> problems while reasoning in plain language, without code execution, a verifier, or supervised
> fine-tuning on math data.

## GSM8K results

All 1,319 test problems, zero-shot, prompt `User: <question> \n[SOLUTION] \n`, graded on the first
`#### <answer>` line, up to 512 new tokens. Sampled metrics use 8 samples per problem
(temperature 0.8, top-k 50, top-p 0.95).

| Metric | Score |
|---|---|
| **Greedy accuracy** (one answer per problem) | **8.87%** (117 / 1,319) |
| maj@8 (majority vote of 8 samples) | 9.86% |
| pass@1 (average single sample) | 5.61% ± 0.34 |
| pass@8 (any of 8 samples correct) | 25.55% |
| Stops cleanly (EOS), greedy / sampled | 88% / 91% |
| Writes a `####` answer line (sampled) | 85% |

pass@8 shows the model reaches the right answer on about a quarter of the test set at least some of
the time. The gap between that and the greedy score is headroom for further RL.

### For context

Published GSM8K scores of other models. The protocols differ (few-shot vs zero-shot, standard harness
vs custom prompt, code vs plain text), so read this as context, not a head-to-head leaderboard.

| Model | Params | GSM8K | Setting |
|---|---|---|---|
| GPT-Neo-125M | 125M | 0.3% | 5-shot, Open LLM Leaderboard |
| SmolLM-135M | 135M | 1.0% | 5-shot |
| SmolLM2-135M | 135M | 1.4% | 5-shot; pretrained on 2T tokens |
| SmolLM2-360M | 360M | 3.2% | 5-shot |
| SmolLM2-360M-Instruct | 360M | 7.4% | 5-shot |
| **Inferno-Exp-1.5-GSM8K** | **76M** | **8.87%** | zero-shot, greedy, plain-language reasoning |
| Llama-2-7B | 7B | 14.6% | as listed in the TinyGSM paper, Table 1 |
| Phi-GSM 125M (TinyGSM) | 125M | 63.1% | Python solutions run by an interpreter; fine-tuned on 1.8B tokens of GSM8K-style problems |

Phi-GSM shows what a dedicated GSM8K fine-tune with code execution can reach at this scale. Inferno takes
a different route: a general model with a small slice of math data, solving problems in natural language.

### A reasoner, not a fact store

Inferno-Exp-1.5-GSM8K is **not a facts-based model** in the usual sense. At 76M parameters it holds little world
knowledge, so don't use it to look things up (dates, names, places, trivia); it will often get those
wrong or make them up. What it is built for is **logical reasoning**: reading a problem, keeping track
of the quantities and conditions, working through the steps in order, and reaching a conclusion. Give
it everything it needs in the prompt and let it do the thinking.

## Training

1. **Pretraining from scratch** on 13.5 GB of raw mainly synthetic text; Released as
   [Inferno-Exp-1.5](https://huggingface.co/AlgoDriveAI/Inferno-Exp-1.5).
2. **Policy tuning with GRPO** (no supervised fine-tuning). For each GSM8K training question the model
   wrote several solutions; solutions that scored above their group's average were reinforced and the
   rest were discouraged, with a KL penalty keeping it close to Inferno-Exp-1.5. Each solution was scored by an LLM teacher on correctness, reasoning quality, efficiency and clarity (the final answer was checked against the GSM8K reference first), with a penalty for not stopping on time.

| | |
|---|---|
| RL updates | 100 GRPO steps |
| Rollouts | 8 solutions per question × 8 questions per step |
| RL settings | KL coefficient 0.05, peak LR 1e-05 (WSD schedule) |
| Parameters | 76,163,200 |
| Layers × width | 14 × 640 |
| Attention | grouped-query, 8 query / 2 KV heads, head dim 80, full-head RoPE |
| Feed-forward | SwiGLU, inner width 1536 |
| Context | 2,048 tokens |
| Tokenizer | Mistral-7B-v0.1 (padded to 32,064); `<\|endoftext\|>` ends a document |

## Prompt format

Use the same format the model was tuned and evaluated with. **Keep the space after `[SOLUTION]`**: the
model performs noticeably better with it.

```
User: <your question> 
[SOLUTION] 
```

That is: `User: `, the question, a space and a newline, then `[SOLUTION]`, a space and a newline.
In Python: `"User: {question} \n[SOLUTION] \n"`. The model answers with numbered steps and ends with a
`#### <answer>` line.

## Usage

Works with Transformers 4.45+ and 5.x. The model code ships in the repo, so load it with `trust_remote_code=True`.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "AlgoDriveAI/Inferno-Exp-1.5-GSM8K"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True).eval()

PROMPT = "User: {question} \n[SOLUTION] \n"   # keep the space after [SOLUTION]
question = "Weng earns $12 an hour for babysitting. Yesterday, she just did 50 minutes of babysitting. How much did she earn?"

ids = tok(PROMPT.format(question=question), return_tensors="pt")
out = model.generate(**ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
```

<details>
<summary><b>Gradio demo</b> (click to expand)</summary>

```python
import torch, gradio as gr
from threading import Thread
from transformers import AutoTokenizer, AutoModelForCausalLM, TextIteratorStreamer

REPO = "AlgoDriveAI/Inferno-Exp-1.5-GSM8K"
PROMPT = "User: {question} \n[SOLUTION] \n"   # keep the space after [SOLUTION]
device = "cuda" if torch.cuda.is_available() else "cpu"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, trust_remote_code=True).to(device).eval()

def solve(question, max_new_tokens, temperature):
    ids = tok(PROMPT.format(question=question.strip()), return_tensors="pt").to(device)
    streamer = TextIteratorStreamer(tok, skip_prompt=True, skip_special_tokens=True)
    kwargs = dict(**ids, max_new_tokens=int(max_new_tokens), streamer=streamer, do_sample=temperature > 0)
    if temperature > 0:
        kwargs.update(temperature=temperature, top_p=0.95)
    Thread(target=model.generate, kwargs=kwargs).start()
    text = ""
    for piece in streamer:
        text += piece
        yield text

gr.Interface(
    solve,
    [gr.Textbox(lines=4, label="Question"),
     gr.Slider(32, 512, 512, step=16, label="Max new tokens"),
     gr.Slider(0.0, 1.2, 0.0, step=0.05, label="Temperature (0 = greedy)")],
    gr.Textbox(lines=14, label="Solution"),
    title="Inferno-Exp-1.5-GSM8K (experimental)",
).launch()
```

</details>

## Evaluation

The numbers above come from AlgoDriveAI's own zero-shot evaluation: the prompt format above, greedy
decoding plus 8 sampled solutions per problem, and the first `#### <answer>` line compared numerically
with the reference answer. For the standard harness setting (5-shot, its own prompt format, so expect a
different number):

```bash
python -m lm_eval --model hf \
  --model_args "pretrained=AlgoDriveAI/Inferno-Exp-1.5-GSM8K,trust_remote_code=True,dtype=float32" \
  --tasks gsm8k --num_fewshot 5 --batch_size 16
```

## Limitations

- **Experimental.** A 76M-parameter model doing arithmetic in plain text makes arithmetic and reasoning
  mistakes; most GSM8K problems are still out of reach.
- **Contamination:** a 13-gram overlap check between the synthetic math data and the GSM8K test set has not been run yet; results will be added here.
- Format is not fully reliable yet: about 15% of sampled solutions never write a `####` line, and
  about 12% of greedy solutions run to the 512-token limit instead of stopping.
- Not a knowledge model: it does not reliably know facts about the world. Put the facts it needs in the prompt.
- Not an instruction-following chat model; use the prompt format above.
- Generation re-reads the full context each step (no KV cache); fine at this size.
- The tokenizer does not add a BOS token; `bos_token` is `<|endoftext|>`, the document boundary seen in training.