--- license: apache-2.0 language: - en library_name: transformers pipeline_tag: text-generation base_model: AlgoDriveAI/Inferno-Exp-1.5 datasets: - openai/gsm8k tags: - math - gsm8k - reasoning - small-language-model - gqa - custom_code - reinforcement-learning - grpo - experimental model-index: - name: Inferno-Exp-1.5-GSM8K results: - task: type: text-generation name: Text Generation dataset: name: GSM8K type: openai/gsm8k config: main split: test metrics: - type: accuracy value: 8.87 name: accuracy (zero-shot, greedy) - type: accuracy value: 9.86 name: maj@8 (zero-shot) --- # Inferno-Exp-1.5-GSM8K > ⚠️ **Experimental.** This is a research checkpoint from AlgoDriveAI's Inferno series. **Inferno-Exp-1.5-GSM8K** is a **76M-parameter math and general reasoning model** trained from scratch. It solves **8.87% of the GSM8K test set** (117 of 1,319 problems) zero-shot, with one greedy answer per problem, reasoning step by step in plain English: no code interpreter, no calculator, no verifier. Only **about 6% of its pretraining text** (800 MB of 13.5 GB of raw text) was synthetic grade-school-math-style material; the rest is general data. After pretraining, it received a light GRPO policy-tuning pass on GSM8K *training* questions. > To our knowledge, Inferno-Exp-1.5-GSM8K is the smallest openly released model to solve about 1 in 11 GSM8K > problems while reasoning in plain language, without code execution, a verifier, or supervised > fine-tuning on math data. ## GSM8K results All 1,319 test problems, zero-shot, prompt `User: \n[SOLUTION] \n`, graded on the first `#### ` line, up to 512 new tokens. Sampled metrics use 8 samples per problem (temperature 0.8, top-k 50, top-p 0.95). | Metric | Score | |---|---| | **Greedy accuracy** (one answer per problem) | **8.87%** (117 / 1,319) | | maj@8 (majority vote of 8 samples) | 9.86% | | pass@1 (average single sample) | 5.61% ± 0.34 | | pass@8 (any of 8 samples correct) | 25.55% | | Stops cleanly (EOS), greedy / sampled | 88% / 91% | | Writes a `####` answer line (sampled) | 85% | pass@8 shows the model reaches the right answer on about a quarter of the test set at least some of the time. The gap between that and the greedy score is headroom for further RL. ### For context Published GSM8K scores of other models. The protocols differ (few-shot vs zero-shot, standard harness vs custom prompt, code vs plain text), so read this as context, not a head-to-head leaderboard. | Model | Params | GSM8K | Setting | |---|---|---|---| | GPT-Neo-125M | 125M | 0.3% | 5-shot, Open LLM Leaderboard | | SmolLM-135M | 135M | 1.0% | 5-shot | | SmolLM2-135M | 135M | 1.4% | 5-shot; pretrained on 2T tokens | | SmolLM2-360M | 360M | 3.2% | 5-shot | | SmolLM2-360M-Instruct | 360M | 7.4% | 5-shot | | **Inferno-Exp-1.5-GSM8K** | **76M** | **8.87%** | zero-shot, greedy, plain-language reasoning | | Llama-2-7B | 7B | 14.6% | as listed in the TinyGSM paper, Table 1 | | Phi-GSM 125M (TinyGSM) | 125M | 63.1% | Python solutions run by an interpreter; fine-tuned on 1.8B tokens of GSM8K-style problems | Phi-GSM shows what a dedicated GSM8K fine-tune with code execution can reach at this scale. Inferno takes a different route: a general model with a small slice of math data, solving problems in natural language. ### A reasoner, not a fact store Inferno-Exp-1.5-GSM8K is **not a facts-based model** in the usual sense. At 76M parameters it holds little world knowledge, so don't use it to look things up (dates, names, places, trivia); it will often get those wrong or make them up. What it is built for is **logical reasoning**: reading a problem, keeping track of the quantities and conditions, working through the steps in order, and reaching a conclusion. Give it everything it needs in the prompt and let it do the thinking. ## Training 1. **Pretraining from scratch** on 13.5 GB of raw mainly synthetic text; Released as [Inferno-Exp-1.5](https://huggingface.co/AlgoDriveAI/Inferno-Exp-1.5). 2. **Policy tuning with GRPO** (no supervised fine-tuning). For each GSM8K training question the model wrote several solutions; solutions that scored above their group's average were reinforced and the rest were discouraged, with a KL penalty keeping it close to Inferno-Exp-1.5. Each solution was scored by an LLM teacher on correctness, reasoning quality, efficiency and clarity (the final answer was checked against the GSM8K reference first), with a penalty for not stopping on time. | | | |---|---| | RL updates | 100 GRPO steps | | Rollouts | 8 solutions per question × 8 questions per step | | RL settings | KL coefficient 0.05, peak LR 1e-05 (WSD schedule) | | Parameters | 76,163,200 | | Layers × width | 14 × 640 | | Attention | grouped-query, 8 query / 2 KV heads, head dim 80, full-head RoPE | | Feed-forward | SwiGLU, inner width 1536 | | Context | 2,048 tokens | | Tokenizer | Mistral-7B-v0.1 (padded to 32,064); `<\|endoftext\|>` ends a document | ## Prompt format Use the same format the model was tuned and evaluated with. **Keep the space after `[SOLUTION]`**: the model performs noticeably better with it. ``` User: [SOLUTION] ``` That is: `User: `, the question, a space and a newline, then `[SOLUTION]`, a space and a newline. In Python: `"User: {question} \n[SOLUTION] \n"`. The model answers with numbered steps and ends with a `#### ` line. ## Usage Works with Transformers 4.45+ and 5.x. The model code ships in the repo, so load it with `trust_remote_code=True`. ```python from transformers import AutoTokenizer, AutoModelForCausalLM repo = "AlgoDriveAI/Inferno-Exp-1.5-GSM8K" tok = AutoTokenizer.from_pretrained(repo) model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True).eval() PROMPT = "User: {question} \n[SOLUTION] \n" # keep the space after [SOLUTION] question = "Weng earns $12 an hour for babysitting. Yesterday, she just did 50 minutes of babysitting. How much did she earn?" ids = tok(PROMPT.format(question=question), return_tensors="pt") out = model.generate(**ids, max_new_tokens=512, do_sample=False) print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True)) ```
Gradio demo (click to expand) ```python import torch, gradio as gr from threading import Thread from transformers import AutoTokenizer, AutoModelForCausalLM, TextIteratorStreamer REPO = "AlgoDriveAI/Inferno-Exp-1.5-GSM8K" PROMPT = "User: {question} \n[SOLUTION] \n" # keep the space after [SOLUTION] device = "cuda" if torch.cuda.is_available() else "cpu" tok = AutoTokenizer.from_pretrained(REPO) model = AutoModelForCausalLM.from_pretrained(REPO, trust_remote_code=True).to(device).eval() def solve(question, max_new_tokens, temperature): ids = tok(PROMPT.format(question=question.strip()), return_tensors="pt").to(device) streamer = TextIteratorStreamer(tok, skip_prompt=True, skip_special_tokens=True) kwargs = dict(**ids, max_new_tokens=int(max_new_tokens), streamer=streamer, do_sample=temperature > 0) if temperature > 0: kwargs.update(temperature=temperature, top_p=0.95) Thread(target=model.generate, kwargs=kwargs).start() text = "" for piece in streamer: text += piece yield text gr.Interface( solve, [gr.Textbox(lines=4, label="Question"), gr.Slider(32, 512, 512, step=16, label="Max new tokens"), gr.Slider(0.0, 1.2, 0.0, step=0.05, label="Temperature (0 = greedy)")], gr.Textbox(lines=14, label="Solution"), title="Inferno-Exp-1.5-GSM8K (experimental)", ).launch() ```
## Evaluation The numbers above come from AlgoDriveAI's own zero-shot evaluation: the prompt format above, greedy decoding plus 8 sampled solutions per problem, and the first `#### ` line compared numerically with the reference answer. For the standard harness setting (5-shot, its own prompt format, so expect a different number): ```bash python -m lm_eval --model hf \ --model_args "pretrained=AlgoDriveAI/Inferno-Exp-1.5-GSM8K,trust_remote_code=True,dtype=float32" \ --tasks gsm8k --num_fewshot 5 --batch_size 16 ``` ## Limitations - **Experimental.** A 76M-parameter model doing arithmetic in plain text makes arithmetic and reasoning mistakes; most GSM8K problems are still out of reach. - **Contamination:** a 13-gram overlap check between the synthetic math data and the GSM8K test set has not been run yet; results will be added here. - Format is not fully reliable yet: about 15% of sampled solutions never write a `####` line, and about 12% of greedy solutions run to the 512-token limit instead of stopping. - Not a knowledge model: it does not reliably know facts about the world. Put the facts it needs in the prompt. - Not an instruction-following chat model; use the prompt format above. - Generation re-reads the full context each step (no KV cache); fine at this size. - The tokenizer does not add a BOS token; `bos_token` is `<|endoftext|>`, the document boundary seen in training.