umegan/behavior2026-pi05-run1

π0.5 BEHAVIOR-1K 2026: run1 (continued fine-tune on filtered 2026 demos with subtask prompts)

A research checkpoint from our BEHAVIOR 2026 Challenge work. This is not our submitted policy; the submission was "Routed π0.5 (Comet + 2026 fine-tune)", self-evaluated Q 0.1648. In a small paired evaluation, run1 performs about the same as that policy (see Evaluation).

Lineage

π0.5 base (Physical Intelligence, openpi) → NVIDIA Comet pi05-b1kpt50-cs32 → ymetz/pi05-b1k-pt50-steps-public steps-cont-9999 (Comet pt50 continued on the 2026 demos) → this checkpoint. Credits to all three. See NOTICE. No license beyond the upstream terms is granted, and the Gemma Terms of Use apply.

Training

• Data: the official BEHAVIOR-1K 2026 challenge demos (behavior-1k/2026-challenge-demos, LeRobot v3.0), automatically filtered:
  • 19,716 / 20,000 episodes kept;
  • 2.75 % of frames trimmed (valid-duration trim, idle pauses > 2 s, idle tails);
  • outlier and broken-annotation episodes dropped.
• Prompts: 60 % official task description, 40 % the current skill/subtask from the per-episode skill annotations (e.g. "Pick up the radio from the coffee table.", aligned to the parquet timeline).
• Sampling: all 100 tasks, tasks 50-99 weighted ×2.
• Recipe: openpi-comet (commit 4bb2aa7), config based on pi05_b1k-base (absolute actions, 32-step chunks).
  • Full fine-tune, FSDP over 4 L40S, batch 128.
  • AdamW, cosine LR 2.5e-6 → 2.5e-7, 300 warmup steps, 6,000 steps (~8.5 h).
  • Init: ymetz steps-cont-9999. Norm stats: Comet pt50 (assets/behavior-1k/2025-challenge-demos).
• Loss: 0.0068 → 0.0064.

Usage

Load like Comet pt50: openpi-comet config pi05_b1k-base with norm-stats asset id behavior-1k/2025-challenge-demos (included under assets/). Prompt it with the official 2026 task description (meta/tasks.jsonl of the demo dataset), or with a subtask instruction. It takes the BEHAVIOR R1Pro observation (3 RGB cameras + proprio) and returns 32×23 action chunks. It was served with a routed websocket server compatible with omnigibson.eval (BEHAVIOR-1K v3.9.3).

Evaluation (small, paired, train instances, BEHAVIOR-1K v3.9.3-post1)

• Setup: 80 rollouts, 10 tasks × 8 train instances (no test instances), each paired with the submitted policy on the same (task, instance).
• Mean Q: run1 0.333 vs submitted policy 0.314. Difference +0.019; paired SE 0.041, so not significant.
• Full successes: 4 vs 8.
• Per task:
  • better: picking_up_trash +0.17, installing_a_modem +0.16, putting_shoes_on_rack +0.14, setting_mousetraps +0.06;
  • worse: make_microwave_popcorn −0.13, clean_a_keyboard −0.13, composting_waste −0.06.

Limitations

• Not better than its parent in our evaluation. It finished tasks fully less often.
• Subtask prompts are only useful with a controller that switches them during a rollout; we did not build one.
• Weights are released as-is for research and reproducibility.