Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth

In this paper, we propose a new alignment technique, Any-Depth Alignment (ADA), which unlocks a model’s innate safety alignment to maintain consistent alignment at any depth of generation. Our study demonstrates:

(1) New Alignment Failure with Deep Prefills. We introduce deep prefill attacks to examine whether models possess a generalizable notion of harmfulness that extends beyond a fixed context depth. Existing alignment strategies fail this test—both in recent open-source OpenAI oss models and in strongly deep-aligned systems like Claude Sonnet 4, where refusal rates collapse under slightly deeper prefills.

(2) "Rethinking" Generation (ADA(RK)). Re-injecting Safety Tokens mid-stream triggers a robust rethinking behavior that restores refusals. This generative defense is training-free and performs on par with, and often better than, deep alignment and self-reflection baselines.

(3) Unlocking Deeper Innate Alignment (ADA(LP)). We trace the rethinking phenomenon to the Safety Tokens whose hidden states are highly separable for harmful content. By leveraging this, ADA(LP) is: (a) Effective, achieving near-100% refusal against deep prefills and reducing adversarial success from over 50% to under 3%; (b) Precise, with minimal over-refusal on benign tasks; and (c) Robust, maintaining performance even when the base model is fine-tuned.

(4) A General Phenomenon Across Diverse LLMs. The unlocking effect is ubiquitous: Safety Tokens related to the assistant header consistently expose a strong, linearly separable harmfulness signal across model families (Llama, Qwen, Mistral, Gemma, DeepSeek variants, gpt-oss), parameter scales, and core designs (dense, Mixture-of-Experts, and reasoning-centric).