FINE_InternVLA_N1_R2R_70

FINE (Future-Informed Navigation Encoding) fine-tuned on 70% of the R2R-CE training demonstrations.

This is the checkpoint from the paper FINE: Future-Informed Navigation Encoding for Data-Efficient Vision-Language Navigation.

Model description

Standard VLN training only uses the executed actions of each demonstration and ignores what the agent is about to see. FINE recovers that future-observation supervision at no extra annotation or interaction cost, by adding two auxiliary branches to the VLN backbone:

• Explicit landmark tokens (18 slots). They follow the ordered landmarks in the instruction and predict the upcoming landmark region in 2D semantic (Qwen2.5-VL patch) and 3D geometric (VGGT) feature space.
• An implicit future token (1 token). It is trained contrastively (InfoNCE) to tell the future state actually reached apart from other futures, including same-scene counterfactual futures rolled out by a video world model.

Both token streams condition System 1 through separate, gated projectors. Future frames, landmark masks, VGGT and world-model rollouts are only used during training. At deployment the policy keeps the original InternVLA-N1 action space and action head and needs no lookahead.

Architecture	InternVLA-N1 dual system: Qwen2.5-VL-7B System 2 + NextDiT (nextdit_async) trajectory System 1, with FINE explicit and implicit tokens
Base model	InternRobotics/InternVLA-N1-DualVLN
Training data	R2R-CE (70% of the training demonstrations), with landmark masks and world-model counterfactual futures
Inputs	RGB-D observation history (8 frames, 224×224), instruction, instruction landmarks
Outputs	Pixel goal / discrete actions (System 2) and a continuous local trajectory (System 1)
Precision	bfloat16
Files	Full model weights (4 safetensors shards), tokenizer, image processor and chat template

Citation

@misc{nguyen2026fine,
      title={FINE: Future-Informed Navigation Encoding for Data-Efficient Vision-Language Navigation},
      author={Khang H. Nguyen and Hoang Pham Quang Nguyen and Ha Phuong Nguyen and Khanh Dinh Binh and Xuan Ha Nguyen and Vien Ngo and Duy Ho Nguyen Minh and Huan Nguyen and An T. Le},
      year={2026},
      eprint={2609.32855},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2609.32855},
}

Acknowledgement

Built on InternNav / InternVLA-N1, Qwen2.5-VL, VGGT, Depth Anything V2, Grounding DINO, NanoWM and FOCA. Their licenses also apply to the corresponding components.