PHR-VLA: Planning Horizon Reasoning for Vision-Language-Action Models
推荐理由
存在代码/项目页线索,利于进入复现
核心判断
论文摘要(中文)
论文提出 PHR-VLA,为视觉语言动作模型加入规划时域推理。训练时用轻量未来状态预测头,把模型内部表示与未来观测得到的潜在动力学对齐;测试时仍由 VLA 根据当前输入输出动作。
Vision-language-action models (VLAs) have shown strong promise for general-purpose robotic manipulation by mapping language instructions and vision observations directly to actions. However, most VLAs primarily condition action prediction on current observations and lack an explicit mechanism for reasoning over future task dynamics, which is particularly important for fine-grained, contact-rich manipulation. We present PHR-VLA, a framework that enables planning-horizon reasoning in VLAs through privileged latent representations of future dynamics. PHR-VLA introduces a lightweight auxiliary future head that, during training, aligns the VLA’s internal representations with latent dynamics extracted from future observations. Evaluation results demonstrate that local, contact-centric, patchlevel latent dynamics supervision from the wrist camera improves success rate on LIBERO from 84.1% to 88.4% and on real-world disassembly tasks from 63.3 % to 82.5 %. Patch-level supervision 1Davood Soleymanzadeh, Bihao Zhang, and Minghui Zheng are with the J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA (e-mail: davoodso@tamu.edu; bhzhang@tamu.edu; mhzheng@tamu.edu). 2Kaidi Zhang, Zhiyuan Zhang, and Yu She are with the Department of Industrial Engineering, Purdue University, West Lafayette, IN 47907, USA (e-mail: zhan5896@purdue.edu; zhan5570@purdue.edu; shey@purdue.edu). 3Xiao Liang is with the Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843 USA (e-mail: xliang@tamu.edu). This work was partially supported by the USA National Science Foundation under Grant No. 2527316, No. 2422826 and No. 2423068. Portions of this research were conducted with the advanced computing resources provided by Texas A&M High Performance Research Computing. from a third-person camera also improves performance on MetaWorld from 56.70 % to 57.8 %. These results demonstrate that privileged latent dynamics alignment provides an effective training signal for improving anticipatory reasoning in VLA policies. Project website: https://davoodsz.github.io/PHR-VLA.github.io/
研究动机(中文总结)
多数 VLA 主要根据当前图像和指令预测动作,没有显式推理未来任务动态;这一缺陷在精细、接触密集的操作任务中特别明显。
We present PHR-VLA, a framework that enables planning-horizon reasoning in VLAs through privileged latent representations of future dynamics.
创新与贡献(中文总结)
利用训练期间可见的未来观测构造特权潜在动力学监督,使 VLA 学到面向规划时域的表示,而无需在执行时直接访问未来信息。
We present PHR-VLA, a framework that enables planning-horizon reasoning in VLAs through privileged latent representations of future dynamics.
方法与证据
方法(中文总结)
添加轻量辅助未来预测头,将 VLA 隐表示与未来观测提取的潜在动力学对齐;比较腕部相机、第三人称视角以及 patch 级监督。
We present PHR-VLA, a framework that enables planning-horizon reasoning in VLAs through privileged latent representations of future dynamics.
实验结果(中文总结)
腕部相机的局部接触区域 patch 级监督使 LIBERO 成功率从 84.1% 升至 88.4%,真实拆解任务从 63.3% 升至 82.5%;第三人称监督使 MetaWorld 从 56.70% 升至 57.8%。
PHR-VLA and PHR-VLA (JEPA) achieve the best overall performance, with an average success rate of 82.5% and 77.5 % outperforming ACT (35.8 %), Diffusion Policy (51.7%), and baseline SmolVLA (63.3%).