RENLABENDOSCOPIC AUTONOMY Chi Kit Ng
← All research projects

Language-conditioned target acquisition 2026 · Preprint

EndoVLA-OOV

Language-Conditioned Out-of-View Target Acquisition in Robotic Endoscopy

Finding a target starts before it enters the frame.

Chi Kit Ng*, Yidong Zhang*, Tianchun Wu*, Zhiqing Tang, Hanlin Zhao, Siu Hing Lui, Tao Yang, Jingsong Lin, Huxin Gao, Yeung Trevor, Raymond Tang, Hongliang Ren†.

* Equal contribution · † Corresponding author

EndoVLA-OOV language-conditioned search and tracking for out-of-view targets.
The scan-then-revisit task: translate a target description into endoscopic camera motion. Figure 1 of the preprint. Source ↗

01 / The idea

Finding a target starts before it enters the frame.

An instruction identifies the region to revisit. Initial, intermediate and current images help the policy search through partial observations, producing continuous commands for the two distal bending axes.

1,050demonstration episodes
6tasks in the dataset
93.3%target acquisition · 28/30

Acquisition result: full-data policy on a second robotic embodiment in an unseen ex-vivo environment. Acquisition and precise centering are evaluated separately. Evaluation details ↗

02 / The approach

How it works

From observations to a physical outcome.

01

Describe the target

Language specifies appearance and an anatomical region.

02

Use visual context

Earlier and current observations guide search beyond the present view.

03

Act continuously

The adapted policy predicts chunks of two-axis bending commands.

Temporal visual context, robot state and the continuous action interface. Figure 2 of the preprint.
Temporal visual context, robot state and the continuous action interface. Figure 2 of the preprint. Source ↗

03 / In action

Watch the demonstration

Full demonstration of language-conditioned target acquisition and tracking.

04 / Reference

Build on this work

@misc{ng2026endovlaoov,
  title = {EndoVLA-OOV: Language-Conditioned Out-of-View Target Acquisition in Robotic Endoscopy},
  author = {Chi Kit Ng and Yidong Zhang and Tianchun Wu and Zhiqing Tang and Hanlin Zhao and Siu Hing Lui and Tao Yang and Jingsong Lin and Huxin Gao and Yeung Trevor and Raymond Tang and Hongliang Ren},
  year = {2026},
  eprint = {2610.24747},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2610.24747}
}

Chi Kit Ng · CUHK

Get in touch.

For research, student projects and opportunities.

Open email app ↗