Preprint · 2026 · Open access

Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models

Tracey Yee Hsin Tay, Xu Yan, Jonathan Ouyang, Daniel Wu, William Jiang, Jonathan C. Kao, Yuchen Cui

arXiv (Cornell University) · Published 8 January 2026 · DOI 10.48550/arxiv.2601.05336

Open PDFView at the publisherCite this paper

In brief

“Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models” is a 2026 preprint by Tracey Yee Hsin Tay and 6 co-authors, published in arXiv (Cornell University). OpenAlex counts 9 citations as of 22 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
9
Most cited in
2026 (9)
References
33
Authors
7

Abstract

Designing intuitive interfaces for robotic control remains a central challenge in enabling effective human-robot interaction, particularly in assistive care settings. Eye gaze offers a fast, non-intrusive, and intent-rich input modality, making it an attractive channel for conveying user goals. In this work, we present GAMMA (Gaze Assisted Manipulation for Modular Autonomy), a system that leverages ego-centric gaze tracking and a vision-language model to infer user intent and autonomously execute robotic manipulation tasks. By contextualizing gaze fixations within the scene, the system maps visual attention to high-level semantic understanding, enabling skill selection and parameterization without task-specific training. We evaluate GAMMA on a range of table-top manipulation tasks and compare it against baseline gaze-based control without reasoning. Results demonstrate that GAMMA provides robust, intuitive, and generalizable control, highlighting the potential of combining foundation models and gaze for natural and scalable robot autonomy. Project website: https://gamma0.vercel.app/

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Tay, T. Y. H., Yan, X., Ouyang, J., Wu, D., Jiang, W., Kao, J. C., & Cui, Y. (2026). Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2601.05336

MLA 9

Tay, Tracey Yee Hsin, et al. “Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2601.05336.

Chicago

Tay, Tracey Yee Hsin, Xu Yan, Jonathan Ouyang, Daniel Wu, William Jiang, Jonathan C. Kao, and Yuchen Cui. 2026. “Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2601.05336.

Harvard

Tay, T. Y. H. et al. (2026) ‘Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models’, arXiv (Cornell University). doi:10.48550/arxiv.2601.05336.

IEEE

T. Y. H. Tay et al., “Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2601.05336.

BibTeX

@misc{tay2026intent,
  title     = {{Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models}},
  author    = {Tay, Tracey Yee Hsin and Yan, Xu and Ouyang, Jonathan and Wu, Daniel and Jiang, William and Kao, Jonathan C. and Cui, Yuchen},
  howpublished = {arXiv (Cornell University)},
  year      = {2026},
  doi       = {10.48550/arxiv.2601.05336},
  url       = {https://doi.org/10.48550/arxiv.2601.05336}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology

  • eye gaze tracking
  • user intention recognition
  • vision-language models
  • foundation models
  • robotic manipulation
  • human-robot interaction

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Tay, T. Y. H., Yan, X., Ouyang, J., Wu, D., Jiang, W., Kao, J. C., & Cui, Y. (2026). Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2601.05336

How many times has this paper been cited?

9 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Tracey Yee Hsin Tay, Xu Yan, Jonathan Ouyang, Daniel Wu, William Jiang, Jonathan C. Kao, Yuchen Cui.

What is the DOI of this paper?

10.48550/arxiv.2601.05336, permanently at https://doi.org/10.48550/arxiv.2601.05336. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.