Preprint · 2024 · Open access

Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation

Noah Becker, Sovailo, Kyrylo, Zhu, Chunyao, Erik Gattung, Kay Hansel, Tim Schneider, Yaonan Zhu, Yasuhisa Hasegawa, Jan Peters

arXiv (Cornell University) · Published 30 April 2024 · DOI 10.48550/arxiv.2404.19585

Open PDFView at the publisherCite this paper

In brief

“Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation” is a 2024 preprint by Noah Becker and 8 co-authors, published in arXiv (Cornell University). OpenAlex counts 13 citations as of 22 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).

Citations
13
Most cited in
2025 (7)
References
23
Authors
9

Abstract

Telerobotics enables humans to overcome spatial constraints and physically interact with the environment in remote locations. However, the sensory feedback provided by the system to the user is often purely visual, limiting the user's dexterity in manipulation tasks. This work addresses this issue by equipping the robot's end-effector with high-resolution visuotactile GelSight sensors. Using low-cost MANUS-Gloves, we provide the user with haptic feedback about forces acting at the points of contact in the form of vibration signals. We employ two different methods for estimating these forces; one based on estimating the movement of markers on the sensor surface and one deep-learning approach. Additionally, we integrate our system into a virtual-reality teleoperation pipeline in which a human user controls both arms of a Tiago robot while receiving visual and haptic feedback. Lastly, we present a novel setup to evaluate normal force, shear force, and slip. We believe that integrating haptic feedback is a crucial step towards dexterous manipulation in teleoperated robotic systems.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Becker, N., Kyrylo, S., Chunyao, Z., Gattung, E., Hansel, K., Schneider, T., Zhu, Y., Hasegawa, Y., & Peters, J. (2024). Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2404.19585

MLA 9

Becker, Noah, et al. “Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation.” arXiv (Cornell University), 2024. https://doi.org/10.48550/arxiv.2404.19585.

Chicago

Becker, Noah, Sovailo, Kyrylo, Zhu, Chunyao, Erik Gattung, Kay Hansel, Tim Schneider, Yaonan Zhu, Yasuhisa Hasegawa, and Jan Peters. 2024. “Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2404.19585.

Harvard

Becker, N. et al. (2024) ‘Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation’, arXiv (Cornell University). doi:10.48550/arxiv.2404.19585.

IEEE

N. Becker et al., “Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation,” arXiv (Cornell University), 2024, doi: 10.48550/arxiv.2404.19585.

BibTeX

@misc{becker2024integrating,
  title     = {{Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation}},
  author    = {Becker, Noah and Kyrylo, Sovailo, and Chunyao, Zhu, and Gattung, Erik and Hansel, Kay and Schneider, Tim and Zhu, Yaonan and Hasegawa, Yasuhisa and Peters, Jan},
  howpublished = {arXiv (Cornell University)},
  year      = {2024},
  doi       = {10.48550/arxiv.2404.19585},
  url       = {https://doi.org/10.48550/arxiv.2404.19585}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202422
202577
202644

Topic and keywords

Physical Sciences › Engineering › Mechanical Engineering › Teleoperation and Haptic Systems

  • haptic feedback
  • teleoperation
  • telerobotics
  • GelSight
  • force estimation
  • virtual reality teleoperation
  • TIAGo robot
  • dexterous manipulation

Work with this paper

Most-cited papers in Engineering

All highly cited Engineering papers

Questions about this paper

How do I cite this paper in APA style?

Becker, N., Kyrylo, S., Chunyao, Z., Gattung, E., Hansel, K., Schneider, T., Zhu, Y., Hasegawa, Y., & Peters, J. (2024). Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2404.19585

How many times has this paper been cited?

13 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Noah Becker, Sovailo, Kyrylo, Zhu, Chunyao, Erik Gattung, Kay Hansel, Tim Schneider, Yaonan Zhu, Yasuhisa Hasegawa, Jan Peters.

What is the DOI of this paper?

10.48550/arxiv.2404.19585, permanently at https://doi.org/10.48550/arxiv.2404.19585. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.