Modeling of Human Welders' Operations in Virtual Reality Human-Robot Interaction

Qiyue Wang, Wenhua Jiao, Rui Yu, Michael T. Johnson, Yuming Zhang

Research output: Contribution to journalArticlepeer-review

25 Scopus citations


This letter presents a virtual reality (VR) human-robot interaction welding system that allows human welders to manipulate a welding robot and undertake welding tasks naturally and intuitively via consumer-grade VR hardware (HTC Vive). In this system, human welders' operations are captured by motion-tracked handle controllers and used as commands to teleoperate a 6-DoF industrial robot (UR-5) and to request welding current from a controllable welding power supply (Liburdi Pulsweld P200). The three-dimensional (3-D) working scene is rendered in real time based on feedback information and shown to the human welder by head-mounted display via a motion-tracked headset. To compensate for the time delay between command motion and real motion of the robot, a hidden Markov model is proposed to model and predict human welders' operations. The K-means clustering algorithm is applied to cluster human welders' operation data (traveling speed) into latent states. Based on the developed prediction algorithm, the motion of human welders is predicted with an root mean square error (RMSE) accuracy of between 2.1 and 4.6 mm/s. The position data used as final commands to teleoperate a robot are predicted with an RMSE accuracy of between 1.1 and 2.3 mm. This letter presents a general cyber-physical model for human-robot interactive welding based on VR, building a foundation for welding robot teleoperation.

Original languageEnglish
Article number8733861
Pages (from-to)2958-2964
Number of pages7
JournalIEEE Robotics and Automation Letters
Issue number3
StatePublished - Jul 2019

Bibliographical note

Publisher Copyright:
© 2016 IEEE.


  • Industrial robots
  • intelligent and flexible manufacturing
  • physical human-robot interaction
  • telerobotics and teleoperation
  • virtual reality and interfaces

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'Modeling of Human Welders' Operations in Virtual Reality Human-Robot Interaction'. Together they form a unique fingerprint.

Cite this