Generative adversarial networks (GAN) model for dynamically adjusted weld pool image toward human-based model predictive control (MPC)

Gas Metal Arc Welding (GMAW) is a critical industrial technique known for its high productivity, flexibility, and adaptability to automation. Despite the significant advancements in robotic welding, challenges remain in fully automating the arc welding process, particularly due to the complex dynamics of the weld pool associated with GMAW. A human-robot collaborative (HRC) system where humans operate robots may conveniently provide the needed adaptive control to the complex GMAW. While in conventional HRC systems humans receive process feedback to make adaptive adjustments, we propose provide humans with predictive future feedback to further ease the human decision and reduce the needed skills/trainings. To this end, this study explores the integration of deep learning models, specifically Generative Adversarial Networks (GANs) combined with Gated Recurrent Units (GRUs), to model and predict the dynamic behavior of the weld pool during GMAW. By leveraging time-series data of torch movement and corresponding weld pool images, the proposed GRU-GAN model generates high-fidelity weld pool images, capturing the intricate relationship between speed variations and weld pool morphology. Through extensive experimentation, including the design of an acceptable Encoder-Decoder structure for the GAN, we demonstrate that incorporating both temporal and speed sequence information significantly enhances the model's predictive capabilities. The findings validate the hypothesis that dynamic torch speed adjustments, akin to those performed by skilled human welders, can be effectively modeled to improve the quality of automated welding processes. Future work will be devoted to human-based model predictive control (MPC) in an HRC environment.