How to Accurately Monitor the Weld Penetration From Dynamic Weld Pool Serial Images Using CNN-LSTM Deep Learning Model?

Rui Yu, Joseph Kershaw, Peng Wang, Yu Ming Zhang

Research output: Contribution to journalArticlepeer-review

24 Scopus citations


This letter illustrates how a challenging problem be solved by assuring the adequacy of the raw information and using an appropriate deep learning network to extract the relevant information. The problem concerned is accurate monitoring of the penetration, in a fully penetrated weld pool, as quantified by the width of the weld on the back-side of the workpiece. This is challenging as the penetration occurs below the workpiece surface and is not visible. A popular method is to use a weld pool image to derive it. Analysis of the physical process suggests that a single weld pool does not contain adequate information but most recent serial weld pools may. As such, although a deep learning model may extract information that is already there, the raw information may not be sufficient. Hence, a model that is capable of extracting information from dynamic serial weld pool images is needed. To this end, a CNN-LSTM (convolutional neural network combined with long-short term memory one) model is proposed. Dynamic weld pools are experimentally generated by changing the welding current and welding speed randomly. The weld pools are imaged using an HDR camera during experiments. Images are also captured from the back-side surface of the workpiece to provide the ground truth for training, validation, and testing. It is found that the highly dynamically changing weld pool can be accurately predicted using serial weld pool images at 0.3 mm for its back-side bead width. Comparison has been made with results from comparative studies to verify the effectiveness of and the contribution from the information adequacy (by using serial images) and the feature extracting capability (by using deep learning).

Original languageEnglish
Pages (from-to)6519-6525
Number of pages7
JournalIEEE Robotics and Automation Letters
Issue number3
StatePublished - Jul 1 2022

Bibliographical note

Publisher Copyright:
© 2016 IEEE.


  • CNN
  • HDR image
  • LSTM
  • Weld
  • deep earning
  • weld penetration

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'How to Accurately Monitor the Weld Penetration From Dynamic Weld Pool Serial Images Using CNN-LSTM Deep Learning Model?'. Together they form a unique fingerprint.

Cite this