Deep Learning-Based Detection of Penetration from Weld Pool Reflection Images

An innovative method was proposed to determine weld joint penetration using machine learning techniques. In our approach, the dot-structured laser images reflected from an oscillating weld pool surface were captured. Experienced welders typically evaluate the weld penetration status based on this reflected laser pattern. To overcome the challenges in identifying features and accurately processing the images using conventional machine vision algorithms, we proposed the use the raw images without any processing as the input to a convolutional neural network (CNN). The labels needed to train the CNN were the measured weld penetration states, obtained from the images on the backside of the workpiece as a set of discrete weld penetration categories. The raw data, images, and penetration state were generated from extensive experiments using an automated robotic gas tungsten arc welding process. Data augmentation was performed to enhance the robustness of the trained network, which led to 270,000 training examples, 45,000 validation examples, and 45,000 test examples. A six-layer convolutional neural network trained with a modified mini-batch gradient descent method led to a final testing accuracy of 90.7%. A voting mechanism based on three continuous images increased the classification accuracy to 97.6%.