Deep learning based real-time and in-situ monitoring of weld penetration: Where we are and what are needed revolutionary solutions?

Welding Procedure Designed assures the desired weld penetration be produced under nominal welding conditions. When conditions deviate from the nominal, penetration and other welding outcomes deviate from their desired/targeted ones. To assure the penetration not below minimally necessary, well-designed Procedure should reserve an appropriate margin per estimated condition deviations. An ideal solution is to use relatively small margin but dynamically adjust welding parameters to maintain the penetration at minimally necessary/within the margin. As such, the penetration state should be monitored in real-time during manufacturing. Unfortunately, it occurs underneath workpieces and is not considered directly observable during manufacturing so that its real-time in-situ monitoring is challenging. In the last half century, researchers have focused on finding promising real-time observable phenomena and correlating such phenomena to penetration. This has been difficult as it is unclear what are critical in observed phenomena and trial-and-error iterative processes are practiced proposing features, developing algorithms to calculate them, fitting model from features, and then modifying model, features, or algorithms if fitting accuracy is not acceptable. Such iterative process is not automated, finding/fitting right features/model takes months if not longer, and success is not assured. In particular, algorithms to calculate features vary from features to features and each of them requires extensive tests. Deep learning automates and combines featuring and fitting to maximally use the raw information directly to achieve highest accuracies. Computation is drastically increased but the iterative process is replaced by an automated one so that the time frame is still drastically reduced. This paper reviews various raw information that has been used as the observed phenomena to input into deep learning models and analyzes why they may correlate to penetration; reviews various deep learning models and analyzes why/how they may and are needed to correlate different raw information to penetration; and briefly reviews major techniques that have been used to train deep learning models in penetration prediction. Based on such analyses, this paper identifies major achievements and issues for efforts taken so far to monitor weld penetration using deep learning approaches. Finally, we identify two fundamental issues that require revolutionary solutions in order to move the deep learning technologies from laboratory studies to manufacturing as directions for future efforts. These two issues are analyzed and some preliminary solution directions are proposed.