3.14 - An Elemental Perspective on Partial Least Squares

Partial Least Squares (PLS) originated with Herman Wold’s Nonlinear Iterative Partial Least Squares (NIPALS) algorithm which linearized models that were nonlinear in the parameters. Subsequently, S. Wold, Martens, and H. Wold adapted the NIPALS method for the overdetermined regression problem and termed their method “partial least squares”. Although many standard software packages implement PLS by way the original NIPALS algorithm (or by a similar algorithm), statisticians largely abandoned that perspective in favor of versions of the same paradigm, as discussed first in Stone and Brooks, that connect PLS to classical multivariate statistical theory and allow for the implementation of PLS by way of solutions to well-posed eigenstructure problems. This eigenstructure perspective lays bare the compromise PLS strikes between summary and purpose, and the clarity provided by that concise mathematical perspective has enabled useful adaptations in a manner that would not have been possible with just an algorithmic presentation. This entry traces the development of PLS and some of those adaptations. Examples are provided when appropriate.