偏最小二乘回归

偏最小二乘的一般多元底层模型是

${\displaystyle X=TP^{\top }+E}$ 

${\displaystyle Y=UQ^{\top }+F}$ 

其中${\displaystyle X}$ 是一个${\displaystyle n\times m}$ 的预测矩阵，${\displaystyle Y}$ 是一个${\displaystyle n\times p}$ 的响应矩阵；${\displaystyle T}$ 和${\displaystyle U}$ 是${\displaystyle n\times l}$ 的矩阵，分别为${\displaystyle X}$ 的投影（“X分数”、“组件”或“因子”矩阵）和${\displaystyle Y}$ 的投影（“Y分数”）；${\displaystyle P}$ 和${\displaystyle Q}$ 分别是${\displaystyle m\times l}$ 和${\displaystyle p\times l}$ 的正交载荷矩阵，以及矩阵${\displaystyle E}$ 和${\displaystyle F}$ 是错误项，假设是独立同分布的随机正态变量。对${\displaystyle X}$ 和${\displaystyle Y}$ 分解来最大化${\displaystyle T}$ 和${\displaystyle U}$ 之间的协方差。

偏最小二乘的许多变量是为了估计因子和载荷矩阵${\displaystyle T,U,P}$ 和${\displaystyle Q}$ 。它们中大多数构造了${\displaystyle X}$ 和${\displaystyle Y}$ 之间线性回归的估计${\displaystyle Y=X{\tilde {B}}+{\tilde {B}}_{0}}$ 。一些偏最小二乘算法只适合${\displaystyle Y}$ 是一个列向量的情况，而其它的算法则处理了${\displaystyle Y}$ 是一个矩阵的一般情况。算法也根据他们是否估计因子矩阵${\displaystyle T}$ 为一个正交矩阵而不同。^{[5][6][7][8][9][10]} 最后的预测在所有不同最小二乘算法中都是一样的，但组件是不同的。

PLS1

PLS1是一个${\displaystyle Y}$ 是向量时广泛使用的算法。它估计${\displaystyle T}$ 是一个正交矩阵。以下是伪代码（大写字母是矩阵，带上标的小写字母是向量，带下标的小写字母和单独的小写字母都是标量）：

 1  function PLS1(${\displaystyle X,y,l}$ )
 2  ${\displaystyle X^{(0)}\gets X}$ 
 3  ${\displaystyle w^{(0)}\gets X^{T}y/||X^{T}y||}$ , an initial estimate of ${\displaystyle w}$ .
 4  ${\displaystyle t^{(0)}\gets Xw^{(0)}}$  
 5  for ${\displaystyle k}$  = 0 to ${\displaystyle l}$ 
 6      ${\displaystyle t_{k}\gets {t^{(k)}}^{T}t^{(k)}}$  (note this is a scalar)
 7      ${\displaystyle t^{(k)}\gets t^{(k)}/t_{k}}$ 
 8      ${\displaystyle p^{(k)}\gets {X^{(k)}}^{T}t^{(k)}}$ 
 9      ${\displaystyle q_{k}\gets {y}^{T}t^{(k)}}$  (note this is a scalar)
10      if ${\displaystyle q_{k}}$  = 0
11          ${\displaystyle l\gets k}$ , break the for loop
12      if ${\displaystyle k<l}$ 
13          ${\displaystyle X^{(k+1)}\gets X^{(k)}-t_{k}t^{(k)}{p^{(k)}}^{T}}$ 
14          ${\displaystyle w^{(k+1)}\gets {X^{(k+1)}}^{T}y}$ 
15          ${\displaystyle t^{(k+1)}\gets X^{(k+1)}w^{(k+1)}}$ 
16  end for
17  define ${\displaystyle W}$  to be the matrix with columns ${\displaystyle w^{(0)},w^{(1)},...,w^{(l-1)}}$ .
    Do the same to form the ${\displaystyle P}$  matrix and ${\displaystyle q}$  vector.
18  ${\displaystyle B\gets W{(P^{T}W)}^{-1}q}$ 
19  ${\displaystyle B_{0}\gets q_{0}-{P^{(0)}}^{T}B}$ 
20  return ${\displaystyle B,B_{0}}$ 

这种形式的算法不需要输入${\displaystyle X}$ 和${\displaystyle Y}$ 定中心，因为算法隐式处理了。这个算法的特点是收缩于${\displaystyle X}$  （减去${\displaystyle t_{k}t^{(k)}{p^{(k)}}^{T}}$ ），但向量${\displaystyle y}$ 不收缩，因为没有必要（可以证明收缩${\displaystyle y}$ 和不收缩的结果是一样的）。用户提供的变量${\displaystyle l}$ 是回归中隐藏因子数量的限制；如果它等于矩阵${\displaystyle X}$ 的秩，算法将产生${\displaystyle B}$ 和${\displaystyle B_{0}}$ 的最小二乘回归估计。

2002年，一个叫做正交投影（英语：Orthogonal Projections to Latent Structures， OPLS）的方法提出。在OPLS中，连续变量数据被分为预测的和不相关的信息。这有利于改进诊断，以及更容易解释可视化。然而，这些变化只是改善模型的可解释性，不是生产力。^[11] L-PLS通过3个连接数据块扩展了偏最小二乘回归。^[12] 同样，OPLS-DA（英语：Discriminant Analysis， 判别分析）可能被应用在处理离散变量，如分类和生物标志物的研究。

大多数统计软件包都提供偏最小二乘回归。^{[来源请求]} R中的‘pls’包提供了一系列算法。^[13]

• 特征提取
• 数据挖掘
• 机器学习
• 回归分析
• 典型相关
• Deming regression
• 多线性子空间学习
• 主成分分析
• 总平方和

• Kramer, R. Chemometric Techniques for Quantitative Analysis. Marcel-Dekker. 1998. ISBN 0-8247-0198-4. 
• Frank, Ildiko E.; Friedman, Jerome H. A Statistical View of Some Chemometrics Regression Tools. Technometrics. 1993, 35 (2): 109–148 [2015-09-28]. doi:10.1080/00401706.1993.10485033. （原始内容存档于2013-02-03）. 
• Haenlein, Michael; Kaplan, Andreas M. A Beginner's Guide to Partial Least Squares Analysis. Understanding Statistics. 2004, 3 (4): 283–297. doi:10.1207/s15328031us0304_4. 
• Henseler, Joerg; Fassott, Georg. Testing Moderating Effects in PLS Path Models. An Illustration of Available Procedures. 2005. 
• Lingjærde, Ole-Christian; Christophersen, Nils. Shrinkage Structure of Partial Least Squares. Scandinavian Journal of Statistics. 2000, 27 (3): 459–473. doi:10.1111/1467-9469.00201. 
• Tenenhaus, Michel. La Régression PLS: Théorie et Pratique. Paris: Technip.. 1998. 
• Rosipal, Roman; Kramer, Nicole. Overview and Recent Advances in Partial Least Squares, in Subspace, Latent Structure and Feature Selection Techniques: 34–51. 2006. 
• Helland, Inge S. PLS regression and statistical models. Scandinavian Journal of Statistics. 1990, 17 (2): 97–114. JSTOR 4616159. 
• Wold, Herman. Estimation of principal components and related models by iterative least squares. Krishnaiaah, P.R. (编). Multivariate Analysis. New York: Academic Press. 1966: 391–420. 
• Wold, Herman. The fix-point approach to interdependent systems. Amsterdam: North Holland. 1981. 
• Wold, Herman. Partial least squares. Kotz, Samuel; Johnson, Norman L. (编). Encyclopedia of statistical sciences 6. New York: Wiley. 1985: 581–591. 
• Wold, Svante; Ruhe, Axel; Wold, Herman; Dunn, W.J. The collinearity problem in linear regression. the partial least squares (PLS) approach to generalized inverses. SIAM Journal on Scientific and Statistical Computing. 1984, 5 (3): 735–743. doi:10.1137/0905052. 
• Garthwaite, Paul H. An Interpretation of Partial Least Squares. Journal of the American Statistical Association. 1994, 89 (425): 122–7. JSTOR 2291207. doi:10.1080/01621459.1994.10476452. 
• Wang, H. (编). Handbook of Partial Least Squares. 2010. ISBN 978-3-540-32825-4. 
• Stone, M.; Brooks, R.J. Continuum Regression: Cross-Validated Sequentially Constructed Prediction embracing Ordinary Least Squares, Partial Least Squares and Principal Components Regression. Journal of the Royal Statistical Society, Series B. 1990, 52 (2): 237–269. JSTOR 2345437. 
• Wan Mohamad Asyraf Bin Wan Afthanorhan. (2013). A Comparison Of Partial Least Square Structural Equation Modeling (PLS-SEM) and Covariance Based Structural EquationModeling (CB-SEM) for Confirmatory Factor Analysis International Journal of Engineering Science and Innovative Technology (IJESIT), 2(5), 9.

• imDEV （页面存档备份，存于互联网档案馆） free Excel add-in for PLS and PLS-DA
• PLS in Brain Imaging
• on-line PLS （页面存档备份，存于互联网档案馆） regression (PLSR) at Virtual Computational Chemistry Laboratory
• Uncertainty estimation for PLS （页面存档备份，存于互联网档案馆）
• A short introduction to PLS regression and its history （页面存档备份，存于互联网档案馆）