多項式擬合和最小二乘問題

背景知識

The method of least squares is a standard approach in regression analysis to the approximate solution of overdetermined systems, i.e., sets of equations in which there are more equations than unknowns. "Least squares" means that the overall solution minimizes the sum of the squares of the errors made in the results of every single equation.

多項式擬合

The first clear and concise exposition of the method of least squares was published by Legendre in 1805.[5] The technique is described as an algebraic procedure for fitting linear equations to data and Legendre demonstrates the new method by analyzing the same data as Laplace for the shape of the earth. The value of Legendre's method of least squares was immediately recognized by leading astronomers and geodesists of the time.

實驗一

%%給定數據對，(x0,y0)
x0=0:0.1:1;
y0=[-.447,1.978,3.11,5.25,5.02,4.66,4.01,4.58,3.45,5.35,9.22];
%%求擬合多項式
n=3;%假設數據對(x0,y0)一共有m對，則一般狀況下，應當保證n<m.觀察數據圖形的曲線，有助於適當階數的肯定。
%%求得x0,y0數組所給數據的n階擬合多項式係數向量p,爲了保證較好的擬合效果，多項式的階數要取得適當，太低，殘差較大；太高,擬合模型將包含噪聲影響。
P=polyfit(x0,y0,n)
%圖示擬合狀況
xx=0:0.01:1;
yy=polyval(P,xx);
% y = polyval(p,x) returns the value of a polynomial of degree n evaluated
% at x. The input argument p is a vector of length n+1 whose elements are
% the coefficients in descending powers of the polynomial to be evaluated.
% 
% y = p1xn + p2xn–1 + … + pnx + pn+1
plot(xx,yy,'-b',x0,y0,'.r','MarkerSize',20)
legend('擬合曲線','原始數據','Location','SouthEast')
xlabel('x')

最小二乘

Least squares problems fall into two categories: linear or ordinary least squares and non-linear least squares, depending on whether or not the residuals are linear in all unknowns. The linear least-squares problem occurs in statistical regression analysis; it has a closed-form solution. The non-linear problem is usually solved by iterative refinement; at each iteration the system is approximated by a linear one, and thus the core calculation is similar in both cases.

實驗二

%%給定自變量x數據組
x0=(0:0.1:1);
%%給定自變量y數據組
y0=[-.447,1.978,3.11,5.25,5.02,4.66,4.01,4.68,4.45,5.35,9.22]';
m=length(x0);
n=3;
%多項式階數
%%初始化數據陣X
X=zeros(m,n+1);
%%將X設置成各列等比的形式
for k=1:n
    X(:,n-k+1)=(x0.^k);
end
X(:,n+1)=ones(m,1);
%%利用最小二乘原理求解多項式係數
aT=(X\y0)'

aT =    51.9713  -80.2234   37.7844   -0.8078