Linear-Regression - Shuai-Xie -Github
多變量線性回歸 預(yù)測房價
ex1data2.txt（房屋尺寸脐瑰，臥室數(shù)量朴恳，房屋價格）
2104,3,399900
1600,3,329900
2400,3,369000
1416,2,232000
3000,4,539900
1985,4,299900
……
Part 1: Feature Normalization
對于多維特征闹击，要使用特征縮放保證其具有相近尺度，這樣梯度下降算法會更快收斂贩疙。這個例子中掀泳，房屋尺寸是臥室數(shù)量接近1000倍。
%% ================ Part 1: Feature Normalization ================
fprintf('Loading data ...\n');
data = load('ex1data2.txt');
X = data(:, 1:2); % 房屋特征2個 (m*2)
y = data(:, 3); % 價格 (m*1)
m = length(y); % 樣本數(shù)量
% 輸出10個樣本點
fprintf('First 10 examples from the dataset: \n');
% X 2個特征跪腹，y 1個價格
% 因為fprintf按列輸出矩陣褂删，所以要將矩陣先轉(zhuǎn)置，使得每一列對應(yīng)一個實例
fprintf(' x = [%.0f %.0f], y = %.0f \n', [X(1:10,:) y(1:10,:)]');
% fprintf(' x = [%.0f %.0f], y = %.0f \n', data(1:10, :)'); % 也可以
fprintf('Program paused. Press enter to continue.\n');
pause;
% 特征縮放
fprintf('Normalizing Features ...\n');
% X是特征縮放后的特征矩陣冲茸，mu是平均值矩陣屯阀，sigma是方差矩陣
[X, mu, sigma] = featureNormalize(X);
X = [ones(m, 1), X]; % X 補充 theta0
% 格式化輸出特征縮放后的矩陣
fprintf(' x = [%.6f %.6f %.6f], y = %.0f \n', [X(1:10,:) y(1:10,:)]');
fprintf('Program paused. Press enter to continue.\n');
pause;
Loading data ...
First 10 examples from the dataset:
x = [2104 3], y = 399900
x = [1600 3], y = 329900
x = [2400 3], y = 369000
x = [1416 2], y = 232000
x = [3000 4], y = 539900
x = [1985 4], y = 299900
x = [1534 3], y = 314900
x = [1427 3], y = 198999
x = [1380 3], y = 212000
x = [1494 3], y = 242500
Program paused. Press enter to continue.
Normalizing Features ...
x = [1.000000 0.130010 -0.223675], y = 399900
x = [1.000000 -0.504190 -0.223675], y = 329900
x = [1.000000 0.502476 -0.223675], y = 369000
x = [1.000000 -0.735723 -1.537767], y = 232000
x = [1.000000 1.257476 1.090417], y = 539900
x = [1.000000 -0.019732 1.090417], y = 299900
x = [1.000000 -0.587240 -0.223675], y = 314900
x = [1.000000 -0.721881 -0.223675], y = 198999
x = [1.000000 -0.781023 -0.223675], y = 212000
x = [1.000000 -0.637573 -0.223675], y = 242500
Program paused. Press enter to continue.
featureNormalize 函數(shù)
function [X_norm, mu, sigma] = featureNormalize(X)
% 標準化 X 的特征到[0,1]之間
% 初始化
X_norm = X; % X(47*2)
% 利用matlab自帶函數(shù)mean缅帘，std求平均值和標準差
mu = mean(X); % mu(1*2) a row vector containing the mean value of each column.
sigma = std(X); % sigma(1*2)
% size(X, 1) 獲得 X 矩陣的行數(shù)，即樣本數(shù) = 47
% size(X, 2) 獲得 X 矩陣的列數(shù)难衰，即特征數(shù) = 2
m = size(X, 1);
% 縮放 X 的 2 個特征
for i = 1:m
X_norm(i, 1) = ( X_norm(i, 1) - mu(1) ) / sigma(1);
X_norm(i, 2) = ( X_norm(i, 2) - mu(2) ) / sigma(2);
end
end
特征縮放
Part 2: Gradient Descent
%% ================ Part 2: Gradient Descent ================
fprintf('Running gradient descent ...\n');
% Choose some alpha value 3個學(xué)習(xí)率
alpha = 0.01; % 學(xué)習(xí)率
alpha1 = 0.03;
alpha2 = 0.1;
num_iters = 400; % 迭代次數(shù)
% Init Theta and Run Gradient Descent
theta = zeros(3, 1); % 3個特征
theta1 = zeros(3, 1);
theta2 = zeros(3, 1);
[theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters); % 多變量梯度下降
[theta1, J_history1] = gradientDescentMulti(X, y, theta1, alpha1, num_iters);
[theta2, J_history2] = gradientDescentMulti(X, y, theta2, alpha2, num_iters);
% Plot the convergence graph
figure;
plot(1:numel(J_history), J_history, '-b', 'LineWidth', 2);
hold on; % 繼續(xù)畫圖
plot(1:numel(J_history1), J_history1, '-r', 'LineWidth', 2);
plot(1:numel(J_history2), J_history2, '-k', 'LineWidth', 2);
% numel: number of elements 返回元素個數(shù)钦无，作為x軸迭代次數(shù)
xlabel('Number of iterations');
ylabel('Cost J');
legend('alpha = 0.01', 'alpha = 0.03', 'alpha = 0.1'); % 圖例
% 顯示theta值
fprintf('Theta computed from gradient descent: \n');
fprintf(' %f \n', theta); % 3*1
% 估計房屋價格 1650 sq-ft, 3 br house
% x1不需要特征縮放為1，將后面的x2盖袭，x3特征縮放
% price = 0; % You should change this
price = [1, (1650 - mu(1,1)) / sigma(1,1), (3 - mu(1,2)) / sigma(1,2)] * theta;
fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
'(using gradient descent): $%f\n'], price);
fprintf('Program paused. Press enter to continue.\n');
pause;
Running gradient descent ...
Theta computed from gradient descent:
334302.063993
100087.116006
3673.548451
Predicted price of a 1650 sq-ft, 3 br house (using gradient descent): $289314.620338
Program paused. Press enter to continue.
不同學(xué)習(xí)率下的收斂曲線
gradientDescentMulti 函數(shù)
function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters)
% Initialize some useful values
J_history = zeros(num_iters, 1);
m = length(y);
% 迭代
for iter = 1:num_iters
tmp = (X' * ( X * theta - y )); % 向量運算 求和項 (3*m)*(m*1)=(3*1)
theta = theta - alpha / m * tmp;
% Save the cost J in every iteration
J_history(iter) = computeCostMulti(X, y, theta); % 根據(jù)theta求J
end
computeCostMulti 函數(shù)
function J = computeCostMulti(X, y, theta)
% Initialize some useful values
m = length(y); % number of training examples
E = X * theta - y; % 偏差矩陣(m*1)
J = (E' * E) / (2 * m); % (E'*E) 方差和
end
Part 3: Normal Equations
正規(guī)方程求 θ
%% ================ Part 3: Normal Equations ================
fprintf('Solving with normal equations...\n');
% Load Data
data = csvread('ex1data2.txt'); % load也可以
X = data(:, 1:2);
y = data(:, 3);
m = length(y);
% Add intercept term to X
X = [ones(m, 1) X];
% Calculate the parameters from the normal equation
theta = normalEqn(X, y);
% Display normal equation result
fprintf('Theta computed from the normal equations: \n');
fprintf(' %f \n', theta);
% Estimate the price of a 1650 sq-ft, 3 br house
% 不需要特征縮放
price = [1, 1650, 3] * theta;
fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
'(using normal equations): $%f\n'], price);
預(yù)測結(jié)果與梯度下降法有出入失暂，正規(guī)預(yù)測的是準確值，梯度下降是遞進值鳄虱。
Solving with normal equations...
Theta computed from the normal equations:
89597.909544
139.210674
-8738.019113
Predicted price of a 1650 sq-ft, 3 br house (using normal equations): $293081.464335
normalEqn 函數(shù)
function [theta] = normalEqn(X, y)
theta = pinv(X' * X) * X' * y; % 調(diào)用方程
end
