Multivariate Linear Regression and Programming Exercise 1 更多見(jiàn)：iii.run

Gradient Descent for Multiple Variables

• Suppose we have n variables, set hypothesis to be:

• Cost Function

  
  

• Gradient Descent Algorithm

  
  
  
  

  Get every feature into approximately [-1, 1]. Just normalize all the parameters :)

• Learning Rate:Not too big(fail to converge), not too small(too slow)

• Polynormal Regression:Use feature scalling. (Somewhat like normalizing dimension)

Programming Exercise 1

下載程序及相關(guān)數(shù)據(jù)

Stanford coursera Andrew Ng 機(jī)器學(xué)習(xí)課程編程作業(yè)（Exercise 1），作業(yè)下載鏈接貌似被墻了，下載鏈接放這些阅。
http://home.ustc.edu.cn/~mmmwhy/machine-learning-ex1.zip

重新推導(dǎo)一下：

其實(shí)這里一共就兩個(gè)式子：

• computeCost
  $$h_\theta (x) = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \theta_3 x_3 + \cdots + \theta_n x_n$$
  $$J(\theta_0,\theta_1 )=\frac1{2m} \sum_{i=1}^{{m}(h_{\theta}(x}{(i)})-y^{(i)})w$$

• gradientDescent
  $$\begin{align} \text{repeat until convergence: } \lbrace & \newline \theta_0 := & \theta_0 - \alpha \frac{1}{m} \sum\limits_{i=1}^{m}(h_\theta(x_{i}) - y_{i}) \newline \theta_1 := & \theta_1 - \alpha \frac1m \sum\limits_{i=1}^m\left((h_\theta(x_i) - y_i) x_i\right) \newline \rbrace& \end{align}$$

python擬合實(shí)現(xiàn)代碼

原本用的是matlab代碼韧骗，我用python實(shí)現(xiàn)了一下，結(jié)果是一樣的：

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt


def readfile(path):
    X=[]
    y=[]
    with open(path,'r') as f:
        for line in f:
            X.append([1,float(line.split(',')[0])])
            y.append(float(line.split(',')[1]))
    return X,y


def dataplot(x,theta,y):
    plt.plot(x, y, 'rx', markersize=10)
    plt.ylabel('Profit in $10,000s')
    plt.xlabel('Population of City in 10,000s')
    plt.plot(X[:,1],X*theta,'-')
    plt.show()


def computeCost(X,y,theta):
    m = len(y)
    J = 0
    for i in range(m):
        J = J + float((X[i]*theta-y[i])**2)
    return J/(2*m)

def gradientDescent(X, y, theta, alpha, num_iters):
    m = len(y)
    num_iters = 1500
    J_history = np.zeros(num_iters)
    for i in range(num_iters):
        S =X.T * (X * theta - np.mat(y).T) / m
        theta = theta - alpha * S;
        J_history[i] = computeCost(X,y,theta)
    return theta

if __name__=="__main__":
    theta = np.mat([[0],[0]])
    iterations = 1500
    alpha = 0.01
    iterations = 1500
    path = "C:\Users\wing\Documents\MATLAB\ex1\ex1data1.txt"
    
    x,y = readfile(path)# 小寫的X不是矩陣开缎，是list涕癣，大寫的X是矩陣。
    X = np.mat(x)
    J = computeCost(X, y, theta)
    theta = gradientDescent(X, y, theta, alpha, iterations)
    dataplot(X[:,1],theta,y)

輸出的圖有點(diǎn)小茁瘦，就這樣吧。

以上