介紹
- 注意：這里的代碼都是在Jupyter Notebook中運(yùn)行，原始的
.ipynb文件可以在我的GitHub主頁上下載 https://github.com/acphart/Hand_on_ML_Algorithm 其中的LinearRegression_single_variabel.ipynb枕屉，直接在Jupyter Notebook中打開運(yùn)行即可，里面還有其他的機(jī)器學(xué)習(xí)算法實(shí)現(xiàn)，喜歡可以順便給個(gè)star哦 ~~~
這里是實(shí)現(xiàn)單變量線性回歸算法的訓(xùn)練過程
import numpy as np
import matplotlib.pyplot as plt
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = 'all'
生成模擬數(shù)據(jù)
- 模擬數(shù)據(jù)的內(nèi)在模式是
)
- 也就是一個(gè)一次函數(shù)飘痛，我們在數(shù)據(jù)中加入隨機(jī)噪聲珊膜，就得到了模擬數(shù)據(jù)
np.random.seed(20180822)
m = 100
Theta = [[2.0], [2.9]]
x0 = np.ones((m,1))
x1 = np.linspace(-2, 5, m).reshape(m, 1)
X = np.hstack((x0, x1))
y = np.dot(X, Theta) + 2.0*np.random.randn(100,1)
_ = plt.scatter(x1, y)
損失函數(shù)、梯度函數(shù)
- 這是實(shí)現(xiàn)的主要工作宣脉，定義我們的損失函數(shù)和梯度函數(shù)
- 損失函數(shù)是
![Loss(\theta, X) = \frac{1}{2m}\sum_{i = 1}^m[h_\theta(x^{(i)}) - y^{(i)}]^2](https://math.jianshu.com/math?formula=Loss(%5Ctheta%2C%20X)%20%3D%20%5Cfrac%7B1%7D%7B2m%7D%5Csum_%7Bi%20%3D%201%7D%5Em%5Bh_%5Ctheta(x%5E%7B(i)%7D)%20-%20y%5E%7B(i)%7D%5D%5E2)
- 梯度函數(shù)是
![\frac{\partial}{\partial \theta_j} Loss(\theta, X) = \frac{1}{m}\sum_{i = 1}^m[h_\theta(x^{(i)}) - y^{(i)}]x_j^{(i)}](https://math.jianshu.com/math?formula=%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20%5Ctheta_j%7D%20Loss(%5Ctheta%2C%20X)%20%3D%20%5Cfrac%7B1%7D%7Bm%7D%5Csum_%7Bi%20%3D%201%7D%5Em%5Bh_%5Ctheta(x%5E%7B(i)%7D)%20-%20y%5E%7B(i)%7D%5Dx_j%5E%7B(i)%7D)
- 這里基本的優(yōu)化就是把循環(huán)求和操作向量化：
![Loss(\theta, X) = \frac{1}{2m}[h_\theta(X) - y]^T[h_\theta(X) - y]](https://math.jianshu.com/math?formula=Loss(%5Ctheta%2C%20X)%20%3D%20%5Cfrac%7B1%7D%7B2m%7D%5Bh_%5Ctheta(X)%20-%20y%5D%5ET%5Bh_%5Ctheta(X)%20-%20y%5D)
![\nabla Loss(\theta, X) = \frac{1}{m}X^T[h_\theta(X) - y]](https://math.jianshu.com/math?formula=%5Cnabla%20Loss(%5Ctheta%2C%20X)%20%3D%20%5Cfrac%7B1%7D%7Bm%7DX%5ET%5Bh_%5Ctheta(X)%20-%20y%5D)
def loss_func(X, y, theta,):
loss = np.dot(X, theta) - y
return 1./(2*m) * np.dot(loss.T, loss)
def grad_func(X, y, theta):
loss = np.dot(X, theta) - y
return 1./m * np.dot(X.T, loss)
梯度下降
- 我們的假設(shè)函數(shù)為：
%20%3D%20%5Ctheta_0%20x_0%2B%20%5Ctheta_1%20x_1%20%5Cquad%20(x_0%20%3D%201))
- 在梯度下降訓(xùn)練過程中车柠，我們利用顏色的深淺把擬合直線畫出來，我們可以看到擬合的效果越來越好
# 設(shè)置學(xué)習(xí)率和收斂停止的開關(guān)
alpha = 0.01
accuracy = 1e-5
# 初始化參數(shù)
theta = np.random.randn(2,1)*0.1
i = 1
index = 1
c = np.array([0.8, 0.8, 0.8]) # 設(shè)置顏色塑猖，顏色逐漸加深
grad = grad_func(X, y, theta) # 初始梯度
while not np.all(abs(grad) < accuracy):
theta = theta - alpha*grad
grad = grad_func(X, y, theta)
# 作出學(xué)習(xí)過程
i = i+1
if i%index == 0:
_ = plt.plot(x1, np.dot(X, theta), color=c)
c = c - 0.1
index = index*4
_ = plt.scatter(x1, y, alpha=0.7)
theta
- 輸出算法訓(xùn)練后擬合的參數(shù)竹祷，和我們的模式函數(shù)的參數(shù)很接近，想要更好的結(jié)果還可以修改收斂停止的開關(guān)羊苟。
array([[ 2.0953245],
[ 2.9086616]])
