machine-learning-ex3.zip 下載鏈接，第四周的課程相對(duì)來(lái)說(shuō)比較簡(jiǎn)單混聊，大致介紹了神經(jīng)網(wǎng)絡(luò)相關(guān)內(nèi)容。更多見(jiàn)：iii.run
注意乾巧，官方用的是matlab句喜，這里我用的全部是python，代碼是不一樣的沟于，更不能當(dāng)做作業(yè)提交咳胃。
Programming Exercise 3 - Multi-class Classification and Neural Networks
題目介紹
For this exercise, you will use logistic regression and neural networks to recognize handwritten digits (from 0 to 9).
ex3data1.mat提供了一個(gè)訓(xùn)練集，X包含5000個(gè)長(zhǎng)度為400的向量旷太，每個(gè)向量可以展示為一個(gè)20*20的圖像拙绊。Y內(nèi)為圖像所對(duì)應(yīng)的數(shù)字。
Code
# %load ../../standard_import.txt
import pandas as pd
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
# load MATLAB files
from scipy.io import loadmat
from scipy.optimize import minimize
from sklearn.linear_model import LogisticRegression
pd.set_option('display.notebook_repr_html', False)
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', 150)
pd.set_option('display.max_seq_items', None)
#%config InlineBackend.figure_formats = {'pdf',}
%matplotlib inline
import seaborn as sns
sns.set_context('notebook')
sns.set_style('white')
Load MATLAB datafiles
data = loadmat('data/ex3data1.mat')
data.keys()
dict_keys(['__header__', '__version__', '__globals__', 'X', 'y'])
weights = loadmat('data/ex3weights.mat')
weights.keys()
dict_keys(['__header__', '__version__', '__globals__', 'Theta1', 'Theta2'])
y = data['y']
# Add constant for intercept
X = np.c_[np.ones((data['X'].shape[0],1)), data['X']]
print('X: {} (with intercept)'.format(X.shape))
print('y: {}'.format(y.shape))
X: (5000, 401) (with intercept)
y: (5000, 1)
theta1, theta2 = weights['Theta1'], weights['Theta2']
print('theta1: {}'.format(theta1.shape))
print('theta2: {}'.format(theta2.shape))
theta1: (25, 401)
theta2: (10, 26)
sample = np.random.choice(X.shape[0], 20)
plt.imshow(X[sample,1:].reshape(-1,20).T)
plt.axis('off');
Multiclass Classification
Logistic regression hypothesis
{% raw %} $$ h_{\theta}(x) = g(\theta^{T}x)$${% endraw %}
{% raw %} $$ g(z)=\frac{1}{1+e^{?z}} $${% endraw %}
def sigmoid(z):
return(1 / (1 + np.exp(-z)))
Regularized Cost Function
{% raw %} $$ J(\theta) = \frac{1}{m}\sum_{i=1}{m}\big[-y{(i)}, log,( h_\theta,(x{(i)}))-(1-y{(i)}),log,(1-h_\theta(x^{(i)}))\big] + \frac{\lambda}{2m}\sum_{j=1}{n}\theta_{j}{2}$${% endraw %}
Vectorized Cost Function
{% raw %} $$ J(\theta) = \frac{1}{m}\big((,log,(g(X\theta))Ty+(,log,(1-g(X\theta))T(1-y)\big) + \frac{\lambda}{2m}\sum_{j=1}{n}\theta_{j}{2}$${% endraw %}
def lrcostFunctionReg(theta, reg, X, y):
m = y.size
h = sigmoid(X.dot(theta))
J = -1*(1/m)*(np.log(h).T.dot(y)+np.log(1-h).T.dot(1-y)) + (reg/(2*m))*np.sum(np.square(theta[1:]))
if np.isnan(J[0]):
return(np.inf)
return(J[0])
def lrgradientReg(theta, reg, X,y):
m = y.size
h = sigmoid(X.dot(theta.reshape(-1,1)))
grad = (1/m)*X.T.dot(h-y) + (reg/m)*np.r_[[[0]],theta[1:].reshape(-1,1)]
return(grad.flatten())
One-vs-all
One-vs-all Classification
def oneVsAll(features, classes, n_labels, reg):
initial_theta = np.zeros((X.shape[1],1)) # 401x1
all_theta = np.zeros((n_labels, X.shape[1])) #10x401
for c in np.arange(1, n_labels+1):
res = minimize(lrcostFunctionReg, initial_theta, args=(reg, features, (classes == c)*1), method=None,
jac=lrgradientReg, options={'maxiter':50})
all_theta[c-1] = res.x
return(all_theta)
theta = oneVsAll(X, y, 10, 0.1)
One-vs-all Prediction
def predictOneVsAll(all_theta, features):
probs = sigmoid(X.dot(all_theta.T))
# Adding one because Python uses zero based indexing for the 10 columns (0-9),
# while the 10 classes are numbered from 1 to 10.
return(np.argmax(probs, axis=1)+1)
pred = predictOneVsAll(theta, X)
print('Training set accuracy: {} %'.format(np.mean(pred == y.ravel())*100))
Training set accuracy: 93.24 %
Multiclass Logistic Regression with scikit-learn
clf = LogisticRegression(C=10, penalty='l2', solver='liblinear')
# Scikit-learn fits intercept automatically, so we exclude first column with 'ones' from X when fitting.
clf.fit(X[:,1:],y.ravel())
LogisticRegression(C=10, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
verbose=0, warm_start=False)
pred2 = clf.predict(X[:,1:])
print('Training set accuracy: {} %'.format(np.mean(pred2 == y.ravel())*100))
Training set accuracy: 96.5 %
Neural Networks
def predict(theta_1, theta_2, features):
z2 = theta_1.dot(features.T)
a2 = np.c_[np.ones((data['X'].shape[0],1)), sigmoid(z2).T]
z3 = a2.dot(theta_2.T)
a3 = sigmoid(z3)
return(np.argmax(a3, axis=1)+1)
pred = predict(theta1, theta2, X)
print('Training set accuracy: {} %'.format(np.mean(pred == y.ravel())*100))
Training set accuracy: 97.52 %
Summary
展示圖片
因?yàn)榻o的數(shù)據(jù)是矩陣的形式泳秀，所以需要分為單個(gè)向量标沪，重新設(shè)置形狀。然后展示出來(lái)嗜傅，跟手寫(xiě)字體識(shí)別是很像金句。
%matplotlib inline
import scipy.io as sio
import matplotlib.pyplot as plt
matfn='C:/Users/wing/Documents/MATLAB/ex3/ex3data1.mat'
data=sio.loadmat(matfn)
im = data['X'][0]
im = im.reshape(20,20)
plt.imshow(im , cmap='gray')
如果想要一次展示多個(gè)圖像呢，寫(xiě)個(gè)循環(huán)就可以了吕嘀。不過(guò)太多的話违寞，可能就看不清了。偶房。
im = []
for i in range(500):
plt.subplot(10, 50, i + 1)
temp = data['X'][i]
im.append(temp.reshape(20,20))
plt.imshow(im[i], cmap='gray')
plt.show()
reshape內(nèi)-1的用法
官方文檔：numpy.reshape - NumPy v1.11 Manual
>>> a = np.array([[1,2,3], [4,5,6]])
>>> np.reshape(a, (3,-1)) # the unspecified value is inferred to be 2
array([[1, 2],
[3, 4],
[5, 6]])
-1表示我懶得計(jì)算該填什么數(shù)字趁曼，由python通過(guò)a和其他的值推測(cè)出來(lái)。
LogisticRegression
from sklearn.linear_model import LogisticRegression
http://www.cnblogs.com/xupeizhi/archive/2013/07/05/3174703.html
http://scikit-learn.org/stable/index.html
