上篇文章倒堕，自己推導了一遍誤差反向傳播算法砂蔽，從而對深度學習有了一定的認識碑定。這期試著自己搭建一個基于python的深度學習神經(jīng)網(wǎng)，來解決一定的問題又官。
需要用到：python3延刘、numpy
數(shù)據(jù)集：MNIST數(shù)據(jù)集（一個被”嚼爛”了的數(shù)據(jù)集, 很多教程都會對它”下手”, 幾乎成為一個 “典范”）
MNIST 數(shù)據(jù)集可在 http://yann.lecun.com/exdb/mnist/ 獲取, 它包含了四個部分:
Training set images: train-images-idx3-ubyte.gz (9.9 MB, 解壓后 47 MB, 包含 60,000 個樣本)
Training set labels: train-labels-idx1-ubyte.gz (29 KB, 解壓后 60 KB, 包含 60,000 個標簽)
Test set images: t10k-images-idx3-ubyte.gz (1.6 MB, 解壓后 7.8 MB, 包含 10,000 個樣本)
Test set labels: t10k-labels-idx1-ubyte.gz (5KB, 解壓后 10 KB, 包含 10,000 個標簽)
MNIST 數(shù)據(jù)集來自美國國家標準與技術(shù)研究所, National Institute of Standards and Technology (NIST). 訓練集 (training set) 由來自 250 個不同人手寫的數(shù)字構(gòu)成, 其中 50% 是高中學生, 50% 來自人口普查局 (the Census Bureau) 的工作人員. 測試集(test set) 也是同樣比例的手寫數(shù)字數(shù)據(jù)。
我們創(chuàng)建一個DataParser.py文件來解析MNIST數(shù)據(jù)：
def load_mnist(path, kind='train'):
labels_path = os.path.join(path, '%s-labels-idx1-ubyte' % kind)
images_path = os.path.join(path, '%s-images-idx3-ubyte' % kind)
with open(labels_path, 'rb')as lbfile:
magic, n = struct.unpack('>II', lbfile.read(8))
labels = np.fromfile(lbfile, dtype=np.uint8)
with open(images_path, 'rb')as imgfile:
magic, num, rows, cols = struct.unpack('>IIII', imgfile.read(16))
images = np.fromfile(imgfile, dtype=np.uint8).reshape(len(labels), 784)
return images /255, labels
除以255把圖片像素值映射到[0六敬，1]上碘赖。
手寫圖片的尺寸是28*28
訓練思路：
我們將訓練過程分為60個epochs，每個epoch里訓練1000條數(shù)據(jù)外构，訓練結(jié)束后用測試數(shù)據(jù)集來計算訓練的準確率普泡。
每次測試時，我們需要打亂測試數(shù)據(jù)审编，以保證測試結(jié)果更準確撼班。
確定網(wǎng)絡(luò)參數(shù)：
由于深度學習需要包括至少2個隱層，所以我們將網(wǎng)絡(luò)結(jié)構(gòu)定義如下：
數(shù)據(jù)層——全連接層1——隱層1——全連接層2——隱層2——輸出層
數(shù)據(jù)層垒酬、全連接層1各有784個神經(jīng)元（對應手寫圖片上的28*28個像素）砰嘁，全連接層2包含256個，輸出層10個（對應0—9的概率）
激活函數(shù)采用Sigmoid函數(shù)勘究，損失函數(shù)采用交叉熵損失函數(shù)（交叉熵損失函數(shù)的導數(shù)分母有一個1-x項矮湘，爾Sigmoid函數(shù)的導數(shù)分子有一個1-x項，兩者相乘的時候完美解決了使用Sigmoid函數(shù)作為激活函數(shù)而產(chǎn)生的梯度發(fā)散問題）
下面我們就可以用python來實現(xiàn)這個網(wǎng)絡(luò)了口糕。
數(shù)據(jù)層定義如下:（數(shù)據(jù)層正向傳播輸出其本身缅阳，且無需反向傳播）
class DataLayer:
def __init__(self):
imgs, lbls = dp.load_mnist('', kind='train')
self.x = imgs
self.y = lbls
self.pos = 0
def forward(self, index):
pos = index
xx = self.x[pos]
ret = (xx.reshape(xx.size,1), self.y[pos])
return ret
def backward(self, d):
pass
測試數(shù)據(jù)層定義如下:（使用shuffle_data每次打亂數(shù)據(jù)）
class TestLayer:
def __init__(self):
imgs, lbls = dp.load_mnist('', kind='t10k')
self.x = imgs
self.y = lbls
def shuffle_data(self):
l = len(self.x)
index = list(range(l))
np.random.shuffle(index)
self.x = self.x[index]
self.y = self.y[index]
def forward(self, index):
xx = self.x[index]
ret = (xx.reshape(xx.size,1), self.y[index])
return ret
全連接層如下：（權(quán)重除以一個值np.sqrt(l_x)，會避免產(chǎn)生異常數(shù)據(jù)）
向前傳播forward：矩陣運算景描，計算z值
反向傳播backward：根據(jù)接收到的誤差更新權(quán)重和偏置
class FullConnect:
def __init__(self, l_x, l_y):
self.weights = np.random.randn(l_y, l_x) / np.sqrt(l_x)
self.bias = np.random.randn(l_y, 1)
self.lr = 0
def forward(self, x):
self.x = x
self.y = np.dot(self.weights, x)+self.bias
return self.y
def backward(self, d):
self.dw = np.dot(d, self.x.T)
self.db = d
self.dx = np.dot(self.weights.T, d)
self.weights -= self.lr * self.dw
self.bias -= self.lr * self.db
return self.dx
激活函數(shù)：
class Sigmoid:
def __init__(self):
pass
def sigmoid(self, x):
return 1 / (1 + np.exp(-x))
def forward(self, x):
self.x = x
self.y = self.sigmoid(x)
return self.y
def backward(self, d):
sig = self.sigmoid(self.x)
self.dx =d * sig * (1 - sig)
return self.dx # 反向傳遞梯度
損失函數(shù)：
class CrossEntropyLoss:
def __init__(self):
pass
def forward(self, x, label):
self.x = x
self.label = np.zeros_like(x)
self.label[label]=1.0
self.loss=np.nan_to_num(-self.label * np.log(x) -(1- self.label)*np.log(1-x))
self.loss=np.sum(self.loss) / x.shape[0]
return self.loss
def backward(self):
self.dx=(self.x-self.label)/self.x /(1- self.x)
return self.dx
確定準確度的方法為：(數(shù)組x中最大值（最大概率）的下標和標記值一樣十办，我們認為結(jié)果準確，反之不正確)
class Accuracy:
def __init__(self):
pass
def forward(self, x, label):
b = x.max()
if x[label] > b-0.0001:
return 1
else:
return 0
然后我們將這些層進行簡單的組裝超棺，就可以進行訓練了：
def train():
block_size = 1000
datalayer1 = DataLayer()
datalayer2 = TestLayer()
inner_layers = []
inner_layers.append(FullConnect(784, 256))
inner_layers.append(Sigmoid())
inner_layers.append(FullConnect(256, 10))
inner_layers.append(Sigmoid())
lostlayer = CrossEntropyLoss()
accuracy = Accuracy()
epochs = 60
for layer in inner_layers:
layer.lr = 0.05
for i in range(epochs):
print('epochs: ', i)
losssum = 0
iters = 0
for j in range(block_size):
losssum = 0
iters += 1
x, label = datalayer1.forward(i * block_size +j)
for layer in inner_layers:
x = layer.forward(x)
loss = lostlayer.forward(x, label)
losssum += loss
d = lostlayer.backward()
for layer in inner_layers[::-1]:
d = layer.backward(d)
if j == block_size-1:
accu = 0
datalayer2.shuffle_data()
for k in range(10000):
x, label = datalayer2.forward(index=k)
for layer in inner_layers:
x = layer.forward(x)
accu += accuracy.forward(x, label)
print('accuracy: ', accu/10000)
我們把學習速度定位0.05橘洞，然后經(jīng)過不到2分鐘的訓練，就能將準確率提升到96%了说搅。輸出數(shù)據(jù)我們可以看到炸枣，第一回合到第二回合性能上有了很大的提升。
epochs: 0
accuracy: 0.6473
epochs: 1
accuracy: 0.846
epochs: 2
accuracy: 0.8799
epochs: 3
accuracy: 0.8828
epochs: 4
accuracy: 0.8964
epochs: 5
accuracy: 0.8988
epochs: 6
accuracy: 0.9022
epochs: 7
accuracy: 0.8958
epochs: 8
accuracy: 0.9058
epochs: 9
accuracy: 0.9157
epochs: 10
accuracy: 0.9055
epochs: 11
accuracy: 0.915
epochs: 12
accuracy: 0.9149
epochs: 13
accuracy: 0.9114
epochs: 14
accuracy: 0.9259
epochs: 15
accuracy: 0.9226
epochs: 16
accuracy: 0.9308
epochs: 17
accuracy: 0.9304
epochs: 18
accuracy: 0.9394
epochs: 19
accuracy: 0.9323
epochs: 20
accuracy: 0.9328
epochs: 21
accuracy: 0.9356
epochs: 22
accuracy: 0.94
epochs: 23
accuracy: 0.9456
epochs: 24
accuracy: 0.9431
epochs: 25
accuracy: 0.9374
epochs: 26
accuracy: 0.9466
epochs: 27
accuracy: 0.9468
epochs: 28
accuracy: 0.9465
epochs: 29
accuracy: 0.9389
epochs: 30
accuracy: 0.9471
epochs: 31
accuracy: 0.9473
epochs: 32
accuracy: 0.9524
epochs: 33
accuracy: 0.953
epochs: 34
accuracy: 0.9501
epochs: 35
accuracy: 0.944
epochs: 36
accuracy: 0.9458
epochs: 37
accuracy: 0.9554
epochs: 38
accuracy: 0.9556
epochs: 39
accuracy: 0.9536
epochs: 40
accuracy: 0.9497
epochs: 41
accuracy: 0.9557
epochs: 42
accuracy: 0.9548
epochs: 43
accuracy: 0.9517
epochs: 44
accuracy: 0.9496
epochs: 45
accuracy: 0.9495
epochs: 46
accuracy: 0.9579
epochs: 47
accuracy: 0.9573
epochs: 48
accuracy: 0.9485
epochs: 49
accuracy: 0.9605
epochs: 50
accuracy: 0.9618
epochs: 51
accuracy: 0.9625
epochs: 52
accuracy: 0.9599
epochs: 53
accuracy: 0.9574
epochs: 54
accuracy: 0.961
epochs: 55
accuracy: 0.9621
epochs: 56
accuracy: 0.9597
epochs: 57
accuracy: 0.9565
epochs: 58
accuracy: 0.9591
epochs: 59
accuracy: 0.9604
到此弄唧，通過搜集網(wǎng)上各種資料适肠，自己實現(xiàn)了第一個深度學習神經(jīng)網(wǎng)絡(luò)，中間不免有不合理的地方候引，繼續(xù)加油：钛！３胃伞逛揩！
