50題真 ? 一文入門TensorFlow2.x
點擊以上鏈接?? 不用配置環(huán)境茵乱，直接在線運行
（本文基于TensorFlow 2.x編寫）
大家好裤纹，擠牙膏擠了一個多月爵憎，終于把tensorflow 2.x的教程做出來了鸡岗。這個教程是使用tensorflow低階api做的，想學(xué)keras的高階api可以直接看我的keras教程40題刷爆Keras戈锻，人生苦短我選Keras
TensorFlow是谷歌基于DistBelief進行研發(fā)的第二代人工智能學(xué)習(xí)系統(tǒng)歼跟，其命名來源于本身的運行原理。Tensor（張量）意味著N維數(shù)組格遭，F(xiàn)low（流）意味著基于數(shù)據(jù)流圖的計算哈街，TensorFlow為張量從流圖的一端流動到另一端計算過程。TensorFlow是將復(fù)雜的數(shù)據(jù)結(jié)構(gòu)傳輸至人工智能神經(jīng)網(wǎng)中進行分析和處理過程的系統(tǒng)拒迅。 TensorFlow可被用于語音識別或圖像識別等多項機器學(xué)習(xí)和深度學(xué)習(xí)領(lǐng)域骚秦，對2011年開發(fā)的深度學(xué)習(xí)基礎(chǔ)架構(gòu)DistBelief進行了各方面的改進她倘，它可在小到一部智能手機、大到數(shù)千臺數(shù)據(jù)中心服務(wù)器的各種設(shè)備上運行作箍。TensorFlow將完全開源帝牡，任何人都可以用。
其他x題系列：
- 50道練習(xí)帶你玩轉(zhuǎn)Pandas
- 這100道練習(xí)蒙揣，帶你玩轉(zhuǎn)Numpy
- 35題初探scikit-learn庫，get機器學(xué)習(xí)好幫手√
- 50題matplotlib從入門到精通
- 40題刷爆Keras开瞭，人生苦短我選Keras
- 60題PyTorch簡易入門指南懒震，做技術(shù)的弄潮兒
- 90題細品吳恩達《機器學(xué)習(xí)》，感受被刷題支配的恐懼
- 170題吳恩達《深度學(xué)習(xí)》面面觀嗤详，一套更比三套強
# 導(dǎo)入一些必要的庫
import numpy as np
import matplotlib.pyplot as plt
import os
import pickle
1.導(dǎo)入tensorflow庫簡寫為tf个扰，并輸出版本
import tensorflow as tf
tf.__version__
'2.1.0'
一、Tensor張量
常量
2.創(chuàng)建一個3x3的0常量張量
c = tf.zeros([3, 3])
3.根據(jù)上題張量的形狀葱色，創(chuàng)建一個一樣形狀的1常量張量
tf.ones_like(c)
<tf.Tensor: shape=(3, 3), dtype=float32, numpy=
array([[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]], dtype=float32)>
4.創(chuàng)建一個2x3递宅，數(shù)值全為6的常量張量
tf.fill([2, 3], 6) # 2x3 全為 6 的常量 Tensor
<tf.Tensor: shape=(2, 3), dtype=int32, numpy=
array([[6, 6, 6],
[6, 6, 6]], dtype=int32)>
5.創(chuàng)建3x3隨機的隨機數(shù)組
tf.random.normal([3,3])
<tf.Tensor: shape=(3, 3), dtype=float32, numpy=
array([[-1.1090602 , -0.14372216, -2.0020268 ],
[-1.246778 , -0.155268 , 1.3298218 ],
[-0.47514197, -0.49891278, 0.6524196 ]], dtype=float32)>
6.通過二維數(shù)組創(chuàng)建一個常量張量
a = tf.constant([[1, 2], [3, 4]])
a
<tf.Tensor: shape=(2, 2), dtype=int32, numpy=
array([[1, 2],
[3, 4]], dtype=int32)>
7.取出張量中的numpy數(shù)組
a.numpy()
array([[1, 2],
[3, 4]], dtype=int32)
8.從1.0-10.0等間距取出5個數(shù)形成一個常量張量
tf.linspace(1.0, 10.0, 5)
<tf.Tensor: shape=(5,), dtype=float32, numpy=array([ 1. , 3.25, 5.5 , 7.75, 10. ], dtype=float32)>
9.從1開始間隔2取1個數(shù)字，到大等于10為止
tf.range(start=1, limit=10, delta=2)
<tf.Tensor: shape=(5,), dtype=int32, numpy=array([1, 3, 5, 7, 9], dtype=int32)>
運算
10.將兩個張量相加
a + a
<tf.Tensor: shape=(2, 2), dtype=int32, numpy=
array([[2, 4],
[6, 8]], dtype=int32)>
11.將兩個張量做矩陣乘法
tf.matmul(a, a)
<tf.Tensor: shape=(2, 2), dtype=int32, numpy=
array([[ 7, 10],
[15, 22]], dtype=int32)>
12.兩個張量做點乘
tf.multiply(a, a)
<tf.Tensor: shape=(2, 2), dtype=int32, numpy=
array([[ 1, 4],
[ 9, 16]], dtype=int32)>
13.將一個張量轉(zhuǎn)置
tf.linalg.matrix_transpose(c)
<tf.Tensor: shape=(3, 3), dtype=float32, numpy=
array([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]], dtype=float32)>
14.將一個12x1張量變形成3行的張量
b = tf.linspace(1.0, 10.0, 12)
tf.reshape(b,[3,4])
# 方法二
tf.reshape(b,[3,-1])
<tf.Tensor: shape=(3, 4), dtype=float32, numpy=
array([[ 1. , 1.8181818, 2.6363635, 3.4545455],
[ 4.272727 , 5.090909 , 5.909091 , 6.7272725],
[ 7.5454545, 8.363636 , 9.181818 , 10. ]], dtype=float32)>
二苍狰、自動微分
這一部分將會實現(xiàn) 在
處的導(dǎo)數(shù)
變量
15.新建一個1x1變量办龄，值為1
x = tf.Variable([1.0]) # 新建張量
x
<tf.Variable 'Variable:0' shape=(1,) dtype=float32, numpy=array([1.], dtype=float32)>
16.新建一個GradientTape追蹤梯度，把要微分的公式寫在里面
with tf.GradientTape() as tape: # 追蹤梯度
y = x * x
17.求y對于x的導(dǎo)數(shù)
grad = tape.gradient(y, x) # 計算梯度
grad
<tf.Tensor: shape=(1,), dtype=float32, numpy=array([2.], dtype=float32)>
三淋昭、線性回歸案例
這一部分將生成添加隨機噪聲的沿100個的數(shù)據(jù)點俐填，再對這些數(shù)據(jù)點進行擬合。
18.生成X,y數(shù)據(jù)翔忽，X為100個隨機數(shù)英融，y=3X+2+noise，noise為100個隨機數(shù)
X = tf.random.normal([100, 1]).numpy()
noise = tf.random.normal([100, 1]).numpy()
y = 3*X+2+noise
可視化這些點
plt.scatter(X, y)
image
19.創(chuàng)建需要預(yù)測的參數(shù)W,b（變量張量）
W = tf.Variable(np.random.randn())
b = tf.Variable(np.random.randn())
print('W: %f, b: %f'%(W.numpy(), b.numpy()))
W: 0.546446, b: -0.565772
20.創(chuàng)建線性回歸預(yù)測模型
def linear_regression(x):
return W * x + b
21.創(chuàng)建損失函數(shù)歇式，此處采用真實值與預(yù)測值的差的平方驶悟，公式為：
def mean_square(y_pred, y_true):
return tf.reduce_mean(tf.square(y_pred-y_true))
22.創(chuàng)建GradientTape，寫入需要微分的過程
with tf.GradientTape() as tape:
pred = linear_regression(X)
loss = mean_square(pred, y)
23.對loss材失，分別求關(guān)于W,b的偏導(dǎo)數(shù)
dW, db = tape.gradient(loss, [W, b])
24.用最簡單樸素的梯度下降更新W,b痕鳍，learning_rate設(shè)置為0.1
W.assign_sub(0.1*dW)
b.assign_sub(0.1*db)
print('W: %f, b: %f'%(W.numpy(), b.numpy()))
W: 0.958635, b: -0.127122
25.以上就是單次迭代的過程，現(xiàn)在我們要繼續(xù)循環(huán)迭代20次豺憔，并且記錄每次的loss,W,b
for i in range(20):
with tf.GradientTape() as tape:
pred = linear_regression(X)
loss = mean_square(pred, y)
dW, db = tape.gradient(loss, [W, b])
W.assign_sub(0.1*dW)
b.assign_sub(0.1*db)
print("step: %i, loss: %f, W: %f, b: %f" % (i+1, loss, W.numpy(), b.numpy()))
step: 1, loss: 8.821126, W: 1.306481, b: 0.235136
step: 2, loss: 6.507570, W: 1.599810, b: 0.534510
step: 3, loss: 4.896123, W: 1.846998, b: 0.782077
step: 4, loss: 3.773298, W: 2.055174, b: 0.986930
step: 5, loss: 2.990682, W: 2.230394, b: 1.156539
step: 6, loss: 2.445040, W: 2.377799, b: 1.297045
step: 7, loss: 2.064524, W: 2.501743, b: 1.413504
step: 8, loss: 1.799106, W: 2.605914, b: 1.510081
step: 9, loss: 1.613936, W: 2.693432, b: 1.590208
step: 10, loss: 1.484730, W: 2.766930, b: 1.656716
step: 11, loss: 1.394562, W: 2.828633, b: 1.711945
step: 12, loss: 1.331630, W: 2.880417, b: 1.757825
step: 13, loss: 1.287701, W: 2.923863, b: 1.795953
step: 14, loss: 1.257035, W: 2.960305, b: 1.827651
step: 15, loss: 1.235626, W: 2.990863, b: 1.854011
step: 16, loss: 1.220678, W: 3.016481, b: 1.875940
step: 17, loss: 1.210241, W: 3.037954, b: 1.894188
step: 18, loss: 1.202953, W: 3.055948, b: 1.909377
step: 19, loss: 1.197864, W: 3.071024, b: 1.922023
step: 20, loss: 1.194310, W: 3.083654, b: 1.932554
畫出最終擬合的曲線
plt.plot(X, y, 'ro', label='Original data')
plt.plot(X, np.array(W * X + b), label='Fitted line')
plt.legend()
plt.show()
image
四额获、神經(jīng)網(wǎng)絡(luò)案例
這部分將會在CIFAR10數(shù)據(jù)集上，訓(xùn)練LeNet5模型
模型結(jié)構(gòu)如下所示：
image
CIFAR10數(shù)據(jù)集為32x32的3通道圖像恭应，標簽共10個種類
定義參數(shù)
26.定義第①步：卷積層的參數(shù)
輸入圖片：3×32×32
卷積核大谐：5×5
卷積核種類：6
所以需要定義5×5×3×6個權(quán)重變量，和6個bias變量
conv1_w = tf.Variable(tf.random.truncated_normal([5,5,3,6], stddev=0.1))
conv1_b = tf.Variable(tf.zeros([6]))
27.定義第③步：卷積層的參數(shù)
輸入：14×14×6
卷積核大兄玳弧：5×5
卷積核種類：16
所以需要定義5×5×6×16個權(quán)重變量境肾，和16個bias變量
conv2_w = tf.Variable(tf.random.truncated_normal([5, 5, 6, 16], stddev=0.1))
conv2_b = tf.Variable(tf.zeros([16]))
28.定義第第⑤步：全連接層的參數(shù)
輸入：5×5×16
輸出：120
fc1_w = tf.Variable(tf.random.truncated_normal([5*5*16, 120], stddev=0.1))
fc1_b = tf.Variable(tf.zeros([120]))
29.定義第⑥步：全連接層的參數(shù)
輸入：120
輸出：84
fc2_w = tf.Variable(tf.random.truncated_normal([120, 84], stddev=0.1))
fc2_b = tf.Variable(tf.zeros([84]))
30.定義第⑦步：全連接層的參數(shù)
輸入：84
輸出：10
fc3_w = tf.Variable(tf.random.truncated_normal([84, 10], stddev=0.1))
fc3_b = tf.Variable(tf.zeros([10]))
搭模型
def lenet5(input_img):
## 31.搭建INPUT->C1的步驟
conv1_1 = tf.nn.conv2d(input_img, conv1_w, strides=[1,1,1,1], padding="VALID")
conv1_2 = tf.nn.relu(tf.nn.bias_add(conv1_1,conv1_b))
## 32.搭建C1->S2的步驟
pool1 = tf.nn.max_pool(conv1_2,ksize=[1,2,2,1],strides=[1,2,2,1],padding="VALID")
## 33.搭建S2->C3的步驟
conv2_1 = tf.nn.conv2d(pool1,conv2_w,strides=[1,1,1,1],padding="VALID")
conv2_2 = tf.nn.relu(tf.nn.bias_add(conv2_1,conv2_b))
## 34.搭建C3->S4的步驟
pool2 = tf.nn.max_pool(conv2_2,ksize=[1,2,2,1],strides=[1,2,2,1],padding="VALID")
## 35.將S4的輸出扁平化
reshaped = tf.reshape(pool2,[-1, 16*5*5])
## 35.搭建S4->C5的步驟
fc1 = tf.nn.relu(tf.matmul(reshaped,fc1_w) + fc1_b)
## 36.搭建C5->F6的步驟
fc2 = tf.nn.relu(tf.matmul(fc1,fc2_w) + fc2_b)
## 37.搭建F6->OUTPUT的步驟
OUTPUT = tf.nn.softmax(tf.matmul(fc2,fc3_w) + fc3_b)
return OUTPUT
38.創(chuàng)建一個Adam優(yōu)化器剔难，學(xué)習(xí)率0.02
optimizer = tf.optimizers.Adam(learning_rate=0.02)
驗證網(wǎng)絡(luò)正確性
（隨便搞點數(shù)據(jù)，驗證一下能不能跑通）
39.隨機一對x,y數(shù)據(jù)奥喻，x的形狀為(1,32,32,3)偶宫，y的形狀為(10,)
test_x = tf.Variable(tf.random.truncated_normal([1,32,32,3]))
test_y = [1,0,0,0,0,0,0,0,0,0]
將數(shù)據(jù)送入模型，進行反向傳播
with tf.GradientTape() as tape:
## 40.將數(shù)據(jù)從入模型
prediction = lenet5(test_x)
print("第一次預(yù)測：", prediction)
## 41.使用交叉熵作為損失函數(shù)环鲤，計算損失
cross_entropy = -tf.reduce_sum(test_y * tf.math.log(prediction))
## 42.計算梯度
trainable_variables = [conv1_w, conv1_b, conv2_w, conv2_b, fc1_w, fc1_b, fc2_w, fc2_b, fc3_w, fc3_b] # 需優(yōu)化參數(shù)列表
grads = tape.gradient(cross_entropy, trainable_variables)
## 43.更新梯度
optimizer.apply_gradients(zip(grads, trainable_variables))
print("反向傳播后的預(yù)測：", lenet5(test_x))
第一次預(yù)測： tf.Tensor(
[[0.04197786 0.25509077 0.10203867 0.1586983 0.08596517 0.08058039
0.03977505 0.1154211 0.03524319 0.08520938]], shape=(1, 10), dtype=float32)
反向傳播后的預(yù)測： tf.Tensor(
[[9.9998593e-01 7.0147706e-08 2.0064485e-08 4.0835562e-06 2.0444522e-06
1.0025160e-08 3.7448625e-10 6.8495629e-06 1.0462685e-06 1.5679738e-08]], shape=(1, 10), dtype=float32)
讀入數(shù)據(jù)纯趋，預(yù)處理
def load_cifar_batch(filename):
""" load single batch of cifar """
with open(filename, 'rb') as f:
datadict = pickle.load(f, encoding='iso-8859-1')
X = datadict['data']
Y = datadict['labels']
X = X.reshape(10000, 3, 32, 32).transpose(0,2,3,1).astype("uint8")
Y = np.array(Y)
return X,Y
def load_cifar(ROOT):
data_X, data_Y = load_cifar_batch('/home/kesci/input/cifar10/data_batch_1')
for b in range(2,6):
f = os.path.join(ROOT, 'data_batch_%d' % (b, ))
batch_X, batch_Y = load_cifar_batch(f)
data_X = np.concatenate([data_X,batch_X])
data_Y = np.concatenate([data_Y,batch_Y])
data_test_X, data_test_Y = load_cifar_batch(os.path.join(ROOT, 'test_batch'))
return data_X, data_Y, data_test_X, data_test_Y
train_X, train_Y, test_X, test_Y = load_cifar('/home/kesci/input/cifar10')
classes = ('plane', 'car', 'bird', 'cat',
'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
train_X.shape, train_X.shape, test_X.shape, test_Y.shape
((50000, 32, 32, 3), (50000, 32, 32, 3), (10000, 32, 32, 3), (10000,))
撈一個數(shù)據(jù)看看樣子
plt.imshow(train_X[0])
plt.show()
print(classes[train_Y[0]])
image
frog
44.預(yù)處理1：將train_y, test_y進行歸一化
train_X = tf.cast(train_X, dtype=tf.float32) / 255
test_X = tf.cast(test_X, dtype=tf.float32) / 255
45.預(yù)處理2：將train_y, test_y進行onehot編碼
train_Y = tf.one_hot(train_Y, depth=10)
test_Y = tf.one_hot(test_Y, depth=10)
訓(xùn)練網(wǎng)絡(luò)
因為前面實驗的時候修改過參數(shù)，所以需要重新初始化所有參數(shù)
conv1_w = tf.Variable(tf.random.truncated_normal([5,5,3,6], stddev=0.1))
conv1_b = tf.Variable(tf.zeros([6]))
conv2_w = tf.Variable(tf.random.truncated_normal([5, 5, 6, 16], stddev=0.1))
conv2_b = tf.Variable(tf.zeros([16]))
fc1_w = tf.Variable(tf.random.truncated_normal([5*5*16, 120], stddev=0.1))
fc1_b = tf.Variable(tf.zeros([120]))
fc2_w = tf.Variable(tf.random.truncated_normal([120, 84], stddev=0.1))
fc2_b = tf.Variable(tf.zeros([84]))
fc3_w = tf.Variable(tf.random.truncated_normal([84, 10], stddev=0.1))
fc3_b = tf.Variable(tf.zeros([10]))
然后再重新定義一個優(yōu)化器
optimizer2 = tf.optimizers.Adam(learning_rate=0.002)
簡簡單單隨便寫一個算準確率的函數(shù)
def accuracy_fn(y_pred, y_true):
preds = tf.argmax(y_pred, axis=1) # 取值最大的索引冷离，正好對應(yīng)字符標簽
labels = tf.argmax(y_true, axis=1)
return tf.reduce_mean(tf.cast(tf.equal(preds, labels), tf.float32))
46.把數(shù)據(jù)送入模型吵冒，開始訓(xùn)練，訓(xùn)練集迭代5遍西剥，每遍分成25個batch痹栖，數(shù)據(jù)集每迭代完一遍，輸出一次訓(xùn)練集上的準確率
EPOCHS = 5 # 整個數(shù)據(jù)集迭代次數(shù)
for epoch in range(EPOCHS):
for i in range(25): # 一整個數(shù)據(jù)集分為10個小batch訓(xùn)練
with tf.GradientTape() as tape:
prediction = lenet5(train_X[i*2000:(i+1)*2000])
cross_entropy = -tf.reduce_sum(train_Y[i*2000:(i+1)*2000] * tf.math.log(prediction))
trainable_variables = [conv1_w, conv1_b, conv2_w, conv2_b, fc1_w, fc1_b, fc2_w, fc2_b, fc3_w, fc3_b] # 需優(yōu)化參數(shù)列表
grads = tape.gradient(cross_entropy, trainable_variables) # 計算梯度
optimizer2.apply_gradients(zip(grads, trainable_variables)) # 更新梯度
# 每訓(xùn)練完一次瞭空，輸出一下訓(xùn)練集的準確率
accuracy = accuracy_fn(lenet5(train_X), train_Y)
print('Epoch [{}/{}], Train loss: {:.3f}, Test accuracy: {:.3f}'
.format(epoch+1, EPOCHS, cross_entropy/2000, accuracy))
Epoch [1/5], Train loss: 1.934, Test accuracy: 0.309
Epoch [2/5], Train loss: 1.758, Test accuracy: 0.376
Epoch [3/5], Train loss: 1.638, Test accuracy: 0.422
Epoch [4/5], Train loss: 1.564, Test accuracy: 0.442
Epoch [5/5], Train loss: 1.488, Test accuracy: 0.472
使用網(wǎng)絡(luò)進行預(yù)測
47.在測試集上進行預(yù)測
test_prediction = lenet5(test_X)
test_acc = accuracy_fn(test_prediction, test_Y)
test_acc.numpy()
0.4665
取一些數(shù)據(jù)查看預(yù)測結(jié)果
plt.figure(figsize=(10,10))
for i in range(25):
plt.subplot(5,5,i+1)
plt.imshow(test_X[i], cmap=plt.cm.binary)
title=classes[np.argmax(test_Y[i])]+'=>'
title+=classes[np.argmax(test_prediction[i])]
plt.xlabel(title)
plt.xticks([])
plt.yticks([])
plt.grid(False)
image
五揪阿、變量保存&讀取
這一部分，我們實現(xiàn)最簡單的保存&讀取變量值
48.新建一個Checkpoint對象咆畏，并且往里灌一個剛剛訓(xùn)練完的數(shù)據(jù)
save = tf.train.Checkpoint()
save.listed = [fc3_b]
save.mapped = {'fc3_b': save.listed[0]}
49.利用save()的方法保存南捂，并且記錄返回的保存路徑
save_path = save.save('/home/kesci/work/data/tf_list_example')
print(save_path)
/home/kesci/work/data/tf_list_example-1
50.新建一個Checkpoint對象，從里讀出數(shù)據(jù)
restore = tf.train.Checkpoint()
fc3_b2 = tf.Variable(tf.zeros([10]))
print(fc3_b2.numpy())
restore.mapped = {'fc3_b': fc3_b2}
restore.restore(save_path)
print(fc3_b2.numpy())
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[-0.04522281 -0.03583474 0.06039428 -0.00184523 0.10119602 -0.04645465
0.06197326 -0.02776093 -0.00871902 -0.04295361]
