任務(wù)：
使用tensorflow訓(xùn)練一個(gè)神經(jīng)網(wǎng)絡(luò)作為分類器皆辽，分類的數(shù)據(jù)點(diǎn)如下：
螺旋形數(shù)據(jù)點(diǎn)
原理：
數(shù)據(jù)點(diǎn)一共有三個(gè)類別菊碟，而且是螺旋形交織在一起斤儿，顯然是線性不可分的裹虫，需要一個(gè)非線性的分類器。這里選擇神經(jīng)網(wǎng)絡(luò)娄琉。
輸入的數(shù)據(jù)點(diǎn)是二維的饶米，因此每個(gè)點(diǎn)只有x,y坐標(biāo)這個(gè)原始特征。這里設(shè)計(jì)的神經(jīng)網(wǎng)絡(luò)有兩個(gè)隱藏層车胡，每層有50個(gè)神經(jīng)元檬输，足夠抓住數(shù)據(jù)點(diǎn)的高維特征（實(shí)際上每層10個(gè)都?jí)蛴昧耍Ｗ詈筝敵鰧邮且粋€(gè)邏輯回歸匈棘，根據(jù)隱藏層計(jì)算出的50個(gè)特征來(lái)預(yù)測(cè)數(shù)據(jù)點(diǎn)的分類（紅丧慈、黃、藍(lán)）主卫。
一般訓(xùn)練數(shù)據(jù)多的話逃默，應(yīng)該用隨機(jī)梯度下降來(lái)訓(xùn)練神經(jīng)網(wǎng)絡(luò)，這里訓(xùn)練數(shù)據(jù)較少（300）簇搅，就直接批量梯度下降了完域。
# 導(dǎo)入包、初始化
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'
# 生成螺旋形的線形不可分?jǐn)?shù)據(jù)點(diǎn)
np.random.seed(0)
N = 100 # 每個(gè)類的數(shù)據(jù)個(gè)數(shù)
D = 2 # 輸入維度
K = 3 # 類的個(gè)數(shù)
X = np.zeros((N*K,D))
num_train_examples = X.shape[0]
y = np.zeros(N*K, dtype='uint8')
for j in xrange(K):
ix = range(N*j,N*(j+1))
r = np.linspace(0.0,1,N) # radius
t = np.linspace(j*4,(j+1)*4,N) + np.random.randn(N)*0.2 # theta
X[ix] = np.c_[r*np.sin(t), r*np.cos(t)]
y[ix] = j
fig = plt.figure()
plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap=plt.cm.Spectral)
plt.xlim([-1,1])
plt.ylim([-1,1])
螺旋形數(shù)據(jù)點(diǎn)
打印輸出輸入X和label的shape
num_label = 3
labels = (np.arange(num_label) == y[:,None]).astype(np.float32)
labels.shape
(300, 3)
X.shape
(300, 2)
用tensorflow構(gòu)建神經(jīng)網(wǎng)絡(luò)
import math
N = 100 # 每個(gè)類的數(shù)據(jù)個(gè)數(shù)
D = 2 # 輸入維度
num_label = 3 # 類的個(gè)數(shù)
num_data = N * num_label
hidden_size_1 = 50
hidden_size_2 = 50
beta = 0.001 # L2 正則化系數(shù)
learning_rate = 0.1 # 學(xué)習(xí)速率
labels = (np.arange(num_label) == y[:,None]).astype(np.float32)
graph = tf.Graph()
with graph.as_default():
x = tf.constant(X.astype(np.float32))
tf_labels = tf.constant(labels)
# 隱藏層1
hidden_layer_weights_1 = tf.Variable(
tf.truncated_normal([D, hidden_size_1], stddev=math.sqrt(2.0/num_data)))
hidden_layer_bias_1 = tf.Variable(tf.zeros([hidden_size_1]))
# 隱藏層2
hidden_layer_weights_2 = tf.Variable(
tf.truncated_normal([hidden_size_1, hidden_size_2], stddev=math.sqrt(2.0/hidden_size_1)))
hidden_layer_bias_2 = tf.Variable(tf.zeros([hidden_size_2]))
# 輸出層
out_weights = tf.Variable(
tf.truncated_normal([hidden_size_2, num_label], stddev=math.sqrt(2.0/hidden_size_2)))
out_bias = tf.Variable(tf.zeros([num_label]))
z1 = tf.matmul(x, hidden_layer_weights_1) + hidden_layer_bias_1
h1 = tf.nn.relu(z1)
z2 = tf.matmul(h1, hidden_layer_weights_2) + hidden_layer_bias_2
h2 = tf.nn.relu(z2)
logits = tf.matmul(h2, out_weights) + out_bias
# L2正則化
regularization = tf.nn.l2_loss(hidden_layer_weights_1) + tf.nn.l2_loss(hidden_layer_weights_2) + tf.nn.l2_loss(out_weights)
loss = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits(labels=tf_labels, logits=logits) + beta * regularization)
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)
train_prediction = tf.nn.softmax(logits)
weights = [hidden_layer_weights_1, hidden_layer_bias_1, hidden_layer_weights_2, hidden_layer_bias_2, out_weights, out_bias]
上一步相當(dāng)于搭建了神經(jīng)網(wǎng)絡(luò)的骨架瘩将，現(xiàn)在需要訓(xùn)練吟税。每1000步訓(xùn)練，打印交叉熵?fù)p失和正確率姿现。
num_steps = 50000
def accuracy(predictions, labels):
return (100.0 * np.sum(np.argmax(predictions, 1) == np.argmax(labels, 1))
/ predictions.shape[0])
def relu(x):
return np.maximum(0,x)
with tf.Session(graph=graph) as session:
tf.global_variables_initializer().run()
print('Initialized')
for step in range(num_steps):
_, l, predictions = session.run([optimizer, loss, train_prediction])
if (step % 1000 == 0):
print('Loss at step %d: %f' % (step, l))
print('Training accuracy: %.1f%%' % accuracy(
predictions, labels))
w1, b1, w2, b2, w3, b3 = weights
# 顯示分類器
h = 0.02
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
np.arange(y_min, y_max, h))
Z = np.dot(relu(np.dot(relu(np.dot(np.c_[xx.ravel(), yy.ravel()], w1.eval()) + b1.eval()), w2.eval()) + b2.eval()), w3.eval()) + b3.eval()
Z = np.argmax(Z, axis=1)
Z = Z.reshape(xx.shape)
fig = plt.figure()
plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral, alpha=0.8)
plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap=plt.cm.Spectral)
plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())
Initialized
Loss at step 0: 1.132545
Training accuracy: 43.7%
Loss at step 1000: 0.257016
Training accuracy: 94.0%
Loss at step 2000: 0.165511
Training accuracy: 98.0%
Loss at step 3000: 0.149266
Training accuracy: 99.0%
Loss at step 4000: 0.142311
Training accuracy: 99.3%
Loss at step 5000: 0.137762
Training accuracy: 99.3%
Loss at step 6000: 0.134356
Training accuracy: 99.3%
Loss at step 7000: 0.131588
Training accuracy: 99.3%
Loss at step 8000: 0.129299
Training accuracy: 99.3%
Loss at step 9000: 0.127340
Training accuracy: 99.3%
Loss at step 10000: 0.125686
Training accuracy: 99.3%
Loss at step 11000: 0.124293
Training accuracy: 99.3%
Loss at step 12000: 0.123130
Training accuracy: 99.3%
Loss at step 13000: 0.122149
Training accuracy: 99.3%
Loss at step 14000: 0.121309
Training accuracy: 99.3%
Loss at step 15000: 0.120542
Training accuracy: 99.3%
Loss at step 16000: 0.119895
Training accuracy: 99.3%
Loss at step 17000: 0.119335
Training accuracy: 99.3%
Loss at step 18000: 0.118836
Training accuracy: 99.3%
Loss at step 19000: 0.118376
Training accuracy: 99.3%
Loss at step 20000: 0.117974
Training accuracy: 99.3%
Loss at step 21000: 0.117601
Training accuracy: 99.3%
Loss at step 22000: 0.117253
Training accuracy: 99.3%
Loss at step 23000: 0.116887
Training accuracy: 99.3%
Loss at step 24000: 0.116561
Training accuracy: 99.3%
Loss at step 25000: 0.116265
Training accuracy: 99.3%
Loss at step 26000: 0.115995
Training accuracy: 99.3%
Loss at step 27000: 0.115750
Training accuracy: 99.3%
Loss at step 28000: 0.115521
Training accuracy: 99.3%
Loss at step 29000: 0.115310
Training accuracy: 99.3%
Loss at step 30000: 0.115111
Training accuracy: 99.3%
Loss at step 31000: 0.114922
Training accuracy: 99.3%
Loss at step 32000: 0.114743
Training accuracy: 99.3%
Loss at step 33000: 0.114567
Training accuracy: 99.3%
Loss at step 34000: 0.114401
Training accuracy: 99.3%
Loss at step 35000: 0.114242
Training accuracy: 99.3%
Loss at step 36000: 0.114086
Training accuracy: 99.3%
Loss at step 37000: 0.113933
Training accuracy: 99.3%
Loss at step 38000: 0.113785
Training accuracy: 99.3%
Loss at step 39000: 0.113644
Training accuracy: 99.3%
Loss at step 40000: 0.113504
Training accuracy: 99.3%
Loss at step 41000: 0.113366
Training accuracy: 99.3%
Loss at step 42000: 0.113229
Training accuracy: 99.3%
Loss at step 43000: 0.113096
Training accuracy: 99.3%
Loss at step 44000: 0.112966
Training accuracy: 99.3%
Loss at step 45000: 0.112838
Training accuracy: 99.3%
Loss at step 46000: 0.112711
Training accuracy: 99.3%
Loss at step 47000: 0.112590
Training accuracy: 99.3%
Loss at step 48000: 0.112472
Training accuracy: 99.3%
Loss at step 49000: 0.112358
Training accuracy: 99.3%
分類器.png
