2018年9月20日筆記
kaggle網(wǎng)站手寫(xiě)數(shù)字分類的比賽鏈接：https://www.kaggle.com/c/digit-recognizer
注冊(cè)賬號(hào)后才能參加kaggle比賽邑退，本文作者成績(jī)前2%捏萍，如下圖所示：
0.嘗試提交
本文作者提供一份能夠獲得較好成績(jī)的文件宛畦，讀者可以提交該文件熟悉提交流程。
下載鏈接: https://pan.baidu.com/s/1QKVMmAnW7Ui1104fhfiljg 提取碼: mqex
該作答文件的提交成績(jī)有0.99814拆魏，如果讀者想提高成績(jī)到0.99985鹏倘，請(qǐng)閱讀后面的章節(jié)。
1.配置環(huán)境
使用卷積神經(jīng)網(wǎng)絡(luò)模型要求有較高的機(jī)器配置绰寞，如果使用CPU版tensorflow會(huì)花費(fèi)大量時(shí)間。
讀者在有nvidia顯卡的情況下铣口，安裝GPU版tensorflow會(huì)提高計(jì)算速度50倍滤钱。
安裝教程鏈接：https://blog.csdn.net/qq_36556893/article/details/79433298
如果沒(méi)有nvidia顯卡，但有visa信用卡脑题，請(qǐng)閱讀我的另一篇文章《在谷歌云服務(wù)器上搭建深度學(xué)習(xí)平臺(tái)》件缸，鏈接：http://www.reibang.com/p/893d622d1b5a
2.下載并解壓數(shù)據(jù)集
MNIST數(shù)據(jù)集下載鏈接: https://pan.baidu.com/s/1fPbgMqsEvk2WyM9hy5Em6w 密碼: wa9p
下載壓縮文件MNIST_data.rar完成后，選擇解壓到當(dāng)前文件夾叔遂，不要選擇解壓到MNIST_data他炊。
文件夾結(jié)構(gòu)如下圖所示：
3.模型訓(xùn)練并保存
本文作者此段代碼是在谷歌云服務(wù)器上運(yùn)行争剿，谷歌云服務(wù)器的GPU顯存有16G。
因?yàn)閭€(gè)人電腦GPU的顯存不足佑稠，讀者可能無(wú)法運(yùn)行秒梅，解決辦法是減少feed_dict中的樣本數(shù)量旗芬。
理解下面一段代碼舌胶，請(qǐng)閱讀本文作者的另外一篇文章《基于tensorflow+CNN的MNIST數(shù)據(jù)集手寫(xiě)數(shù)字分類》，鏈接：http://www.reibang.com/p/a652f1cb95b4
import warnings
warnings.filterwarnings('ignore')
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
import random
import numpy as np
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
batch_size = 300
X_holder = tf.placeholder(tf.float32)
y_holder = tf.placeholder(tf.float32)
X = np.vstack([mnist.train.images, mnist.test.images, mnist.validation.images])
y = np.vstack([mnist.train.labels, mnist.test.labels, mnist.validation.labels])
print(X.shape, y.shape)
X_images = tf.reshape(X_holder, [-1, 28, 28, 1])
#convolutional layer 1
conv1_Weights = tf.Variable(tf.truncated_normal([5, 5, 1, 32], stddev=0.1), name='conv1_Weights')
conv1_biases = tf.Variable(tf.constant(0.1, shape=[32]), name='conv1_biases')
conv1_conv2d = tf.nn.conv2d(X_images, conv1_Weights, strides=[1, 1, 1, 1], padding='SAME') + conv1_biases
conv1_activated = tf.nn.relu(conv1_conv2d)
conv1_pooled = tf.nn.max_pool(conv1_activated, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
#convolutional layer 2
conv2_Weights = tf.Variable(tf.truncated_normal([5, 5, 32, 64], stddev=0.1), name='conv2_Weights')
conv2_biases = tf.Variable(tf.constant(0.1, shape=[64]), name='conv2_biases')
conv2_conv2d = tf.nn.conv2d(conv1_pooled, conv2_Weights, strides=[1, 1, 1, 1], padding='SAME') + conv2_biases
conv2_activated = tf.nn.relu(conv2_conv2d)
conv2_pooled = tf.nn.max_pool(conv2_activated, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
#full connected layer 1
connect1_flat = tf.reshape(conv2_pooled, [-1, 7 * 7 * 64])
connect1_Weights = tf.Variable(tf.truncated_normal([7 * 7 * 64, 1024], stddev=0.1), name='connect1_Weights')
connect1_biases = tf.Variable(tf.constant(0.1, shape=[1024]), name='connect1_biases')
connect1_Wx_plus_b = tf.add(tf.matmul(connect1_flat, connect1_Weights), connect1_biases)
connect1_activated = tf.nn.relu(connect1_Wx_plus_b)
#full connected layer 2
connect2_Weights = tf.Variable(tf.truncated_normal([1024, 10], stddev=0.1), name='connect2_Weights')
connect2_biases = tf.Variable(tf.constant(0.1, shape=[10]), name='connect2_biases')
connect2_Wx_plus_b = tf.add(tf.matmul(connect1_activated, connect2_Weights), connect2_biases)
predict_y = tf.nn.softmax(connect2_Wx_plus_b)
#loss and train
loss = tf.reduce_mean(-tf.reduce_sum(y_holder * tf.log(predict_y), 1))
optimizer = tf.train.AdamOptimizer(0.0001)
train = optimizer.minimize(loss)
init = tf.global_variables_initializer()
session = tf.Session()
session.run(init)
saver = tf.train.Saver()
for i in range(20000):
selected_index = random.sample(range(len(y)), k=batch_size)
selected_X = X[selected_index]
selected_y = y[selected_index]
session.run(train, feed_dict={X_holder:selected_X, y_holder:selected_y})
if i % 100 == 0:
correct_prediction = tf.equal(tf.argmax(predict_y, 1), tf.argmax(y_holder, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
train_accuracy = session.run(accuracy, feed_dict={X_holder:mnist.train.images, y_holder:mnist.train.labels})
test_accuracy = session.run(accuracy, feed_dict={X_holder:mnist.test.images, y_holder:mnist.test.labels})
validation_accuracy = session.run(accuracy, feed_dict={X_holder:mnist.validation.images, y_holder:mnist.validation.labels})
print('step:%d train accuracy:%.4f test accuracy:%.4f validation accuracy:%.4f' %(i, train_accuracy, test_accuracy, validation_accuracy))
if train_accuracy == 1 and test_accuracy == 1 and validation_accuracy == 1:
save_path = saver.save(session, 'mnist_cnn_model/mnist_cnn.ckpt')
print('Save to path:', save_path)
4.加載模型
本文作者提供獲得最佳成績(jī)0.99985的模型疮丛，讀者可以加載該模型幔嫂，并用此模型預(yù)測(cè)并提交成績(jī)。
模型下載鏈接: https://pan.baidu.com/s/1zVLHdGiZflspV9jPWn_ECA 提取碼: nktv
如果讀者有服務(wù)器誊薄，可以嘗試獲取保存的模型履恩，下載按鈕如下圖所示：
import warnings
warnings.filterwarnings('ignore')
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
batch_size = 100
X_holder = tf.placeholder(tf.float32)
y_holder = tf.placeholder(tf.float32)
X_images = tf.reshape(X_holder, [-1, 28, 28, 1])
#convolutional layer 1
conv1_Weights = tf.Variable(tf.truncated_normal([5, 5, 1, 32], stddev=0.1), name='conv1_Weights')
conv1_biases = tf.Variable(tf.constant(0.1, shape=[32]), name='conv1_biases')
conv1_conv2d = tf.nn.conv2d(X_images, conv1_Weights, strides=[1, 1, 1, 1], padding='SAME') + conv1_biases
conv1_activated = tf.nn.relu(conv1_conv2d)
conv1_pooled = tf.nn.max_pool(conv1_activated, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
#convolutional layer 2
conv2_Weights = tf.Variable(tf.truncated_normal([5, 5, 32, 64], stddev=0.1), name='conv2_Weights')
conv2_biases = tf.Variable(tf.constant(0.1, shape=[64]), name='conv2_biases')
conv2_conv2d = tf.nn.conv2d(conv1_pooled, conv2_Weights, strides=[1, 1, 1, 1], padding='SAME') + conv2_biases
conv2_activated = tf.nn.relu(conv2_conv2d)
conv2_pooled = tf.nn.max_pool(conv2_activated, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
#full connected layer 1
connect1_flat = tf.reshape(conv2_pooled, [-1, 7 * 7 * 64])
connect1_Weights = tf.Variable(tf.truncated_normal([7 * 7 * 64, 1024], stddev=0.1), name='connect1_Weights')
connect1_biases = tf.Variable(tf.constant(0.1, shape=[1024]), name='connect1_biases')
connect1_Wx_plus_b = tf.add(tf.matmul(connect1_flat, connect1_Weights), connect1_biases)
connect1_activated = tf.nn.relu(connect1_Wx_plus_b)
#full connected layer 2
connect2_Weights = tf.Variable(tf.truncated_normal([1024, 10], stddev=0.1), name='connect2_Weights')
connect2_biases = tf.Variable(tf.constant(0.1, shape=[10]), name='connect2_biases')
connect2_Wx_plus_b = tf.add(tf.matmul(connect1_activated, connect2_Weights), connect2_biases)
predict_y = tf.nn.softmax(connect2_Wx_plus_b)
#loss and train
loss = tf.reduce_mean(-tf.reduce_sum(y_holder * tf.log(predict_y), 1))
optimizer = tf.train.AdamOptimizer(0.0001)
train = optimizer.minimize(loss)
session = tf.Session()
saver = tf.train.Saver()
saver.restore(session, 'mnist_cnn_model/mnist_cnn.ckpt')
correct_prediction = tf.equal(tf.argmax(predict_y, 1), tf.argmax(y_holder, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('load model successful')
train_images, train_labels = mnist.train.next_batch(5000)
test_images, test_labels = mnist.test.next_batch(5000)
train_accuracy = session.run(accuracy, feed_dict={X_holder:train_images, y_holder:train_labels})
test_accuracy = session.run(accuracy, feed_dict={X_holder:test_images, y_holder:test_labels})
print('train accuracy:%.4f test accuracy:%.4f' %(train_accuracy, test_accuracy))
上面一段代碼的運(yùn)行結(jié)果如下：
Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
INFO:tensorflow:Restoring parameters from mnist_cnn_model/mnist_cnn.ckpt
load model successful
train accuracy:1.0000 test accuracy:1.0000
5.模型預(yù)測(cè)
此第5張能夠成功運(yùn)行的前提是已經(jīng)成功運(yùn)行第4章的代碼，即加載模型成功呢蔫。
將測(cè)試樣本分成6份切心，可以解決因?yàn)轱@存不足無(wú)法運(yùn)行的問(wèn)題。
import pandas as pd
test_df = pd.read_csv('test.csv')
X = test_df.values
print('特征矩陣的形狀：', X.shape)
X1 = X[:5000]
X2 = X[5000:10000]
X3 = X[10000:15000]
X4 = X[15000:20000]
X5 = X[20000:25000]
X6 = X[25000:]
y1 = session.run(predict_y, feed_dict={X_holder:X1})
y2 = session.run(predict_y, feed_dict={X_holder:X2})
y3 = session.run(predict_y, feed_dict={X_holder:X3})
y4 = session.run(predict_y, feed_dict={X_holder:X4})
y5 = session.run(predict_y, feed_dict={X_holder:X5})
y6 = session.run(predict_y, feed_dict={X_holder:X6})
import numpy as np
y = np.vstack([y1, y2, y3, y4, y5, y6])
y_argmax = np.argmax(y, 1)
y_argmax.shape
print('預(yù)測(cè)值的形狀：', y_argmax.shape)
commit_df = pd.DataFrame({'ImageId': range(1, 1+len(y_argmax)),
'Label': y_argmax})
fileName = 'kaggle_commit3.csv'
commit_df.to_csv(fileName, index=False)
print('預(yù)測(cè)結(jié)果已經(jīng)保存到文件', fileName)
上面一段代碼的運(yùn)行結(jié)果如下：
特征矩陣的形狀： (28000, 784)
預(yù)測(cè)值的形狀： (28000,)
預(yù)測(cè)結(jié)果已經(jīng)保存到文件 kaggle_commit3.csv
6.提交作答文件
比賽鏈接：https://www.kaggle.com/c/digit-recognizer
點(diǎn)擊下面的按鈕提交作答文件片吊。
如下圖所示绽昏，點(diǎn)擊上方紅色方框標(biāo)注處可以選擇作答文件提交上傳。
上傳成功后還需要點(diǎn)擊下方紅色方框提交俏脊。
提交成功后全谤，可以實(shí)時(shí)查看作答成績(jī)。
7.總結(jié)
1.自己電腦配置不足爷贫，使用云服務(wù)器極大的加快了工程部署和模型訓(xùn)練速度认然；
2.在kaggle經(jīng)典入門(mén)賽取得前2%的成績(jī)，把簡(jiǎn)單的事做到極致漫萄；
3.本文作者提供可以加載的模型只能取得0.99571的成績(jī)卷员。
