LeNet-5：Gradient-Based Learning Applied to Document Recognition

1998年的LeNet-5是CNN的經(jīng)典之作歹苦，但是該模型在后來未能火起來犬缨，主要原因是當時的計算力不足和數(shù)據(jù)量的不足花颗。并且當時的SVM在可承受計算量的情況下诫给，達到，甚至超過了神經(jīng)模型柄沮。CNN雖然在當時沒能受到重視审孽，但是并沒有掩蓋CNN的強大能力。

下圖是LeNet的架構(gòu)圖：

LeNet-5架構(gòu)圖

LeNet由兩層conv躺孝，兩層pool享扔，三層fc組成。

以下是用TensorFlow和Keras混合編寫的LeNet-5模型：

import tensorflow as tf
keras = tf.keras
from tensorflow.python.keras.layers import Conv2D,MaxPool2D,Dropout,Flatten,Dense

def inference(inputs,
              num_classes=10,
              is_training=True,
              dropout_keep_prob=0.5):
  '''

  inputs: a tensor of images
  num_classes: the num of category.
  is_training: set ture when it used for training
  dropout_keep_prob: the rate of dropout during training
  '''

  x = inputs
  # conv1
  x = Conv2D(6, [5,5], 1, activation='relu', name='conv1')(x) 
  # pool1
  x = MaxPool2D([2,2], 2, name='pool1')(x)
  # conv2
  x = Conv2D(16, [5,5], 1, activation='relu', name='conv2')(x)
  # pool2
  x = MaxPool2D([2,2], 2, name='pool2')(x)
  x = Flatten(name='pool2_flatten')(x)
  if is_training:
    x = Dropout(rate=dropout_keep_prob)(x)
  # fc3
  x = Dense(120, activation='relu', name='fc3')(x)
  if is_training:
    x = Dropout(rate=dropout_keep_prob)(x)
  # fc4
  x = Dense(84, activation='relu', name='fc4')(x)
  # logits
  logits = Dense(num_classes, activation='softmax')(x)
  return logits

if __name__ == '__main__':
  x = tf.placeholder(tf.float32, [None, 784])
  images = tf.reshape(x,[-1,28,28,1])
  labels = tf.placeholder(tf.float32, [None, 10])
  dropout_keep_prob = tf.placeholder(tf.float32)
  logits = inference(inputs=images,
                    num_classes=10,
                    is_training=True,
                    dropout_keep_prob=dropout_keep_prob)

  with tf.variable_scope('costs'):
    cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(
        logits=logits, labels=labels), name='xent')

  with tf.variable_scope('train'):
    train_op = tf.train.AdamOptimizer().minimize(cost)

  from tensorflow.examples.tutorials.mnist import input_data
  mnist_data = input_data.read_data_sets('./mnist_data', one_hot=True)

  acc_test = tf.divide(
      tf.reduce_sum(keras.metrics.categorical_accuracy(labels,logits)),
      len(mnist_data.test.labels))

  with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    Writer = tf.summary.FileWriter('./tmp',sess.graph)

    for i in range(200):
      batch_x,batch_y = mnist_data.train.next_batch(50)
      _, c = sess.run([train_op, cost], feed_dict={
          x:batch_x, labels:batch_y, dropout_keep_prob:0.5})
      acc_test1 = sess.run(acc_test, feed_dict={
          x:mnist_data.test.images, labels:mnist_data.test.labels, dropout_keep_prob:1})
      print('step:%04d  cost:%.4f  test_acc:%.3f'%(i+1,c,acc_test1))

代碼以Keras來實現(xiàn)模型的inference過程植袍，其他部分使用Tensorflow惧眠，這樣可以大大減少構(gòu)建模型的復雜度。
LeNet-5的具體配置：