AlexNet(Alex Krizhevsky,ILSVRC2012冠軍)適合做圖像分類。層自左向右尾抑、自上向下讀取垂睬，關(guān)聯(lián)層分為一組缴渊，高度指蚁、寬度減小瞧哟，深度增加遣疯。深度增加減少網(wǎng)絡(luò)計算量。

訓(xùn)練模型數(shù)據(jù)集 Stanford計算機(jī)視覺站點Stanford Dogs http://vision.stanford.edu/aditya86/ImageNetDogs/ 。數(shù)據(jù)下載解壓到模型代碼同一路徑imagenet-dogs目錄下谈山。包含的120種狗圖像触机。80%訓(xùn)練笔宿，20%測試。產(chǎn)品模型需要預(yù)留原始數(shù)據(jù)交叉驗證较鼓。每幅圖像JPEG格式(RGB)谆级，尺寸不一吕朵。

圖像轉(zhuǎn)TFRecord文件，有助加速訓(xùn)練尸执，簡化圖像標(biāo)簽匹配，圖像分離利用檢查點文件對模型進(jìn)行不間斷測試晒夹。轉(zhuǎn)換圖像格式把顏色空間轉(zhuǎn)灰度读跷，圖像修改統(tǒng)一尺寸，標(biāo)簽除上每幅圖像。訓(xùn)練前只進(jìn)行一次預(yù)處理，時間較長锭沟。

glob.glob 枚舉指定路徑目錄，顯示數(shù)據(jù)集文件結(jié)構(gòu)⊥溆瑁“*”通配符可以實現(xiàn)模糊查找。文件名中8個數(shù)字對應(yīng)ImageNet類別WordNetID。ImageNet網(wǎng)站可用WordNetID查圖像細(xì)節(jié): http://www.image-net.org/synset?wnid=n02085620 交汤。

文件名分解為品種和相應(yīng)的文件名雏赦，品種對應(yīng)文件夾名稱。依據(jù)品種對圖像分組芙扎。枚舉每個品種圖像星岗，20%圖像劃入測試集。檢查每個品種測試圖像是否至少有全部圖像的18%戒洼。目錄和圖像組織到兩個與每個品種相關(guān)的字典俏橘，包含各品種所有圖像。分類圖像組織到字典中圈浇，簡化選擇分類圖像及歸類過程寥掐。

預(yù)處理階段靴寂，依次遍歷所有分類圖像，打開列表中文件召耘。用dataset圖像填充TFRecord文件百炬，把類別包含進(jìn)去。dataset鍵值對應(yīng)文件列表標(biāo)簽污它。record_location 存儲TFRecord輸出路徑剖踊。枚舉dataset，當(dāng)前索引用于文件劃分衫贬，每隔100m幅圖像德澈，訓(xùn)練樣本信息寫入新的TFRecord文件，加快寫操作進(jìn)程祥山。無法被TensorFlow識別為JPEG圖像，用try/catch忽略掉伏。轉(zhuǎn)為灰度圖減少計算量和內(nèi)存占用缝呕。tf.cast把RGB值轉(zhuǎn)換到[0,1)區(qū)間內(nèi)。標(biāo)簽按字符串存儲較高效斧散，最好轉(zhuǎn)換為整數(shù)索引或獨熱編碼秩1張量供常。

打開每幅圖像，轉(zhuǎn)換為灰度圖鸡捐，調(diào)整尺寸栈暇，添加到TFRecord文件。tf.image.resize_images函數(shù)把所有圖像調(diào)整為相同尺寸箍镜，不考慮長寬比源祈，有扭曲。裁剪色迂、邊界填充能保持圖像長寬比香缺。

按照TFRecord文件讀取圖像，每次加載少量圖像及標(biāo)簽歇僧。修改圖像形狀有助訓(xùn)練和輸出可視化图张。匹配所有在訓(xùn)練集目錄下TFRecord文件加載訓(xùn)練圖像。每個TFRecord文件包含多幅圖像诈悍。tf.parse_single_example只從文件提取單個樣本祸轮。批運(yùn)算可同時訓(xùn)練多幅圖像或單幅圖像，需要足夠系統(tǒng)內(nèi)存侥钳。

圖像轉(zhuǎn)灰度值為[0,1)浮點類型适袜，匹配convolution2d期望輸入。卷積輸出第1維和最后一維不改變舷夺，中間兩維發(fā)生變化痪蝇。tf.contrib.layers.convolution2d創(chuàng)建模型第1層鄙陡。weights_initializer設(shè)置正態(tài)隨機(jī)值，第一組濾波器填充正態(tài)分布隨機(jī)數(shù)躏啰。濾波器設(shè)置trainable趁矾，信息輸入網(wǎng)絡(luò)，權(quán)值調(diào)整给僵，提高模型準(zhǔn)確率毫捣。
max_pool把輸出降采樣。ksize帝际、strides ([1,2,2,1])蔓同，卷積輸出形狀減半。輸出形狀減小蹲诀，不改變?yōu)V波器數(shù)量(輸出通道)或圖像批數(shù)據(jù)尺寸斑粱。減少分量，與圖像(濾波器)高度脯爪、寬度有關(guān)则北。更多輸出通道，濾波器數(shù)量增加痕慢，2倍于第一層尚揣。多個卷積和池化層減少輸入高度、寬度掖举，增加深度快骗。很多架構(gòu)，卷積層和池化層超過5層塔次。訓(xùn)練調(diào)試時間更長方篮，能匹配更多更復(fù)雜模式。
圖像每個點與輸出神經(jīng)元建立全連接励负。softmax恭取，全連接層需要二階張量。第1維區(qū)分圖像熄守，第2維輸入張量秩1張量蜈垮。tf.reshape 指示和使用其余所有維，-1把最后池化層調(diào)整為巨大秩1張量裕照。
池化層展開攒发，網(wǎng)絡(luò)當(dāng)前狀態(tài)與預(yù)測全連接層整合。weights_initializer接收可調(diào)用參數(shù)晋南，lambda表達(dá)式返回截斷正態(tài)分布惠猿，指定分布標(biāo)準(zhǔn)差。dropout 削減模型中神經(jīng)元重要性负间。tf.contrib.layers.fully_connected 輸出前面所有層與訓(xùn)練中分類的全連接偶妖。每個像素與分類關(guān)聯(lián)姜凄。網(wǎng)絡(luò)每一步將輸入圖像轉(zhuǎn)化為濾波減小尺寸。濾波器與標(biāo)簽匹配趾访。減少訓(xùn)練态秧、測試網(wǎng)絡(luò)計算量，輸出更具一般性扼鞋。

訓(xùn)練數(shù)據(jù)真實標(biāo)簽和模型預(yù)測結(jié)果申鱼，輸入到訓(xùn)練優(yōu)化器(優(yōu)化每層權(quán)值)計算模型損失。數(shù)次迭代云头，每次提升模型準(zhǔn)確率捐友。大部分分類函數(shù)(tf.nn.softmax)要求數(shù)值類型標(biāo)簽。每個標(biāo)簽轉(zhuǎn)換代表包含所有分類列表索引整數(shù)溃槐。tf.map_fn 匹配每個標(biāo)簽并返回類別列表索引匣砖。map依據(jù)目錄列表創(chuàng)建包含分類列表。tf.map_fn 可用指定函數(shù)對數(shù)據(jù)流圖張量映射昏滴，生成僅包含每個標(biāo)簽在所有類標(biāo)簽列表索引秩1張量猴鲫。tf.nn.softmax用索引預(yù)測。

調(diào)試CNN影涉，觀察濾波器(卷積核)每輪迭代變化变隔。設(shè)計良好CNN规伐，第一個卷積層工作蟹倾，輸入權(quán)值被隨機(jī)初始化。權(quán)值通過圖像激活猖闪，激活函數(shù)輸出(特征圖)隨機(jī)鲜棠。特征圖可視化，輸出外觀與原始圖相似培慌，被施加靜力(static)豁陆。靜力由所有權(quán)值的隨機(jī)激發(fā)。經(jīng)過多輪迭代吵护，權(quán)值被調(diào)整擬合訓(xùn)練反饋盒音，濾波器趨于一致。網(wǎng)絡(luò)收斂馅而，濾波器與圖像不同細(xì)小模式類似祥诽。tf.image_summary得到訓(xùn)練后的濾波器和特征圖簡單視圖。數(shù)據(jù)流圖圖像概要輸出(image summary output)從整體了解所使用的濾波器和輸入圖像特征圖瓮恭。TensorDebugger雄坪，迭代中以GIF動畫查看濾波器變化。

文本輸入存儲在SparseTensor屯蹦，大部分分量為0维哈。CNN使用稠密輸入绳姨，每個值都重要，輸入大部分分量非0阔挠。

import tensorflow as tf
import glob
from itertools import groupby
from collections import defaultdict
sess = tf.InteractiveSession()
image_filenames = glob.glob("./imagenet-dogs/n02*/*.jpg")
image_filenames[0:2]
training_dataset = defaultdict(list)
testing_dataset = defaultdict(list)
image_filename_with_breed = map(lambda filename: (filename.split("/")[2], filename), image_filenames)
for dog_breed, breed_images in groupby(image_filename_with_breed, lambda x: x[0]):
    for i, breed_image in enumerate(breed_images):
        if i % 5 == 0:
            testing_dataset[dog_breed].append(breed_image[1])
        else:
            training_dataset[dog_breed].append(breed_image[1])
    breed_training_count = len(training_dataset[dog_breed])
    breed_testing_count = len(testing_dataset[dog_breed])
    breed_training_count_float = float(breed_training_count)
    breed_testing_count_float = float(breed_testing_count)
    assert round(breed_testing_count_float / (breed_training_count_float + breed_testing_count_float), 2) > 0.18, "Not enough testing images."
print "training_dataset testing_dataset END ------------------------------------------------------"
def write_records_file(dataset, record_location):
    writer = None
    current_index = 0
    for breed, images_filenames in dataset.items():
        for image_filename in images_filenames:
            if current_index % 100 == 0:
                if writer:
                    writer.close()
                record_filename = "{record_location}-{current_index}.tfrecords".format(
                    record_location=record_location,
                    current_index=current_index)
                writer = tf.python_io.TFRecordWriter(record_filename)
                print record_filename + "------------------------------------------------------" 
            current_index += 1
            image_file = tf.read_file(image_filename)
            try:
                image = tf.image.decode_jpeg(image_file)
            except:
                print(image_filename)
                continue
            grayscale_image = tf.image.rgb_to_grayscale(image)
            resized_image = tf.image.resize_images(grayscale_image, [250, 151])
            image_bytes = sess.run(tf.cast(resized_image, tf.uint8)).tobytes()
            image_label = breed.encode("utf-8")
            example = tf.train.Example(features=tf.train.Features(feature={
                'label': tf.train.Feature(bytes_list=tf.train.BytesList(value=[image_label])),
                'image': tf.train.Feature(bytes_list=tf.train.BytesList(value=[image_bytes]))
            }))
            writer.write(example.SerializeToString())
    writer.close()
write_records_file(testing_dataset, "./output/testing-images/testing-image")
write_records_file(training_dataset, "./output/training-images/training-image")
print "write_records_file testing_dataset training_dataset END------------------------------------------------------"
filename_queue = tf.train.string_input_producer(
tf.train.match_filenames_once("./output/training-images/*.tfrecords"))
reader = tf.TFRecordReader()
_, serialized = reader.read(filename_queue)
features = tf.parse_single_example(
serialized,
    features={
        'label': tf.FixedLenFeature([], tf.string),
        'image': tf.FixedLenFeature([], tf.string),
    })
record_image = tf.decode_raw(features['image'], tf.uint8)
image = tf.reshape(record_image, [250, 151, 1])
label = tf.cast(features['label'], tf.string)
min_after_dequeue = 10
batch_size = 3
capacity = min_after_dequeue + 3 * batch_size
image_batch, label_batch = tf.train.shuffle_batch(
    [image, label], batch_size=batch_size, capacity=capacity, min_after_dequeue=min_after_dequeue)
print "load image from TFRecord END------------------------------------------------------"
float_image_batch = tf.image.convert_image_dtype(image_batch, tf.float32)
conv2d_layer_one = tf.contrib.layers.convolution2d(
    float_image_batch,
    num_outputs=32,
    kernel_size=(5,5),
    activation_fn=tf.nn.relu,
    weights_initializer=tf.random_normal,
    stride=(2, 2),
    trainable=True)
pool_layer_one = tf.nn.max_pool(conv2d_layer_one,
    ksize=[1, 2, 2, 1],
    strides=[1, 2, 2, 1],
    padding='SAME')
conv2d_layer_one.get_shape(), pool_layer_one.get_shape()
print "conv2d_layer_one pool_layer_one END------------------------------------------------------"
conv2d_layer_two = tf.contrib.layers.convolution2d(
    pool_layer_one,
    num_outputs=64,
    kernel_size=(5,5),
    activation_fn=tf.nn.relu,
    weights_initializer=tf.random_normal,
    stride=(1, 1),
    trainable=True)
pool_layer_two = tf.nn.max_pool(conv2d_layer_two,
    ksize=[1, 2, 2, 1],
    strides=[1, 2, 2, 1],
    padding='SAME')
conv2d_layer_two.get_shape(), pool_layer_two.get_shape()
print "conv2d_layer_two pool_layer_two END------------------------------------------------------"
flattened_layer_two = tf.reshape(
    pool_layer_two,
    [
        batch_size,
        -1
    ])
flattened_layer_two.get_shape()
print "flattened_layer_two END------------------------------------------------------"
hidden_layer_three = tf.contrib.layers.fully_connected(
    flattened_layer_two,
    512,
    weights_initializer=lambda i, dtype: tf.truncated_normal([38912, 512], stddev=0.1),
    activation_fn=tf.nn.relu
)
hidden_layer_three = tf.nn.dropout(hidden_layer_three, 0.1)
final_fully_connected = tf.contrib.layers.fully_connected(
    hidden_layer_three,
    120,
    weights_initializer=lambda i, dtype: tf.truncated_normal([512, 120], stddev=0.1)
)
print "final_fully_connected END------------------------------------------------------"
labels = list(map(lambda c: c.split("/")[-1], glob.glob("./imagenet-dogs/*")))
train_labels = tf.map_fn(lambda l: tf.where(tf.equal(labels, l))[0,0:1][0], label_batch, dtype=tf.int64)
loss = tf.reduce_mean(
    tf.nn.sparse_softmax_cross_entropy_with_logits(
        final_fully_connected, train_labels))
batch = tf.Variable(0)
learning_rate = tf.train.exponential_decay(
    0.01,
    batch * 3,
    120,
    0.95,
    staircase=True)
optimizer = tf.train.AdamOptimizer(
    learning_rate, 0.9).minimize(
    loss, global_step=batch)
train_prediction = tf.nn.softmax(final_fully_connected)
print "train_prediction END------------------------------------------------------"
filename_queue.close(cancel_pending_enqueues=True)
coord.request_stop()
coord.join(threads)
print "END------------------------------------------------------"

參考資料：
《面向機(jī)器智能的TensorFlow實踐》

歡迎加我微信交流：qingxingfengzi
我的微信公眾號:qingxingfengzigz
我老婆張幸清的微信公眾號：qingqingfeifangz