介紹
這里記錄keras文檔FAQ中在工作中用到的一些問題和技巧济榨。參考自這里
主要包括：
- 多GPU訓練
- 獲取中間層的輸出
- 凍結（freeze）某些層
多GPU運行
運行一個模型在多個gpu上有兩種方法：數(shù)據并行了赌、設備并行
數(shù)據并行
數(shù)據并行是將一個模型在每個GPU上都部署一份進行訓練，同時處理川慌，加速訓練。
keras有內置的工具keras.utils.multi_gpu_model，該模塊可以為任何自定義模型產生一個數(shù)據并行模型蜒犯，在多gpu上達到線性擬合加速（quasi-linear speedup）抽活。
更多可以參考multi_gpu_model
這里給出一個例子
from keras.utils import multi_gpu_model
parallel_model = multi_gpu_model(model, gpus=8)
parallel_model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
# This `fit` call will be distributed on 8 GPUs.
parallel_model.fit(x, y, epochs=20, batch_size=256) # batch size: 256, each GPU will process 32 samples.
設備并行
設備并行是在不同的GPU上運行一個模型的多個分支硫戈，多用于模型中有多個并行的結構，例如AlexNet的卷積就是放到多個GPU上運行的下硕。提供一個例子
# Model where a shared LSTM is used to encode two different sequences in parallel
input_a = keras.Input(shape=(140, 256))
input_b = keras.Input(shape=(140, 256))
shared_lstm = keras.layers.LSTM(64)
# Process the first sequence on one GPU
with tf.device_scope('/gpu:0'):
encoded_a = shared_lstm(tweet_a)
# Process the next sequence on another GPU
with tf.device_scope('/gpu:1'):
encoded_b = shared_lstm(tweet_b)
# Concatenate results on CPU
with tf.device_scope('/cpu:0'):
merged_vector = keras.layers.concatenate([encoded_a, encoded_b], axis=-1)
如何獲取某一層的輸出
從Model中獲取輸出
創(chuàng)建一個模型丁逝，直接輸出模型預測的結果汁胆。如下。
from keras.models import Model
model = ... # create the original model
layer_name = 'my_layer'
intermediate_layer_model = Model(inputs=model.input,
outputs=model.get_layer(layer_name).output)
intermediate_output = intermediate_layer_model.predict(data)
使用keras function
from keras import backend as K
# with a Sequential model
get_3rd_layer_output = K.function([model.layers[0].input], [model.layers[3].output])
layer_output = get_3rd_layer_output([x])[0]
如果模型有dropout霜幼、BN
如果模型有dropout嫩码、BN這種訓練期有效、測試期無效的層罪既，需要給一個指標（flag）铸题。如下
get_3rd_layer_output = K.function([model.layers[0].input, K.learning_phase()], [model.layers[3].output])
# output in test mode = 0
layer_output = get_3rd_layer_output([x, 0])[0]
# output in train mode = 1
layer_output = get_3rd_layer_output([x, 1])[0]
如何凍結（freeze）某些層
凍結代表在訓練時期，某一些層的參數(shù)是不變的琢感。這個多用于微調模型丢间。
只需要在創(chuàng)建某一層的時候設定trainable參數(shù)為False。
frozen_layer = Dense(32, trainable=False)
或者在創(chuàng)建之后設定驹针，如下烘挫。
x = Input(shape=(32,))
layer = Dense(32)
layer.trainable = False
y = layer(x)
frozen_model = Model(x, y)
# in the model below, the weights of `layer` will not be updated during training
frozen_model.compile(optimizer='rmsprop', loss='mse')
layer.trainable = True
trainable_model = Model(x, y)
# with this model the weights of the layer will be updated during training
# (which will also affect the above model since it uses the same layer instance)
trainable_model.compile(optimizer='rmsprop', loss='mse')
frozen_model.fit(data, labels) # this does NOT update the weights of `layer`
trainable_model.fit(data, labels) # this updates the weights of `layer`
