文章代碼來源:《deep learning on keras》夺克,非常好的一本書,大家如果英語好嚎朽,推薦直接閱讀該書铺纽,如果時(shí)間不夠,可以看看此系列文章哟忍,文章為我自己翻譯的內(nèi)容加上自己的一些思考狡门,水平有限陷寝,多有不足,請多指正其馏,翻譯版權(quán)所有凤跑,若有轉(zhuǎn)載,請先聯(lián)系本人叛复。
個(gè)人方向?yàn)閿?shù)值計(jì)算仔引,日后會(huì)向深度學(xué)習(xí)和計(jì)算問題的融合方面靠近,若有相近專業(yè)人士褐奥,歡迎聯(lián)系咖耘。
系列文章:
一、搭建屬于你的第一個(gè)神經(jīng)網(wǎng)絡(luò)
二抖僵、訓(xùn)練完的網(wǎng)絡(luò)去哪里找
三鲤看、【keras實(shí)戰(zhàn)】波士頓房價(jià)預(yù)測
四缘揪、keras的function API
五耍群、keras callbacks使用
六、機(jī)器學(xué)習(xí)基礎(chǔ)Ⅰ:機(jī)器學(xué)習(xí)的四個(gè)標(biāo)簽
七找筝、機(jī)器學(xué)習(xí)基礎(chǔ)Ⅱ:評估機(jī)器學(xué)習(xí)模型
八蹈垢、機(jī)器學(xué)習(xí)基礎(chǔ)Ⅲ:數(shù)據(jù)預(yù)處理、特征工程和特征學(xué)習(xí)
九袖裕、機(jī)器學(xué)習(xí)基礎(chǔ)Ⅳ:過擬合和欠擬合
十曹抬、機(jī)器學(xué)習(xí)基礎(chǔ)Ⅴ:機(jī)器學(xué)習(xí)的一般流程十一、計(jì)算機(jī)視覺中的深度學(xué)習(xí):卷積神經(jīng)網(wǎng)絡(luò)介紹
十二急鳄、計(jì)算機(jī)視覺中的深度學(xué)習(xí):從零開始訓(xùn)練卷積網(wǎng)絡(luò)
十三谤民、計(jì)算機(jī)視覺中的深度學(xué)習(xí):使用預(yù)訓(xùn)練網(wǎng)絡(luò)
十四、計(jì)算機(jī)視覺中的神經(jīng)網(wǎng)絡(luò):可視化卷積網(wǎng)絡(luò)所學(xué)到的東西
書中打了一個(gè)形象的比喻:我們之前訓(xùn)練模型就像扔紙飛機(jī)一樣疾宏,疊好了张足,給個(gè)初速度,到底怎么飛坎藐,落到哪里为牍,我們無法控制,今天我們要學(xué)會(huì)的就是如何造一個(gè)可以被控制的飛機(jī)岩馍。
使用callbacks來模型正在訓(xùn)練的時(shí)候來控制
我們之前訓(xùn)練的過程是先訓(xùn)練一遍碉咆,然后得到一個(gè)驗(yàn)證集的識(shí)別率變化趨勢,從而知道最佳的epoch蛀恩,設(shè)置epoch疫铜,再訓(xùn)練一遍,得到最終結(jié)果双谆,這樣很浪費(fèi)時(shí)間壳咕。
一個(gè)好方法就是在測試識(shí)別率不再上升的時(shí)候励稳,我們終止訓(xùn)練就可以了,callback可以幫助我們做到這一點(diǎn)囱井,callback是一個(gè)obj類型的驹尼,它可以讓模型去擬合,也常在各個(gè)點(diǎn)被調(diào)用庞呕。它和所有模型的狀態(tài)和表現(xiàn)的數(shù)據(jù)新翎,能夠采取措施打斷訓(xùn)練,保存模型住练,加載不同的權(quán)重地啰,或者替代模型狀態(tài)。
callbacks可以用來做這些事情:
- 模型斷點(diǎn)續(xù)訓(xùn):保存當(dāng)前模型的所有權(quán)重
- 提早結(jié)束:當(dāng)模型的損失不再下降的時(shí)候就終止訓(xùn)練讲逛,當(dāng)然亏吝,會(huì)保存最優(yōu)的模型。
- 動(dòng)態(tài)調(diào)整訓(xùn)練時(shí)的參數(shù)盏混,比如優(yōu)化的學(xué)習(xí)速度蔚鸥。
- 等等
earlystopping和modelcheckpoint
import keras
# Callbacks are passed to the model fit the `callbacks` argument in `fit`,
# which takes a list of callbacks. You can pass any number of callbacks.
callbacks_list = [
# This callback will interrupt training when we have stopped improving
keras.callbacks.EarlyStopping(
# This callback will monitor the validation accuracy of the model
monitor='acc',
# Training will be interrupted when the accuracy
# has stopped improving for *more* than 1 epochs (i.e. 2 epochs)
patience=1,
),
# This callback will save the current weights after every epoch
keras.callbacks.ModelCheckpoint(
filepath='my_model.h5', # Path to the destination model file
# The two arguments below mean that we will not overwrite the
# model file unless `val_loss` has improved, which
# allows us to keep the best model every seen during training.
monitor='val_loss',
save_best_only=True,
)
]
# Since we monitor `acc`, it should be part of the metrics of the model.
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])
# Note that since the callback will be monitor validation accuracy,
# we need to pass some `validation_data` to our call to `fit`.
model.fit(x, y,
epochs=10,
batch_size=32,
callbacks=callbacks_list,
validation_data=(x_val, y_val))
monitor為選擇的檢測指標(biāo),我們這里選擇檢測'acc'識(shí)別率為指標(biāo)许赃,patience就是我們能讓訓(xùn)練停止變好多少epochs才終止訓(xùn)練止喷,這里選擇了1,而modelcheckpoint就起到了存儲(chǔ)最優(yōu)的模型的作用混聊,filepath為我們存儲(chǔ)的位置和模型名稱弹谁,以.h5為后綴,monitor為檢測的指標(biāo)句喜,這里我們檢測驗(yàn)證集里面的成功率预愤,save_best_only代表我們只保存最優(yōu)的訓(xùn)練結(jié)果。
而validation_data就是給定的驗(yàn)證集數(shù)據(jù)咳胃。
學(xué)習(xí)率減少callback
callbacks_list = [
keras.callbacks.ReduceLROnPlateau(
# This callback will monitor the validation loss of the model
monitor='val_loss',
# It will divide the learning by 10 when it gets triggered
factor=0.1,
# It will get triggered after the validation loss has stopped improving
# for at least 10 epochs
patience=10,
)
]# Note that since the callback will be monitor validation loss,
# we need to pass some `validation_data` to our call to `fit`.
model.fit(x, y,
epochs=10,
batch_size=32,
callbacks=callbacks_list,
validation_data=(x_val, y_val))
翻譯一下植康,就是如果連續(xù)10個(gè)批次,val_loss不再下降拙绊,就把學(xué)習(xí)率弄到原來的0.1倍向图。
自己造callback
如果內(nèi)置的那些callback操作還滿足不了你的需求,這里給出了如何自己造callback的方法标沪。
# Called at the start of every epoch
on_epoch_begin
# Called at the end of every epoch
on_epoch_end
# Called right before processing each batch
on_batch_begin
# Called right after processing each batch
on_batch_end
# Called at the start of training
on_train_begin
# Called at the end of training
on_train_end
下面給出一個(gè)將激活值以數(shù)組的形式存進(jìn)磁盤的callback:
import keras
import numpy as np
class ActivationLogger(keras.callbacks.Callback):
def set_model(self, model):
# This method is called by the parent model
# before training, to inform the callback
# of what model will be calling it
self.model = model
layer_outputs = [layer.output for layer in model.layers]
# This is a model instance that returns the activations of every layer
self.activations_model = keras.models.Model(model.input, layer_outputs)
def on_epoch_end(self, epoch, logs=None):
if self.validation_data is None:
raise RuntimeError('Requires validation_data.')
# Obtain first input sample of the validation data
validation_sample = self.validation_data[0][0:1]
activations = self.activations_model.predict(validation_sample)
# Save arrays to disk
f = open('activations_at_epoch_' + str(epoch) + '.npz', 'w')
np.savez(f, activations)
f.close()