簡(jiǎn)介
Caffe網(wǎng)絡(luò)通常由數(shù)據(jù)層、網(wǎng)絡(luò)層（通常我們認(rèn)為的網(wǎng)絡(luò)結(jié)構(gòu)）等限、Loss層組成愕鼓。Caffe中數(shù)據(jù)以Blob的形式在Layer之間傳播囤采。
每個(gè)Layer包含三個(gè)基本的操作Setup代态、Forward狐粱、Backward。
Setup: 在模型初始化時(shí)重置 layers 及其相互之間的連接 ;
Forward: 從 bottom 層中接收數(shù)據(jù)胆数，進(jìn)行計(jì)算后將輸出送入到 top 層中;
Backward: 給定相對(duì)于 top 層輸出的梯度，計(jì)算其相對(duì)于輸入的梯度互墓，并傳遞到 bottom層必尼。
Caffe中各層的參數(shù)可以查閱 https://github.com/BVLC/caffe/blob/master/src/caffe/proto/caffe.proto
Caffe的安裝參考：http://www.reibang.com/p/992351958ed4
下面以特征點(diǎn)檢測(cè)模型為例。新建模型定義文件landmark.prototxt
首先給模型起一個(gè)名字
name:"ONet"
一、定義網(wǎng)絡(luò)結(jié)構(gòu)
1判莉、數(shù)據(jù)層
Caffe又7種數(shù)據(jù)層（Data豆挽、MemoryData、HDF5Data券盅、HDF5Output帮哈、ImageData、WindowData锰镀、DummyData）娘侍，除此之外還可以自定義數(shù)據(jù)層。
這里我們自定義數(shù)據(jù)層泳炉。
在landmark.prototxt中定義數(shù)據(jù)層：
layer{
name: "data"
type: "Python"
top: "data"
top: "label"
python_param{
module: "my_data_layer"
layer: "ImageDataLayer"
param_str:"{\'source\':\'../data/preproc/data/112/landmark_aug.txt\', \'batch_size\':384, \'shuffle\':True, \'size\':(112,112)}"
}
include: {phase: TRAIN}
}
layer{
name: "data"
type: "Python"
top: "data"
top: "label"
python_param{
module: "my_data_layer"
layer: "ImageDataLayer"
param_str:"{\'source\':\'../data/preproc/data/112/landmark_aug.txt\', \'batch_size\':384, \'shuffle\':False, \'size\':(112,112)}"
}
include: {phase: TEST}
}
name表示該層的名字憾筏，可以隨意取。
其中type:"Python"需要在編譯安裝Caffe的時(shí)候開啟Python Layer的支持花鹅，在Makefile.config中把這一行WITH_PYTHON_LAYER=1的注釋去掉氧腰。
top表示該層的輸出
bottom表示該層的輸入，這里沒有輸入刨肃。
可以有多個(gè)top和bottom古拴。
注意：在數(shù)據(jù)層中，至少有一個(gè)命名為data的top https://zhuanlan.zhihu.com/p/34606014
module表示自定義Layer的python代碼路徑,這個(gè)文件需要和landmark.prototxt在同一目錄下真友，注意不要加后綴.py黄痪。
layer表示python代碼中的類名。
param_str用來傳遞自定義的變量锻狗，類型為字符串
include: {phase: TRAIN}表示訓(xùn)練時(shí)使用的Layer.
include: {phase: TEST}表示測(cè)試時(shí)使用的Layer.
對(duì)應(yīng)的Python代碼满力，文件名為my_data_layer.py：
import caffe
import numpy as np
import random
import sys
class ImageDataLayer(caffe.Layer):
def setup(self, bottom, top):
self.top_names = ['data', 'label']
params = eval(self.param_str)
self.batch_size =params['batch_size']
self.shuffle = params['shuffle']
self.batch_loader = BatchLoader(params)
height, width = params['size']
top[0].reshape(self.batch_size, 3, height, width)
top[1].reshape(self.batch_size, 127*2)
def forward(self, bottom, top):
batch = self.batch_loader.next()
imgs = []
labels = []
if self.shuffle:
random.shuffle(batch)
for it in batch:
items = it.split()
img = cv2.imread(items[0])
if img is None:
print("cv2.read %s is None, exit.".format(items[0]))
sys.exit(1)
img = np.transpose(img, (2,0,1)) #convert (height, width, 3) to (3, height, width)
label = map(float, items[1:])
imgs.append(img)
labels.append(label)
top[0].data = np.array(imgs)
top[1].data = np.array(labels)
def reshape(self, bottom, top):
pass
def backward(self, top, propagate_down, bottom):
pass
class BatchLoader(object):
def __init__(self, params):
self.source = params['source']
self.batch_size = params['batch_size']
self.size = params['size']
self.isshuffle = params['shuffle']
self.datalist = open(self.source,"r").read().splitlines()
def __iter__(self):
return self
def next(self):
len_data = len(self.datalist)
for i in range(0, len_data, self.batch_size):
pad = i+self.batch_size - len_data
pad = pad if pad >0 else 0
batch = self.datalist[i:i+self.batch_size] + random.sample(self.datalist[0:i], pad)
yield batch
2、中間網(wǎng)絡(luò)層
定義第一層卷積
layer{
name: "conv1"
type: "Convolution"
bottom: "data"
top: "conv1"
param{
lr_mult: 1
}
param{
lr_mult: 2
}
convolution_param{
num_output: 32
kernel_size: 3
stride: 1
weight_filler:{
type: "xavier"
}
bias_filler{
type: "constant"
}
}
}
其中第一個(gè)lr_mult:1表示權(quán)重的學(xué)習(xí)率為：1base_lr (在solver.prototxt中定義)
第二個(gè)lr_mult:2表示偏置的學(xué)習(xí)率為：2base_lr轻纪。
num_output表示輸出通道數(shù)
kernel_size表示卷積核大小油额，寬高都為kernel_size。如果寬高不相等就分別設(shè)置kernel_h刻帚、kernel_w潦嘶。
stride表示步長(zhǎng)。
weight_filler type權(quán)重初始化方式
bias_filler type偏置初始化方式崇众，設(shè)置為constant時(shí)默認(rèn)值為0
pad:填充值掂僵，默認(rèn)為0。設(shè)置為2時(shí)左右各填充2個(gè)顷歌。
池化層：
layer{
name: "pool1"
type: "Pooling"
bottom: "conv1"
top: "pool1"
pooling_param{
pool: MAX
kernel_size: 3
stride: 2
pad: 1
}
}
prelu層：
layer{
name: "prelu1"
type: "PReLU"
bottom: "pool1"
top: "pool1"
}
用Reshape層實(shí)現(xiàn)Flatten操作(當(dāng)然有專用的Flatten層)：
layer{
name: "flatten"
type: "Reshape"
bottom: "prelu4"
top: "flatten"
reshape_param{
shape{
dim: 0
dim: -1
}
}
}
全連接層：
layer{
name: "landmark_pred"
type: "InnerProduct"
bottom: "prelu5"
top: "landmark_pred"
param{
lr_mult: 1
}
param{
lr_mult: 2
}
inner_product_param{
num_output: 254
weight_filler:{
type: "xavier"
}
bias_filler{
type: "constant"
}
}
}
自定義Loss層：
layer{
name: "landmark_loss"
type: "Python"
top: "landmark_loss"
bottom: "landmark_pred"
bottom: "label"
python_param{
module: "wing_loss_layer"
layer: "WingLossLayer"
param_str : "{\'w\':1.0, \'eplison\':0.2}"
}
# set loss weight so Caffe knows this is a loss layer.
# since PythonLayer inherits directly from Layer, this isn't automatically
# known to Caffe
loss_weight: 1
}
需要加上loss_weight參數(shù)锰蓬，否則會(huì)不收斂
自定義Loss(wing loss)的python代碼:
import caffe
import numpy as np
class WingLossLayer(caffe.Layer):
def setup(self, bottom, top):
if len(bottom) != 2:
raise Exception("Need two bottom for WingLossLayer")
params = eval(self.param_str)
self.w = params['w']
self.eplison = params['eplison']
def reshape(self, bottom, top):
if bottom[0].count != bottom[1].count:
raise Exception("Inputs must have the save dimension")
self.diff = np.zeros_like(bottom[0].data, dtype=np.float32)
top[0].reshape(1)
def forward(self, bottom, top):
#tag,need reshape bottom[0] and bottom[1],maybe lmdb don't need
self.diff = bottom[0].data - bottom[1].data
idx = np.abs(self.diff) < self.w
idx1 = np.abs(self.diff) >= self.w
top[0].data[...] = (\
np.sum(self.w * np.log(1.0/self.eplison * np.abs(self.diff[idx]) + 1.)) +\
np.sum(np.abs(self.diff[idx1]) - (self.w - self.w * np.log(1.0 + self.w/self.eplison)))\
) / bottom[0].num
def backward(self, top, propagate_down, bottom):
idx0 = (0. < self.diff) & (self.diff < self.w)
idx1 = (-self.w < self.diff) & (self.diff < 0.)
idx2 = self.diff >= self.w
idx3 = self.diff <= -self.w
#print "idx2"
for i in range(0,2):
if not propagate_down[i]:
continue
if i == 0:
sign = 1
else:
sign = -1
bottom[i].diff[idx0] = sign * 1.0 * (self.w / (1. + 1.0/self.eplison * np.abs(self.diff[idx0]))) / bottom[i].num
bottom[i].diff[idx1] = sign * (-1.0) * (self.w / (1. + 1.0/self.eplison * np.abs(self.diff[idx1]))) / bottom[i].num
bottom[i].diff[idx2] = sign * 1.0 / bottom[i].num
bottom[i].diff[idx3] = sign * (-1.0) / bottom[i].num
3、Caffe的BN層
Caffe的BN層由BatchNorm 層和Scale層組成眯漩。BatchNorm減均值芹扭，Scale層除方差麻顶。示例如下：
layer{
name: "conv1/bn"
type: "BatchNorm"
bottom: "conv1"
top: "conv1/bn"
batch_norm_param{
moving_average_fraction: 0.997
eps: 1e-3
}
}
layer{
name: "conv1/scale"
type: "Scale"
bottom: "conv1/bn"
top: "conv1/scale"
scale_param{
bias_term: true
}
}
【參考】
Caffe 中 BN（BatchNorm ) 層的參數(shù)均值、方差和滑動(dòng)系數(shù)解讀
caffe中的BatchNorm層
Caffe中的BatchNorm實(shí)現(xiàn)
淺談Batch Normalization及其Caffe實(shí)現(xiàn)
4舱卡、定義DepthwiseConv層
【參考】
https://mc.ai/depthwise-separable-convolution%E2%80%8A-%E2%80%8Ain-caffe-framework/
How to get Depthwise Separable Convolution in Caffe ?
In caffe framework, We can use normal convolution layer as depthwise convolution layer by specifying number of groups as equal to number of input channels.
https://github.com/shicai/MobileNet-Caffe/blob/master/mobilenet_deploy.prototxt
https://github.com/farmingyard/caffe-mobilenet/blob/master/mobilenet_1by2_deploy.prototxt
二辅肾、定義優(yōu)化參數(shù)
設(shè)置優(yōu)化參數(shù)，文件名定為solver.prototxt
net: "landmark.prototxt"
test_iter: 100
test_interval: 500
base_lr: 0.0001
momentum: 0.9
momentum2: 0.999
type: "Adam"
lr_policy: "fixed"
display: 100
max_iter: 30000
snapshot: 5000
snapshot_prefix: "../../checkpoint/caffe"
solver_mode: GPU
三轮锥、訓(xùn)練
執(zhí)行命令：caffe.bin train --solver=solver.prototxt -gpu
四矫钓、部署
部署的時(shí)候需用部署專用的模型結(jié)構(gòu)，其實(shí)就是去掉了訓(xùn)練階段的數(shù)據(jù)層和loss層（一般如此）舍杜，然后在首層加上Input層新娜，Input層如下：
layer {
name: "data"
type: "Input"
top: "data"
input_param { shape: { dim: 1 dim: 3 dim: 112 dim: 112 } }
}
1、pycaffe部署蝴簇，就是用python API進(jìn)行推理
import caffe
import numpy as np
import cv2
import random
import os
caffe.set_mode_cpu()
class Inference():
def __init__(self, deploy_proto, model):
self.net = caffe.Net(deploy_proto, model, caffe.TEST)
self.transformer = caffe.io.Transformer({'data':self.net.blobs['data'].data.shape})
self.transformer.set_transpose('data', (2,0,1))
#self.transformer.set_mean('data', np.array([127.5,127.5,127.5]))
#self.transformer.set_raw_scale('data', 1/128.0)
#self.transformer.set_channel_swap('data', (2,1,0))
def forward(self, img):
self.net.blobs['data'].data[...] = self.transformer.preprocess('data',img)
out = self.net.forward()
landmarks = self.net.blobs['landmark_pred'].data[0]
return landmarks
#調(diào)用
if __name__ == "__main__":
model_path = "onet1_deploy.prototxt"
weight_path = "../../checkpoint/caffe-onet1/onet_iter_50000.caffemodel"
net = Inference(model_path, weight_path)
img_orig = cv2.imread(path)
img = np.asarray(img_orig).astype(np.float32)
img = (img-127.5)/128.0
landmarks = net.forward(img)
【參考】
Caffe for Python 官方教程(翻譯)
http://www.voidcn.com/article/p-pgjwtpri-st.html
caffe學(xué)習(xí)（六）：使用python調(diào)用訓(xùn)練好的模型來分類（Ubuntu）
Caffe學(xué)習(xí)筆記(七)：使用訓(xùn)練好的model做預(yù)測(cè)(mnist)
Caffe學(xué)習(xí)系列(20)：用訓(xùn)練好的caffemodel來進(jìn)行分類
Caffe python layer方法執(zhí)行時(shí)機(jī)
參考
http://caffecn.cn/?/page/tutorial
http://manutdzou.github.io/2016/05/15/Caffe-Document.html
caffe添加python數(shù)據(jù)層（ImageData）
caffe中添加Python層
【caffe中添加C++層】Caffe添加自定義層-自定義loss C++ caffe layer: https://github.com/JunrQ/caffe-layer (內(nèi)含wing loss layer杯活、depthwise conv layer、coord2heatmap layer熬词、heatmap loss layer)
【自定義Loss】caffe-python-layer 的自定義
用python自定義caffe loss層
https://github.com/BVLC/caffe/blob/master/examples/pycaffe/layers/pyloss.py
【wing loss】 : https://github.com/DaChaoXc/caffe-layer-code/blob/master/wingLoss.py
caffe常見優(yōu)化器使用參數(shù)
caffe的特殊層
caffe solver文件個(gè)參數(shù)的意義
caffe solver參數(shù)詳解
caffe學(xué)習(xí)筆記3：Loss和多個(gè)Loss合并問題
Windows Caffe 學(xué)習(xí)筆記（四）搭建自己的網(wǎng)絡(luò)旁钧，訓(xùn)練和測(cè)試MNIST手寫字體庫
https://github.com/RiweiChen/DeepFace/blob/master/FaceAlignment/try1_2/train_val.prototxt
caffe 中base_lr、weight_decay互拾、lr_mult歪今、decay_mult代表什么意思？
caffe入門應(yīng)用方法(一)——網(wǎng)絡(luò)層參數(shù)配置解析
Caffe 中 BN（BatchNorm ) 層的參數(shù)均值颜矿、方差和滑動(dòng)系數(shù)解讀
Caffe傻瓜系列(3):激活層（Activiation Layers)及參數(shù)
https://gist.github.com/jyegerlehner/b2f073aa8e213f0a9167
CAFFE官方教程學(xué)習(xí)筆記
Caffe通過代碼生成prototxt網(wǎng)絡(luò)文件：Caffe的深度學(xué)習(xí)訓(xùn)練全過程
