Semantic Segmentation
- 語義分割: 即圖像像素級(jí)別的分類
- 將圖像分塊,每一個(gè)區(qū)域代表有意義的物體
- 對(duì)每一個(gè)物體賦值一個(gè)物理標(biāo)簽
主要的應(yīng)用
- 自動(dòng)駕駛
- 醫(yī)療圖像
DeepLab的主要思路
DeepLabV1與DeepLabV2
- 使用DCNN來分類并產(chǎn)生一個(gè)關(guān)于分割的一個(gè)粗糙的預(yù)測(平滑的,帶有模糊的heat map)
- 通過條件隨機(jī)場(CRF)重新調(diào)整結(jié)果
如圖所示:
DCNN
帶洞深度卷積
- Striding
- Pooling
CRF
此處主要涉及與DCNN的相關(guān)比較
- 首先DCNN將分類的準(zhǔn)確性與定位的準(zhǔn)確性進(jìn)行了相關(guān)權(quán)衡
- DCNN所生成的熱力圖可以很好的預(yù)測分類結(jié)果以及物體的大致位置
- 精確的大綱效率比較低
然而CRF將像素之間的關(guān)系進(jìn)行建模, 通過下面的方式:
- 相近的像素有更大的概率為同一個(gè)label
- CRF將對(duì)像素標(biāo)簽的賦值轉(zhuǎn)化為每一個(gè)像素的概率
- 通過迭代的方式來調(diào)整結(jié)果直至收斂
效果如下圖所示:
CRF算法步驟效果.png
DeepLabV1
網(wǎng)絡(luò)相關(guān)描述:
- DeepLab v1 is constructed by modifying VGG-16
- Fully connected layers of VGG-16 are converted to convolutional layers
- Subsampling is skipped after last two max-pooling layers
- Convolutional filters in the layers that follow pooling are modified to atrous
- Model weights of Imagenet-pretrained VGG-16 network are finetuned
網(wǎng)絡(luò)結(jié)構(gòu)圖:
DeepLabV1.png
圖像分割的結(jié)果:
DeepLabV1_result.png
DeepLabV2
網(wǎng)絡(luò)相關(guān)描述
- Better segmentation of objects at multiple scales (using ASPP)
- Adapting ResNet image classification DCNN
- Learning rate policy
ASPP
之所以引入ASPP(Atrous Spatial Pyramid Pooling)的原因如下:
- 圖片中物體的尺寸大小不一
- Computationally efficient scheme of resampling a given feature layer at multiple rates prior to convolution
- 使用多個(gè)并行的帶洞卷積層來進(jìn)行不同采樣率的卷積
ASPP的一些細(xì)節(jié)以及提升的結(jié)果圖:
ASPP_details.png
ASPP_result.png
網(wǎng)絡(luò)的輸出結(jié)果圖:
DeepLabV2_result.png
DeepLabV2 & DeepLabV3
優(yōu)點(diǎn)
- Speed: By virtue of the ‘a(chǎn)trous’ algorithm, dense DCNN operates at 8 fps, while fully-connected CRF requires 0.5 second
- Accuracy: state-of-the-art results achieved on several state-of-art datasets
- Simplicity: the system is composed of a cascade of two fairly wellestablished modules, DCNNs and CRFs
DeepLabV3
與前兩個(gè)版本的變動(dòng):
- The proposed framework is general and could be applied to any network
- Several copies of the last ResNet block are duplicated, and arranged in cascade
- Batch normalization is included within ASPP
- CRF is not used
關(guān)于ASPP的變動(dòng):
- Batch normalization is included within ASPP
- As the sampling rate becomes larger, number of valid filter weights becomes smaller
- Global average pooling on last feature map of the model
ASPP的更新的一些細(xì)節(jié):
ASPP_in_V3.png
DeepLabV3的最終效果:
DeepLabV3_result.png
DeepLabV3+
這里面使用一張圖來概括最大的更新:
DeepLabV3+.png
其較為詳細(xì)的結(jié)構(gòu)圖如下:
DeepLabV3+_pipline.png
另外Xception模型所作出的更改如圖所示:
Xception_modified.png
最終在PASCAL VOC 2012上的測試結(jié)果與其他方法比較結(jié)果下表:
Result_table.png
最后,放出此模型最終的一些可視化展示:
DeepLabV3+_result.png
[完]
