結(jié)合源碼分析EfficientNet

論文地址：https://arxiv.org/abs/1905.11946
pytorch代碼地址:https://github.com/lukemelas/EfficientNet-PyTorch

論文主要思想

該論文主要講述模型縮放對模型性能的影響。通過對網(wǎng)絡(luò)深度（層數(shù)）喷橙、寬度（通道數(shù)）以及輸入圖片的分辨率的調(diào)整來獲取更佳的網(wǎng)絡(luò)模型。最終獲得的模型相比原來的模型不僅參數(shù)量少中跌，而且準(zhǔn)確率高。論文主要是依靠下圖進行論述

各參數(shù)對模型準(zhǔn)確率與運算次數(shù)的影響

傳統(tǒng)的模型縮放方法一般只調(diào)整一個變量，例如：ResNet通過對網(wǎng)絡(luò)層數(shù)的縮放饲趋，獲得ResNet-18和ResNet-200幽邓。MobileNets通過對通道數(shù)的縮放來調(diào)整模型炮温。也有通過增大輸入圖片的大小來提取模型的準(zhǔn)確率。
本文方法為在現(xiàn)有baseline網(wǎng)絡(luò)下同時調(diào)整層數(shù)颊艳、通道數(shù)和分辨率三個變量來縮放模型茅特，以獲得更好的準(zhǔn)確率和運行速度。因此baseline網(wǎng)絡(luò)的模型設(shè)計也非常重要棋枕，文中的EfficientNet-B0模型為下圖:
EfficientNet-B0網(wǎng)絡(luò)模型

后續(xù)工作便為調(diào)整d,w,r三個參數(shù)以達到最佳狀態(tài)白修，對于EfficientNet-B0網(wǎng)絡(luò)，將φ 設(shè)1重斑，最后聯(lián)合搜索出的最佳參數(shù)為a=1.2兵睛，β=1.1，γ=1.15窥浪。調(diào)整的公式為
聯(lián)合調(diào)整3個參數(shù)的公式

EfficientNet-B0網(wǎng)絡(luò)模型細(xì)節(jié)（代碼）

整體模型中的特征提取過程主要由stem,blocks和head構(gòu)成祖很，最后將獲得的feature map用全連接層做類型判別。

EfficientNet(
  (_conv_stem): Conv2dStaticSamePadding(
    3, 32, kernel_size=(3, 3), stride=(2, 2), bias=False
    (static_padding): ZeroPad2d(padding=(0, 1, 0, 1), value=0.0)
  )
  (_bn0): BatchNorm2d(32, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
  (_blocks): ModuleList(
    (0): MBConvBlock(
      (_depthwise_conv): Conv2dStaticSamePadding(
        32, 32, kernel_size=(3, 3), stride=[1, 1], groups=32, bias=False
        (static_padding): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
      )
      (_bn1): BatchNorm2d(32, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        32, 8, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        8, 32, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(16, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (1): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        16, 96, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(96, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        96, 96, kernel_size=(3, 3), stride=[2, 2], groups=96, bias=False
        (static_padding): ZeroPad2d(padding=(0, 1, 0, 1), value=0.0)
      )
      (_bn1): BatchNorm2d(96, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        96, 4, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        4, 96, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        96, 24, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(24, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (2): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(144, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        144, 144, kernel_size=(3, 3), stride=(1, 1), groups=144, bias=False
        (static_padding): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
      )
      (_bn1): BatchNorm2d(144, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        144, 6, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        6, 144, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        144, 24, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(24, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (3): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(144, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        144, 144, kernel_size=(5, 5), stride=[2, 2], groups=144, bias=False
        (static_padding): ZeroPad2d(padding=(1, 2, 1, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(144, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        144, 6, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        6, 144, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        144, 40, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(40, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (4): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(240, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        240, 240, kernel_size=(5, 5), stride=(1, 1), groups=240, bias=False
        (static_padding): ZeroPad2d(padding=(2, 2, 2, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(240, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        240, 10, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        10, 240, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(40, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (5): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(240, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        240, 240, kernel_size=(3, 3), stride=[2, 2], groups=240, bias=False
        (static_padding): ZeroPad2d(padding=(0, 1, 0, 1), value=0.0)
      )
      (_bn1): BatchNorm2d(240, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        240, 10, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        10, 240, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        240, 80, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(80, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (6): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        80, 480, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(480, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        480, 480, kernel_size=(3, 3), stride=(1, 1), groups=480, bias=False
        (static_padding): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
      )
      (_bn1): BatchNorm2d(480, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        480, 20, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        20, 480, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        480, 80, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(80, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (7): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        80, 480, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(480, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        480, 480, kernel_size=(3, 3), stride=(1, 1), groups=480, bias=False
        (static_padding): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
      )
      (_bn1): BatchNorm2d(480, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        480, 20, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        20, 480, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        480, 80, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(80, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (8): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        80, 480, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(480, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        480, 480, kernel_size=(5, 5), stride=[1, 1], groups=480, bias=False
        (static_padding): ZeroPad2d(padding=(2, 2, 2, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(480, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        480, 20, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        20, 480, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        480, 112, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(112, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (9): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        112, 672, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(672, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        672, 672, kernel_size=(5, 5), stride=(1, 1), groups=672, bias=False
        (static_padding): ZeroPad2d(padding=(2, 2, 2, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(672, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        672, 28, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        28, 672, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        672, 112, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(112, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (10): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        112, 672, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(672, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        672, 672, kernel_size=(5, 5), stride=(1, 1), groups=672, bias=False
        (static_padding): ZeroPad2d(padding=(2, 2, 2, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(672, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        672, 28, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        28, 672, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        672, 112, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(112, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (11): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        112, 672, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(672, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        672, 672, kernel_size=(5, 5), stride=[2, 2], groups=672, bias=False
        (static_padding): ZeroPad2d(padding=(1, 2, 1, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(672, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        672, 28, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        28, 672, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        672, 192, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(192, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (12): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(1152, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        1152, 1152, kernel_size=(5, 5), stride=(1, 1), groups=1152, bias=False
        (static_padding): ZeroPad2d(padding=(2, 2, 2, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(1152, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        1152, 48, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        48, 1152, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        1152, 192, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(192, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (13): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(1152, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        1152, 1152, kernel_size=(5, 5), stride=(1, 1), groups=1152, bias=False
        (static_padding): ZeroPad2d(padding=(2, 2, 2, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(1152, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        1152, 48, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        48, 1152, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        1152, 192, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(192, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (14): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(1152, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        1152, 1152, kernel_size=(5, 5), stride=(1, 1), groups=1152, bias=False
        (static_padding): ZeroPad2d(padding=(2, 2, 2, 2), value=0.0)
      )
      (_bn1): BatchNorm2d(1152, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        1152, 48, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        48, 1152, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        1152, 192, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(192, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
    (15): MBConvBlock(
      (_expand_conv): Conv2dStaticSamePadding(
        192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn0): BatchNorm2d(1152, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_depthwise_conv): Conv2dStaticSamePadding(
        1152, 1152, kernel_size=(3, 3), stride=[1, 1], groups=1152, bias=False
        (static_padding): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
      )
      (_bn1): BatchNorm2d(1152, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        1152, 48, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        48, 1152, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        1152, 320, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(320, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )
  )
  (_conv_head): Conv2dStaticSamePadding(
    320, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False
    (static_padding): Identity()
  )
  (_bn1): BatchNorm2d(1280, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
  (_avg_pooling): AdaptiveAvgPool2d(output_size=1)
  (_dropout): Dropout(p=0.2, inplace=False)
  (_fc): Linear(in_features=1280, out_features=1000, bias=True)
  (_swish): MemoryEfficientSwish()
)

以224x224的一張彩色圖片為例漾脂，其維度為[1,3,224,224]假颇。
stem的前向傳播，其中swish在之后的運算中大量使用骨稿，命名字面意思說能夠有效利用內(nèi)存笨鸡，但還不能理解其中原理。

(_conv_stem): Conv2dStaticSamePadding(
    3, 32, kernel_size=(3, 3), stride=(2, 2), bias=False
    (static_padding): ZeroPad2d(padding=(0, 1, 0, 1), value=0.0)
)
(_bn0): BatchNorm2d(32, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
(_swish): MemoryEfficientSwish()

維度變化：[1,3,224,224]--pad-->[1,3,225,225]--conv-->[1,32,112,112]--bn0-->[1,32,112,112]--swish-->[1,32,112,112]
blocks的前向傳播坦冠，以其中第一個MBConvBlock為例：

  (0): MBConvBlock(
      (_depthwise_conv): Conv2dStaticSamePadding(
        32, 32, kernel_size=(3, 3), stride=[1, 1], groups=32, bias=False
        (static_padding): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
      )
      (_bn1): BatchNorm2d(32, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_se_reduce): Conv2dStaticSamePadding(
        32, 8, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_se_expand): Conv2dStaticSamePadding(
        8, 32, kernel_size=(1, 1), stride=(1, 1)
        (static_padding): Identity()
      )
      (_project_conv): Conv2dStaticSamePadding(
        32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False
        (static_padding): Identity()
      )
      (_bn2): BatchNorm2d(16, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
      (_swish): MemoryEfficientSwish()
    )

前向傳播中的維度變化

其中depthwise深度卷積的講解可以參考該篇博客http://www.reibang.com/p/38dc74d12fcf?utm_source=oschina-app
經(jīng)過blocks層的特征提取形耗，可以得到[1,320,7,7]的feature map。
head層設(shè)計

  (_conv_head): Conv2dStaticSamePadding(
    320, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False
    (static_padding): Identity()
  )
  (_bn1): BatchNorm2d(1280, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
  (_swish): MemoryEfficientSwish()

前向傳播維度變化：[1,320,7,7]--head-->[1,1280,7,7]--bn1-->[1,1280,7,7]--swish-->[1,1280,7,7]
池化層及最后的全連接層

  (_avg_pooling): AdaptiveAvgPool2d(output_size=1)
[1,1280]--dropout-->[1,1280]--fc-->[1,1000]
  (_dropout): Dropout(p=0.2, inplace=False)
  (_fc): Linear(in_features=1280, out_features=1000, bias=True)

經(jīng)過conv_head后的維度[1,1280,7,7]--avg_pool-->[1,1280,1,1]--view-->[1,1280]--dropout-->[1,1280]--fc-->[1,1000]辙浑，一張圖片的最終輸出為[1,1000]激涤，因為預(yù)測類別數(shù)目為1000。至此網(wǎng)絡(luò)的前向傳播過程已經(jīng)講完判呕。
網(wǎng)絡(luò)中存在dropout和drop connect兩種對于過擬合的解決方法倦踢，兩者的區(qū)別可以參考該博客https://www.cnblogs.com/tornadomeet/p/3430312.html
最后通過修改a送滞，β，γ及不同的φ 參數(shù)來縮放baseline網(wǎng)絡(luò)硼一，以獲得EfficientNet-B1到B7累澡。最后對于如何獲取最佳的參數(shù)，只有不停的去試般贼，但我們是否有大量的硬件設(shè)備和時間呢?這也是十分尷尬的情況愧哟。

參考博客：

depthwise的講解：http://www.reibang.com/p/38dc74d12fcf?utm_source=oschina-app
dropout和drop connect的區(qū)別：https://www.cnblogs.com/tornadomeet/p/3430312.html
對EfficieentNet的評價：https://www.zhihu.com/question/326833457/answer/700322601

最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
  • 人面猴
    序言：七十年代末，一起剝皮案震驚了整個濱河市哼蛆，隨后出現(xiàn)的幾起案子蕊梧，更是在濱河造成了極大的恐慌，老刑警劉巖腮介，帶你破解...
    沈念sama閱讀 221,635評論 6贊 515
  • 死咒
    序言：濱河連續(xù)發(fā)生了三起死亡事件肥矢，死亡現(xiàn)場離奇詭異，居然都是意外死亡叠洗，警方通過查閱死者的電腦和手機甘改，發(fā)現(xiàn)死者居然都...
    沈念sama閱讀 94,543評論 3贊 399
  • 救了他兩次的神仙讓他今天三更去死
    文/潘曉璐 我一進店門，熙熙樓的掌柜王于貴愁眉苦臉地迎上來灭抑，“玉大人十艾，你說我怎么就攤上這事√诮冢” “怎么了忘嫉？”我有些...
    開封第一講書人閱讀 168,083評論 0贊 360
  • 道士緝兇錄：失蹤的賣姜人
    文/不壞的土叔 我叫張陵，是天一觀的道長案腺。 經(jīng)常有香客問我庆冕，道長，這世上最難降的妖魔是什么劈榨？ 我笑而不...
    開封第一講書人閱讀 59,640評論 1贊 296
  • ?港島之戀（遺憾婚禮）
    正文 為了忘掉前任访递，我火速辦了婚禮，結(jié)果婚禮上同辣，老公的妹妹穿的比我還像新娘拷姿。我一直安慰自己，他們只是感情好邑闺，可當(dāng)我...
    茶點故事閱讀 68,640評論 6贊 397
  • 惡毒庶女頂嫁案：這布局不是一般人想出來的
    文/花漫 我一把揭開白布跌前。 她就那樣靜靜地躺著棕兼，像睡著了一般陡舅。 火紅的嫁衣襯著肌膚如雪。 梳的紋絲不亂的頭發(fā)上伴挚，一...
    開封第一講書人閱讀 52,262評論 1贊 308
  • 城市分裂傳說
    那天靶衍，我揣著相機與錄音灾炭，去河邊找鬼。 笑死颅眶，一個胖子當(dāng)著我的面吹牛蜈出，可吹牛的內(nèi)容都是我干的。 我是一名探鬼主播涛酗，決...
    沈念sama閱讀 40,833評論 3贊 421
  • 雙鴛鴦連環(huán)套：你想象不到人心有多黑
    文/蒼蘭香墨 我猛地睜開眼铡原，長吁一口氣：“原來是場噩夢啊……” “哼！你這毒婦竟也來了商叹？” 一聲冷哼從身側(cè)響起燕刻，我...
    開封第一講書人閱讀 39,736評論 0贊 276
  • 萬榮殺人案實錄
    序言：老撾萬榮一對情侶失蹤，失蹤者是張志新（化名）和其女友劉穎剖笙，沒想到半個月后卵洗，有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體，經(jīng)...
    沈念sama閱讀 46,280評論 1贊 319
  • ?護林員之死
    正文 獨居荒郊野嶺守林人離奇死亡弥咪，尸身上長有42處帶血的膿包…… 初始之章·張勛 以下內(nèi)容為張勛視角 年9月15日...
    茶點故事閱讀 38,369評論 3贊 340
  • ?白月光啟示錄
    正文 我和宋清朗相戀三年过蹂，在試婚紗的時候發(fā)現(xiàn)自己被綠了。 大學(xué)時的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片聚至。...
    茶點故事閱讀 40,503評論 1贊 352
  • 活死人
    序言：一個原本活蹦亂跳的男人離奇死亡酷勺，死狀恐怖，靈堂內(nèi)的尸體忽然破棺而出晚岭，到底是詐尸還是另有隱情鸥印，我是刑警寧澤，帶...
    沈念sama閱讀 36,185評論 5贊 350
  • ?日本核電站爆炸內(nèi)幕
    正文 年R本政府宣布坦报，位于F島的核電站库说，受9級特大地震影響，放射性物質(zhì)發(fā)生泄漏片择。R本人自食惡果不足惜潜的，卻給世界環(huán)境...
    茶點故事閱讀 41,870評論 3贊 333
  • 男人毒藥：我在死后第九天來索命
    文/蒙蒙 一、第九天 我趴在偏房一處隱蔽的房頂上張望字管。 院中可真熱鬧啰挪，春花似錦、人聲如沸嘲叔。這莊子的主人今日做“春日...
    開封第一講書人閱讀 32,340評論 0贊 24
  • 一樁弒父案，背后竟有這般陰謀
    文/蒼蘭香墨 我抬頭看了看天上的太陽硫戈。三九已至锰什，卻和暖如春，著一層夾襖步出監(jiān)牢的瞬間，已是汗流浹背汁胆。 一陣腳步聲響...
    開封第一講書人閱讀 33,460評論 1贊 272
  • 情欲美人皮
    我被黑心中介騙來泰國打工梭姓， 沒想到剛下飛機就差點兒被人妖公主榨干…… 1. 我叫王不留，地道東北人嫩码。 一個月前我還...
    沈念sama閱讀 48,909評論 3贊 376
  • 代替公主和親
    正文 我出身青樓誉尖，卻偏偏與公主長得像，于是被迫代替她去往敵國和親铸题。 傳聞我的和親對象是個殘疾皇子铡恕，可洞房花燭夜當(dāng)晚...
    茶點故事閱讀 45,512評論 2贊 359