Approach
- Filter selection
The key idea is: if we can use a subset of channels in layer (i + 1)’s input to approximate the output in layer i + 1, the other channels can be safely removed from the input of layer i + 1, the other channels can be safely removed from the input of layer i + 1.
Here, |S| is the number of elements in a subset S, and r is a pre-defined compression rate. Now, given a set of m (the product of number of images and number of locations) training examples (x_i; y_i), the greedy algorithm is as follow.
- Pruning
Weak channels in layer (i + 1)’s input and their corresponding filters in layer i would be pruned away, leading to a much smaller model. - Fine-tuning
Fine-tuning is a necessary step to recover the generalization ability damaged by filter pruning. Then iterate to step 1 to prune the next layer.
Experiment
ImageNet
References:
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression, Jian-Hao Luo, 2017, ICCV
