數(shù)據(jù)集

GTSRB dataset :
http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset#Downloads

該數(shù)據(jù)集包含43類交通標(biāo)志洞焙，提供的資料中包含標(biāo)注信息拍柒。

【0】 數(shù)據(jù)準(zhǔn)備

• 根據(jù)標(biāo)注裁剪圖像
  在每類的文件夾中从藤，包含若干.ppm格式的圖片及一個(gè).csv文件，csv中包含每個(gè)ppm圖像的標(biāo)注信息第队，根據(jù)標(biāo)注信息進(jìn)行圖片裁剪.
  將43類放到同一文件夾Final_Training下，裁剪示例代碼：

import os
import sys
from PIL import Image
path = 'C:/Users/Documents/Dataset/GTSRB/Final_Training'

csv_files = []
for dirpath, dirnames, filenames in os.walk(path, topdown=False):
    for filename in filenames:
        if filename.endswith('.csv'):
            csv_files.append(os.path.join(dirpath, filename))


class TrafficSign:
    trafficSign_name = ''
    left_top_x = 0,
    left_top_y = 0,
    right_bottom_x = 0,
    right_bottom_y = 0,
    width = 0,
    height = 0,
    label = ''

    def tostring(self):
        print([self.trafficSign_name,
               self.width, self.height,
               self.left_top_x, self.left_top_y,
               self.right_bottom_x, self.right_bottom_y,
               self.label])


for csv in csv_files:
    base_path = os.path.dirname(csv)
    # read csv data
    trafficSigns = []
    with open(csv) as file:
        for line in file:
            if line.find('.ppm') == -1:
                continue
            raw_data = line.split(';')
            trafficSign = TrafficSign()
            trafficSign.trafficSign_name = raw_data[0]
            trafficSign.width = int(raw_data[1])
            trafficSign.height = int(raw_data[2])
            trafficSign.left_top_x = int(raw_data[3])
            trafficSign.left_top_y = int(raw_data[4])
            trafficSign.right_bottom_x = int(raw_data[5])
            trafficSign.right_bottom_y = int(raw_data[6])
            # trafficSign.label = raw_data[7]
            trafficSigns.append(trafficSign)

    # crop each image according to the csv in this folder
    for dirpath, dirnames, filenames in os.walk(base_path, topdown=False):
        for filename in filenames:
            if not filename.endswith('.ppm'):
                continue
            fullPath = os.path.join(dirpath, filename)
            for sign in trafficSigns:
                if filename == sign.trafficSign_name:
                    image = Image.open(fullPath)
                    # start cropping according to this sign
                    region = (sign.left_top_x, sign.left_top_y, sign.right_bottom_x, sign.right_bottom_y)
                    image_crop = image.crop(region)
                    # update the new image path
                    newFullPath = fullPath.replace('GTSRB', 'GTSRB_img_Crop')
                    newFullPath = newFullPath.replace('.ppm', '.bmp')
                    if not os.path.exists(os.path.dirname(newFullPath)):
                        os.makedirs(os.path.dirname(newFullPath))
                    # save the images
                    image_crop.save(newFullPath)
                    break

裁剪后的圖片如下所示：

image

• 劃分訓(xùn)練集和測(cè)試集
  觀察可以發(fā)現(xiàn)炫彩，交通標(biāo)志應(yīng)該是由遠(yuǎn)至近的序列中標(biāo)注裁剪出來(lái)的对途，所以會(huì)呈現(xiàn)由小到大的規(guī)律，所以在準(zhǔn)備訓(xùn)練集和測(cè)試集時(shí)啸蜜，隨機(jī)選擇一定比例的方式(我選擇80%訓(xùn)練坑雅，20%測(cè)試)，示例代碼：

import os
import random
import shutil
path = 'C:/Users/Documents/Dataset/GTSRB_img_Crop/Final_Training'
dirs = []
split_percentage = 0.2
for dirpath, dirnames, filenames in os.walk(path, topdown=False):
   for dirname in dirnames:
       fullpath = os.path.join(dirpath, dirname)
       fileCount = len([name for name in os.listdir(fullpath) if os.path.isfile(os.path.join(fullpath, name))])
       files = os.listdir(fullpath)
       for index in range((int)(split_percentage * fileCount)):
           newIndex = random.randint(0, fileCount - 1)
           fullFilePath = os.path.join(fullpath, files[newIndex])
           newFullFilePath = fullFilePath.replace('Final_Training', 'Final_Validation')
           base_new_path = os.path.dirname(newFullFilePath)
           if not os.path.exists(base_new_path):
               os.makedirs(base_new_path)
           # move the file
           try:
               shutil.move(fullFilePath, newFullFilePath)
           except IOError as error:
               print('skip moving from %s => %s' % (fullFilePath, newFullFilePath))

【1】訓(xùn)練和驗(yàn)證

結(jié)構(gòu)十分簡(jiǎn)單衬横，四個(gè)卷積層加上全連接層輸出即可裹粤。其中的個(gè)別的超參數(shù)選擇，我是參照了GTSRB比賽中成績(jī)最好的那篇文章中提到的一些配置：

CNN with 3 Spatial Transformers, DeepKnowledge Seville, álvaro Arcos-García and Juan A. álvarez-García and Luis M. Soria-Morillo, Neural Networks
link

在這篇文章中冕香，提到使用48*48的歸一化尺寸以及一些其他的建議蛹尝，可以詳細(xì)參閱。如下示例代碼簡(jiǎn)單跑一下（）：

import shutil
import os
import matplotlib.pyplot as plt

train_set_base_dir = 'C:/Users/Documents/Dataset/GTSRB_img_Crop/Final_Training'
validation_set_base_dir = 'C:/Users/Documents/Dataset/GTSRB_img_Crop/Final_Validation'

# start image preprocess
from keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(
    rescale=1. / 255
)
train_data_generator = train_datagen.flow_from_directory(
    directory=train_set_base_dir,
    target_size=(48, 48),
    batch_size=32,
    class_mode='categorical')


validation_datagen = ImageDataGenerator(
    rescale=1. /255
)

validation_data_generator = validation_datagen.flow_from_directory(
    directory=validation_set_base_dir,
    target_size=(48, 48),
    batch_size=32,
    class_mode='categorical'
)

# define a simple CNN network
from keras.models import Sequential
from keras.layers import Conv2D, MaxPool2D, Flatten, Dense, Dropout

model = Sequential()

# add Con2D layers
model.add(Conv2D(filters=32, kernel_size=(3, 3), activation='relu', input_shape=(48, 48, 3)))
model.add(MaxPool2D(pool_size=(2, 2), padding='valid'))

model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2), padding='valid'))

model.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2), padding='valid'))

model.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2), padding='valid'))

# flatten
model.add(Flatten())

# dropOut layer
model.add(Dropout(0.2))

# add one simple layer for classification
model.add(Dense(units=512, activation='relu'))

# add output layer
model.add(Dense(units=43, activation='softmax'))

# compile model
model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['acc'])

# print model info
model.summary()
json_str = model.to_json()
print(json_str)
# fit_generator to fill in the dataset
history = model.fit_generator(
    generator=train_data_generator,
    steps_per_epoch=100,
    epochs=30,
    validation_data=validation_data_generator,
    validation_steps=50)

# train done, save the models
model.save('C:/test/WorkingLogs/20181214/traffic_signs.h5')

# plot the roc curve
acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1, len(acc) + 1)
plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()

簡(jiǎn)易的網(wǎng)絡(luò)結(jié)構(gòu)及參與訓(xùn)練測(cè)試的樣本信息如下：

Found 32117 images belonging to 43 classes.
Found 7092 images belonging to 43 classes.
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 46, 46, 32)        896       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 23, 23, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 21, 21, 64)        18496     
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 10, 10, 64)        0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 8, 8, 128)         73856     
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 4, 4, 128)         0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 2, 2, 128)         147584    
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 1, 1, 128)         0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 128)               0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 512)               66048     
_________________________________________________________________
dense_2 (Dense)              (None, 43)                22059     
=================================================================
Total params: 328,939
Trainable params: 328,939
Non-trainable params: 0
_________________________________________________________________

30 epochs的結(jié)果是：

100/100 [==============================] - 164s 2s/step - loss: 0.2009 - acc: 0.9556 - val_loss: 0.1103 - val_acc: 0.9755

image

image

【3】結(jié)論

從結(jié)果可以看出悉尾，即使是簡(jiǎn)單的網(wǎng)絡(luò)結(jié)構(gòu)突那，在精確標(biāo)注的大量數(shù)據(jù)下可以獲得很好的效果。同時(shí)還可以通過(guò)pre-trained模型如VGG-16等提取特征再加入某些層進(jìn)行fine-tuned等构眯。
在上面推薦的那片論文中愕难，還提出使用spatial-transformer層進(jìn)行優(yōu)化，也很值得嘗試。