2018年10月12日筆記

tensorflow是谷歌google的深度學(xué)習(xí)框架如绸，tensor中文叫做張量宦言，flow叫做流门驾。
CNN是convolutional neural network的簡(jiǎn)稱娃豹，中文叫做卷積神經(jīng)網(wǎng)絡(luò)憔涉。
文本分類是NLP（自然語(yǔ)言處理）的經(jīng)典任務(wù)。

0.編程環(huán)境

操作系統(tǒng)：Win10
python版本：3.6
集成開(kāi)發(fā)環(huán)境：jupyter notebook
tensorflow版本：1.6

1.致謝聲明

本文是作者學(xué)習(xí)《使用卷積神經(jīng)網(wǎng)絡(luò)以及循環(huán)神經(jīng)網(wǎng)絡(luò)進(jìn)行中文文本分類》的成果箭跳，感激前輩晨另；
github鏈接：https://github.com/gaussic/text-classification-cnn-rnn

2.配置環(huán)境

使用卷積神經(jīng)網(wǎng)絡(luò)模型要求有較高的機(jī)器配置，如果使用CPU版tensorflow會(huì)花費(fèi)大量時(shí)間谱姓。
讀者在有nvidia顯卡的情況下借尿，安裝GPU版tensorflow會(huì)提高計(jì)算速度50倍。
安裝教程鏈接：https://blog.csdn.net/qq_36556893/article/details/79433298
如果沒(méi)有nvidia顯卡屉来，但有visa信用卡路翻，請(qǐng)閱讀我的另一篇文章《在谷歌云服務(wù)器上搭建深度學(xué)習(xí)平臺(tái)》，鏈接：http://www.reibang.com/p/893d622d1b5a

3.下載并解壓數(shù)據(jù)集

數(shù)據(jù)集下載鏈接: https://pan.baidu.com/s/10QtokJ8_tkK6I3GifalxWg 提取碼: uytb
壓縮文件CNN垃圾郵件分類中有3個(gè)文件茄靠，如下圖所示：

image.png

4.完整代碼

完整代碼已經(jīng)在數(shù)據(jù)集文件中給出，即cnn_package.ipynb文件获询；
從工程開(kāi)發(fā)的角度考慮涨岁，本文作者封裝了一個(gè)類TextClassification，對(duì)于樣本數(shù)量在10萬(wàn)左右的分類任務(wù)較為適用吉嚣。
后面章節(jié)中將講解實(shí)現(xiàn)細(xì)節(jié)梢薪。

5.編寫類TextClassification

認(rèn)為編寫類有困難，可以先閱讀本文作者的另外一篇文章《基于tensorflow+CNN的新浪新聞文本分類》尝哆，鏈接：http://www.reibang.com/p/b1000d5345bb秉撇，這一篇文章的代碼沒(méi)有進(jìn)行封裝，而且講解較為詳細(xì)秋泄。
讀者閱讀下文中的行數(shù)時(shí)琐馆，可以先把代碼復(fù)制到j(luò)upyter notebook的代碼塊中。
在代碼塊中按Esc鍵恒序，進(jìn)入命令模式瘦麸，代碼塊左邊的豎線會(huì)顯示藍(lán)色，如下圖所示：

image.png

from sklearn.model_selection import train_test_split
import pickle
from collections import Counter
import tensorflow.contrib.keras as kr
from sklearn.preprocessing import LabelEncoder
import tensorflow as tf
import random
import numpy as np
import pandas as pd
from sklearn.metrics import confusion_matrix
from sklearn.metrics import precision_recall_fscore_support
import warnings
warnings.filterwarnings('ignore')
import time

class TextConfig():
    vocab_size = 5000
    seq_length = 600
    embedding_dim = 64  # 詞向量維度
    num_filters = 256  # 卷積核數(shù)目
    kernel_size = 5  # 卷積核尺
    hidden_dim = 128  # 全連接層神經(jīng)元
    dropout_keep_prob = 0.5  # dropout保留比例
    learning_rate = 1e-3  # 學(xué)習(xí)率
    batch_size = 32  # 每批訓(xùn)練大小
    num_iteration = 5000 #迭代次數(shù)
    print_per_batch = num_iteration / 20 #打印間隔

class TextClassification():
    def config(self):
        textConfig = TextConfig()
        self.vocab_size = textConfig.vocab_size
        self.seq_length = textConfig.seq_length
        self.embedding_dim = textConfig.embedding_dim
        self.num_filters = textConfig.num_filters
        self.kernel_size = textConfig.kernel_size
        self.hidden_dim = textConfig.hidden_dim
        self.dropout_keep_prob = textConfig.dropout_keep_prob
        self.learning_rate = textConfig.learning_rate
        self.batch_size = textConfig.batch_size
        self.print_per_batch = textConfig.print_per_batch
        self.num_iteration = textConfig.num_iteration
    
    def __init__(self, *args):
        self.config()
        if len(args) == 2:
            content_list = args[0]
            label_list = args[1]
            train_X, test_X, train_y, test_y = train_test_split(content_list, label_list)
            self.train_content_list = train_X
            self.train_label_list = train_y
            self.test_content_list = test_X
            self.test_label_list = test_y
            self.content_list = self.train_content_list + self.test_content_list
        elif len(args) == 4:
            self.train_content_list = args[0]
            self.train_label_list = args[1]
            self.test_content_list = args[2]
            self.test_label_list = args[3]
            self.content_list = self.train_content_list + self.test_content_list
        else:
            print('false to init TextClassification object')
        self.autoGetNumClasses()
    
    def autoGetNumClasses(self):
        label_list = self.train_label_list + self.test_label_list
        self.num_classes = np.unique(label_list).shape[0]
    
    def getVocabularyList(self, content_list, vocabulary_size):
        allContent_str = ''.join(content_list)
        counter = Counter(allContent_str)
        vocabulary_list = [k[0] for k in counter.most_common(vocabulary_size)]
        return ['PAD'] + vocabulary_list

    def prepareData(self):
        vocabulary_list = self.getVocabularyList(self.content_list, self.vocab_size)
        if len(vocabulary_list) < self.vocab_size:
            self.vocab_size = len(vocabulary_list)
        contentLength_list = [len(k) for k in self.train_content_list]
        if max(contentLength_list) < self.seq_length:
            self.seq_length = max(contentLength_list)
        self.word2id_dict = dict([(b, a) for a, b in enumerate(vocabulary_list)])
        self.labelEncoder = LabelEncoder()
        self.labelEncoder.fit(self.train_label_list)

    def content2idList(self, content):
        return [self.word2id_dict[word] for word in content if word in self.word2id_dict]

    def content2X(self, content_list):
        idlist_list = [self.content2idList(content) for content in content_list]
        X = kr.preprocessing.sequence.pad_sequences(idlist_list, self.seq_length)
        return X

    def label2Y(self, label_list):
        y = self.labelEncoder.transform(label_list)
        Y = kr.utils.to_categorical(y, self.num_classes)
        return Y

    def buildModel(self):
        tf.reset_default_graph()
        self.X_holder = tf.placeholder(tf.int32, [None, self.seq_length])
        self.Y_holder = tf.placeholder(tf.float32, [None, self.num_classes])
        embedding = tf.get_variable('embedding', [self.vocab_size, self.embedding_dim])
        embedding_inputs = tf.nn.embedding_lookup(embedding, self.X_holder)
        conv = tf.layers.conv1d(embedding_inputs, self.num_filters, self.kernel_size)
        max_pooling = tf.reduce_max(conv, reduction_indices=[1])
        full_connect = tf.layers.dense(max_pooling, self.hidden_dim)
        full_connect_dropout = tf.contrib.layers.dropout(full_connect, keep_prob=self.dropout_keep_prob)
        full_connect_activate = tf.nn.relu(full_connect_dropout)
        softmax_before = tf.layers.dense(full_connect_activate, self.num_classes)
        self.predict_Y = tf.nn.softmax(softmax_before)
        cross_entropy = tf.nn.softmax_cross_entropy_with_logits_v2(labels=self.Y_holder, logits=softmax_before)
        self.loss = tf.reduce_mean(cross_entropy)
        optimizer = tf.train.AdamOptimizer(self.learning_rate)
        self.train = optimizer.minimize(self.loss)
        self.predict_y = tf.argmax(self.predict_Y, 1)
        isCorrect = tf.equal(tf.argmax(self.Y_holder, 1), self.predict_y)
        self.accuracy = tf.reduce_mean(tf.cast(isCorrect, tf.float32))

    def trainModel(self):
        self.prepareData()
        self.buildModel()
        init = tf.global_variables_initializer()
        self.session = tf.Session()
        self.session.run(init)
        train_X = self.content2X(self.train_content_list)
        train_Y = self.label2Y(self.train_label_list)
        test_X = self.content2X(self.test_content_list)
        test_Y = self.label2Y(self.test_label_list)
        startTime = time.time()
        for i in range(self.num_iteration):
            selected_index = random.sample(list(range(len(train_Y))), k=self.batch_size)
            batch_X = train_X[selected_index]
            batch_Y = train_Y[selected_index]
            self.session.run(self.train, {self.X_holder: batch_X, self.Y_holder: batch_Y})
            step = i + 1
            if step % self.print_per_batch == 0 or step == 1:
                selected_index = random.sample(list(range(len(test_Y))), k=200)
                batch_X = test_X[selected_index]
                batch_Y = test_Y[selected_index]
                loss_value, accuracy_value = self.session.run([self.loss, self.accuracy],\
                    {self.X_holder: batch_X, self.Y_holder: batch_Y})
                used_time = time.time() - startTime
                print('step:%d loss:%.4f accuracy:%.4f used time:%.2f seconds' %
                      (step, loss_value, accuracy_value, used_time))

    def predict(self, content_list):
        if type(content_list) == str:
            content_list = [content_list]
        batch_X = self.content2X(content_list)
        predict_y = self.session.run(self.predict_y, {self.X_holder:batch_X})
        predict_label_list = self.labelEncoder.inverse_transform(predict_y)
        return predict_label_list

    def predictAll(self):
        predict_label_list = []
        batch_size = 100
        for i in range(0, len(self.test_content_list), batch_size):
            content_list = self.test_content_list[i: i + batch_size]
            predict_label = self.predict(content_list)
            predict_label_list.extend(predict_label)
        return predict_label_list

    def printConfusionMatrix(self):
        predict_label_list = self.predictAll()
        df = pd.DataFrame(confusion_matrix(self.test_label_list, predict_label_list),
                     columns=self.labelEncoder.classes_,
                     index=self.labelEncoder.classes_)
        print('\n Confusion Matrix:')
        print(df)

    def printReportTable(self):
        predict_label_list = self.predictAll()
        reportTable = self.eval_model(self.test_label_list,
                                 predict_label_list,
                                 self.labelEncoder.classes_)
        print('\n Report Table:')
        print(reportTable)
        
    def eval_model(self, y_true, y_pred, labels):
        # 計(jì)算每個(gè)分類的Precision, Recall, f1, support
        p, r, f1, s = precision_recall_fscore_support(y_true, y_pred)
        # 計(jì)算總體的平均Precision, Recall, f1, support
        tot_p = np.average(p, weights=s)
        tot_r = np.average(r, weights=s)
        tot_f1 = np.average(f1, weights=s)
        tot_s = np.sum(s)
        res1 = pd.DataFrame({
            u'Label': labels,
            u'Precision': p,
            u'Recall': r,
            u'F1': f1,
            u'Support': s
        })
        res2 = pd.DataFrame({
            u'Label': ['總體'],
            u'Precision': [tot_p],
            u'Recall': [tot_r],
            u'F1': [tot_f1],
            u'Support': [tot_s]
        })
        res2.index = [999]
        res = pd.concat([res1, res2])
        return res[['Label', 'Precision', 'Recall', 'F1', 'Support']]

6.調(diào)用類中方法

第1-6行代碼調(diào)用pickle庫(kù)的load方法讀取pickle文件中的數(shù)據(jù)跃赚；
第7行代碼實(shí)例化TextClassification對(duì)象笆搓；
第8行代碼調(diào)用模型對(duì)象的trainModel方法，即做模型訓(xùn)練纬傲；
第9行代碼調(diào)用模型對(duì)象的printConfusionMatrix方法满败，即打印混淆矩陣；
第10行代碼調(diào)用模型對(duì)象的printReportTable方法叹括，即打印報(bào)告表算墨；

import pickle

with open('mailContent_list.pickle', 'rb') as file:
    content_list = pickle.load(file)
with open('mailLabel_list.pickle', 'rb') as file:
    label_list = pickle.load(file)
model = TextClassification(content_list, label_list)
model.trainModel()
model.printConfusionMatrix()
model.printReportTable()

上面一段代碼的運(yùn)行結(jié)果如下圖所示，警告部分不影響程序運(yùn)行：

image.png

7.總結(jié)

1.本文是作者第6個(gè)NLP小型項(xiàng)目汁雷，數(shù)據(jù)共有60000多條净嘀。
2.分類模型的評(píng)估指標(biāo)F1score為0.994左右，總體來(lái)說(shuō)這個(gè)分類模型很優(yōu)秀侠讯，能夠投入實(shí)際應(yīng)用挖藏。
3.本文進(jìn)行了類的封裝，小型中文文本分類項(xiàng)目經(jīng)過(guò)數(shù)據(jù)處理得到內(nèi)容列表content_list和標(biāo)簽列表label_list之后厢漩，即可直接使用類做模型訓(xùn)練和預(yù)測(cè)膜眠，并且得到詳細(xì)的預(yù)測(cè)結(jié)果報(bào)告表。