2018年10月13日筆記

tensorflow是谷歌google的深度學(xué)習(xí)框架，tensor中文叫做張量厉颤，flow叫做流。
CNN是convolutional neural network的簡稱，中文叫做卷積神經(jīng)網(wǎng)絡(luò)飒焦。
文本分類是NLP（自然語言處理）的經(jīng)典任務(wù)那槽。

0.編程環(huán)境

操作系統(tǒng)：Win10
python版本：3.6
集成開發(fā)環(huán)境：jupyter notebook
tensorflow版本：1.6

1.致謝聲明

本文是作者學(xué)習(xí)《使用卷積神經(jīng)網(wǎng)絡(luò)以及循環(huán)神經(jīng)網(wǎng)絡(luò)進(jìn)行中文文本分類》的成果悼沿，感激前輩；
github鏈接：https://github.com/gaussic/text-classification-cnn-rnn

2.配置環(huán)境

使用卷積神經(jīng)網(wǎng)絡(luò)模型要求有較高的機(jī)器配置骚灸，如果使用CPU版tensorflow會花費(fèi)大量時間糟趾。
讀者在有nvidia顯卡的情況下，安裝GPU版tensorflow會提高計算速度50倍。
安裝教程鏈接：https://blog.csdn.net/qq_36556893/article/details/79433298
如果沒有nvidia顯卡义郑，但有visa信用卡蝶柿，請閱讀我的另一篇文章《在谷歌云服務(wù)器上搭建深度學(xué)習(xí)平臺》，鏈接：http://www.reibang.com/p/893d622d1b5a

3.下載并解壓數(shù)據(jù)集

數(shù)據(jù)集下載鏈接: https://pan.baidu.com/s/1cfHHr5aLk76AE7qi7L122g 提取碼: tvgn
壓縮文件夾基于CNN的搜狐新聞文本分類中有8個文件非驮，如下圖所示：

image.png

4.完整代碼

完整代碼已經(jīng)在數(shù)據(jù)集文件中給出施逾，即cnn_package.ipynb文件和getTrainTestDataSet.ipynb文件；
getTrainTestDataSet.ipynb文件中代碼的作用是將文本文件轉(zhuǎn)換為二進(jìn)制文件例获，即4個pickle文件汉额；
從工程開發(fā)的角度考慮，本文作者在cnn_package.ipynb文件中封裝了一個類TextClassification榨汤，對于樣本數(shù)量在10萬左右的分類任務(wù)較為適用蠕搜。
后面章節(jié)中將講解實(shí)現(xiàn)細(xì)節(jié)。

5.獲取數(shù)據(jù)

第1行代碼導(dǎo)入庫pandas收壕，起別名pd妓灌；
第2行代碼導(dǎo)入庫pickle；
第4-5行代碼分別讀取訓(xùn)練集文件sohu_train.txt和測試集文件sohu_test.txt蜜宪。
第6行代碼將獲取訓(xùn)練集內(nèi)容列表train_content_list虫埂；
第7行代碼將獲取訓(xùn)練集標(biāo)簽列表train_label_list；
第8行代碼將獲取測試集內(nèi)容列表test_content_list圃验；
第9行代碼將獲取測試集標(biāo)簽列表test_label_list掉伏。
第10-17行代碼調(diào)用pickle庫的dump方法將python的對象轉(zhuǎn)換為二進(jìn)制文件。

import pandas as pd
import pickle

train_df = pd.read_csv('sohu_train.txt', sep='\t', header=None)
test_df = pd.read_csv('sohu_test.txt', sep='\t', header=None)
train_content_list = [k for k in train_df[1]]
train_label_list = [k for k in train_df[0]]
test_content_list = [k for k in test_df[1]]
test_label_list = [k for k in test_df[0]]
with open('train_content_list.pickle', 'wb') as file:
    pickle.dump(train_content_list, file)
with open('train_label_list.pickle', 'wb') as file:
    pickle.dump(train_label_list, file)
with open('test_content_list.pickle', 'wb') as file:
    pickle.dump(test_content_list, file)
with open('test_label_list.pickle', 'wb') as file:
    pickle.dump(test_label_list, file)

6.編寫類TextClassification

認(rèn)為編寫類有困難澳窑，可以先閱讀本文作者的另外一篇文章《基于tensorflow+CNN的新浪新聞文本分類》斧散，鏈接：http://www.reibang.com/p/b1000d5345bb，這一篇文章的代碼沒有進(jìn)行封裝摊聋，而且講解較為詳細(xì)鸡捐。
讀者閱讀下文中的行數(shù)時，可以先把代碼復(fù)制到j(luò)upyter notebook的代碼塊中麻裁。
在代碼塊中按Esc鍵箍镜，進(jìn)入命令模式瞻鹏，代碼塊左邊的豎線會顯示藍(lán)色，如下圖所示：

image.png

from sklearn.model_selection import train_test_split
import pickle
from collections import Counter
import tensorflow.contrib.keras as kr
from sklearn.preprocessing import LabelEncoder
import tensorflow as tf
import random
import numpy as np
import pandas as pd
from sklearn.metrics import confusion_matrix
from sklearn.metrics import precision_recall_fscore_support
import warnings
warnings.filterwarnings('ignore')
import time

class TextConfig():
    vocab_size = 5000
    seq_length = 600
    embedding_dim = 64  # 詞向量維度
    num_filters = 256  # 卷積核數(shù)目
    kernel_size = 5  # 卷積核尺
    hidden_dim = 128  # 全連接層神經(jīng)元
    dropout_keep_prob = 0.5  # dropout保留比例
    learning_rate = 1e-3  # 學(xué)習(xí)率
    batch_size = 32  # 每批訓(xùn)練大小
    num_iteration = 5000 #迭代次數(shù)
    print_per_batch = num_iteration / 20 #打印間隔

class TextClassification():
    def config(self):
        textConfig = TextConfig()
        self.vocab_size = textConfig.vocab_size
        self.seq_length = textConfig.seq_length
        self.embedding_dim = textConfig.embedding_dim
        self.num_filters = textConfig.num_filters
        self.kernel_size = textConfig.kernel_size
        self.hidden_dim = textConfig.hidden_dim
        self.dropout_keep_prob = textConfig.dropout_keep_prob
        self.learning_rate = textConfig.learning_rate
        self.batch_size = textConfig.batch_size
        self.print_per_batch = textConfig.print_per_batch
        self.num_iteration = textConfig.num_iteration
    
    def __init__(self, *args):
        self.config()
        if len(args) == 2:
            content_list = args[0]
            label_list = args[1]
            train_X, test_X, train_y, test_y = train_test_split(content_list, label_list)
            self.train_content_list = train_X
            self.train_label_list = train_y
            self.test_content_list = test_X
            self.test_label_list = test_y
            self.content_list = self.train_content_list + self.test_content_list
        elif len(args) == 4:
            self.train_content_list = args[0]
            self.train_label_list = args[1]
            self.test_content_list = args[2]
            self.test_label_list = args[3]
            self.content_list = self.train_content_list + self.test_content_list
        else:
            print('false to init TextClassification object')
        self.autoGetNumClasses()
    
    def autoGetNumClasses(self):
        label_list = self.train_label_list + self.test_label_list
        self.num_classes = np.unique(label_list).shape[0]
    
    def getVocabularyList(self, content_list, vocabulary_size):
        allContent_str = ''.join(content_list)
        counter = Counter(allContent_str)
        vocabulary_list = [k[0] for k in counter.most_common(vocabulary_size)]
        return ['PAD'] + vocabulary_list

    def prepareData(self):
        vocabulary_list = self.getVocabularyList(self.content_list, self.vocab_size)
        if len(vocabulary_list) < self.vocab_size:
            self.vocab_size = len(vocabulary_list)
        contentLength_list = [len(k) for k in self.train_content_list]
        if max(contentLength_list) < self.seq_length:
            self.seq_length = max(contentLength_list)
        self.word2id_dict = dict([(b, a) for a, b in enumerate(vocabulary_list)])
        self.labelEncoder = LabelEncoder()
        self.labelEncoder.fit(self.train_label_list)

    def content2idList(self, content):
        return [self.word2id_dict[word] for word in content if word in self.word2id_dict]

    def content2X(self, content_list):
        idlist_list = [self.content2idList(content) for content in content_list]
        X = kr.preprocessing.sequence.pad_sequences(idlist_list, self.seq_length)
        return X

    def label2Y(self, label_list):
        y = self.labelEncoder.transform(label_list)
        Y = kr.utils.to_categorical(y, self.num_classes)
        return Y

    def buildModel(self):
        tf.reset_default_graph()
        self.X_holder = tf.placeholder(tf.int32, [None, self.seq_length])
        self.Y_holder = tf.placeholder(tf.float32, [None, self.num_classes])
        embedding = tf.get_variable('embedding', [self.vocab_size, self.embedding_dim])
        embedding_inputs = tf.nn.embedding_lookup(embedding, self.X_holder)
        conv = tf.layers.conv1d(embedding_inputs, self.num_filters, self.kernel_size)
        max_pooling = tf.reduce_max(conv, reduction_indices=[1])
        full_connect = tf.layers.dense(max_pooling, self.hidden_dim)
        full_connect_dropout = tf.contrib.layers.dropout(full_connect, keep_prob=self.dropout_keep_prob)
        full_connect_activate = tf.nn.relu(full_connect_dropout)
        softmax_before = tf.layers.dense(full_connect_activate, self.num_classes)
        self.predict_Y = tf.nn.softmax(softmax_before)
        cross_entropy = tf.nn.softmax_cross_entropy_with_logits_v2(labels=self.Y_holder, logits=softmax_before)
        self.loss = tf.reduce_mean(cross_entropy)
        optimizer = tf.train.AdamOptimizer(self.learning_rate)
        self.train = optimizer.minimize(self.loss)
        self.predict_y = tf.argmax(self.predict_Y, 1)
        isCorrect = tf.equal(tf.argmax(self.Y_holder, 1), self.predict_y)
        self.accuracy = tf.reduce_mean(tf.cast(isCorrect, tf.float32))

    def trainModel(self):
        self.prepareData()
        self.buildModel()
        init = tf.global_variables_initializer()
        self.session = tf.Session()
        self.session.run(init)
        train_X = self.content2X(self.train_content_list)
        train_Y = self.label2Y(self.train_label_list)
        test_X = self.content2X(self.test_content_list)
        test_Y = self.label2Y(self.test_label_list)
        startTime = time.time()
        for i in range(self.num_iteration):
            selected_index = random.sample(list(range(len(train_Y))), k=self.batch_size)
            batch_X = train_X[selected_index]
            batch_Y = train_Y[selected_index]
            self.session.run(self.train, {self.X_holder: batch_X, self.Y_holder: batch_Y})
            step = i + 1
            if step % self.print_per_batch == 0 or step == 1:
                selected_index = random.sample(list(range(len(test_Y))), k=200)
                batch_X = test_X[selected_index]
                batch_Y = test_Y[selected_index]
                loss_value, accuracy_value = self.session.run([self.loss, self.accuracy],\
                    {self.X_holder: batch_X, self.Y_holder: batch_Y})
                used_time = time.time() - startTime
                print('step:%d loss:%.4f accuracy:%.4f used time:%.2f seconds' %
                      (step, loss_value, accuracy_value, used_time))

    def predict(self, content_list):
        if type(content_list) == str:
            content_list = [content_list]
        batch_X = self.content2X(content_list)
        predict_y = self.session.run(self.predict_y, {self.X_holder:batch_X})
        predict_label_list = self.labelEncoder.inverse_transform(predict_y)
        return predict_label_list

    def predictAll(self):
        predict_label_list = []
        batch_size = 100
        for i in range(0, len(self.test_content_list), batch_size):
            content_list = self.test_content_list[i: i + batch_size]
            predict_label = self.predict(content_list)
            predict_label_list.extend(predict_label)
        return predict_label_list

    def printConfusionMatrix(self):
        predict_label_list = self.predictAll()
        df = pd.DataFrame(confusion_matrix(self.test_label_list, predict_label_list),
                     columns=self.labelEncoder.classes_,
                     index=self.labelEncoder.classes_)
        print('\n Confusion Matrix:')
        print(df)

    def printReportTable(self):
        predict_label_list = self.predictAll()
        reportTable = self.eval_model(self.test_label_list,
                                 predict_label_list,
                                 self.labelEncoder.classes_)
        print('\n Report Table:')
        print(reportTable)
        
    def eval_model(self, y_true, y_pred, labels):
        # 計算每個分類的Precision, Recall, f1, support
        p, r, f1, s = precision_recall_fscore_support(y_true, y_pred)
        # 計算總體的平均Precision, Recall, f1, support
        tot_p = np.average(p, weights=s)
        tot_r = np.average(r, weights=s)
        tot_f1 = np.average(f1, weights=s)
        tot_s = np.sum(s)
        res1 = pd.DataFrame({
            u'Label': labels,
            u'Precision': p,
            u'Recall': r,
            u'F1': f1,
            u'Support': s
        })
        res2 = pd.DataFrame({
            u'Label': ['總體'],
            u'Precision': [tot_p],
            u'Recall': [tot_r],
            u'F1': [tot_f1],
            u'Support': [tot_s]
        })
        res2.index = [999]
        res = pd.concat([res1, res2])
        return res[['Label', 'Precision', 'Recall', 'F1', 'Support']]

7.調(diào)用類中方法

第1-10行代碼調(diào)用pickle庫的load方法讀取pickle文件中的數(shù)據(jù)砌溺；
第11-14行代碼實(shí)例化TextClassification對象；
第15行代碼調(diào)用模型對象的trainModel方法变隔，即做模型訓(xùn)練规伐；
第16行代碼調(diào)用模型對象的printConfusionMatrix方法，即打印混淆矩陣匣缘；
第17行代碼調(diào)用模型對象的printReportTable方法猖闪，即打印報告表鲜棠；

import pickle

with open('train_content_list.pickle', 'rb') as file:
    train_content_list = pickle.load(file)
with open('train_label_list.pickle', 'rb') as file:
    train_label_list = pickle.load(file)
with open('test_content_list.pickle', 'rb') as file:
    test_content_list = pickle.load(file)
with open('test_label_list.pickle', 'rb') as file:
    test_label_list = pickle.load(file)
model = TextClassification(train_content_list,
                           train_label_list,
                           test_content_list,
                           test_label_list)
model.trainModel()
model.printConfusionMatrix()
model.printReportTable()

上面一段代碼的運(yùn)行結(jié)果如下圖所示，警告部分不影響程序運(yùn)行：

image.png

image.png

7.總結(jié)

1.本文是作者第7個NLP小型項(xiàng)目培慌，數(shù)據(jù)共有36000條豁陆。
2.分類模型的評估指標(biāo)F1score為0.95左右，總體來說這個分類模型很優(yōu)秀吵护，能夠投入實(shí)際應(yīng)用盒音。
3.本文進(jìn)行了類的封裝，小型中文文本分類項(xiàng)目經(jīng)過數(shù)據(jù)處理得到內(nèi)容列表content_list和標(biāo)簽列表label_list之后何址，即可直接使用類做模型訓(xùn)練和預(yù)測里逆，并且得到詳細(xì)的預(yù)測結(jié)果報告表。