基于keras 實現(xiàn)
利用之前訓(xùn)練好的詞向量资溃，基于keras使用1D卷積神經(jīng)網(wǎng)絡(luò)完成文本分類任務(wù)。
python gensim 訓(xùn)練詞向量
準(zhǔn)備工作
1、訓(xùn)練好的詞向量
2拼岳、用于訓(xùn)練的文本（已完成分詞，每篇文章且含有對應(yīng)label）
from __future__ import print_function
import os
import sys
import numpy as np
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.utils import to_categorical
from keras.layers import Dense, Input, Flatten
from keras.layers import Conv1D, MaxPooling1D, Embedding
from keras.models import Model
import gensim
import pandas as pd
MAX_SEQUENCE_LENGTH = 1000 # 每篇文章選取1000個詞
MAX_NB_WORDS = 10000 # 將字典設(shè)置為含有1萬個詞
EMBEDDING_DIM = 300 # 詞向量維度况芒，300維
VALIDATION_SPLIT = 0.2 # 測試集大小惜纸，全部數(shù)據(jù)的20%
step 1 選取詞頻最高的一部分詞
預(yù)訓(xùn)練好的詞向量200萬個詞每個300維，這個腳本的目的是實驗性的將流程跑通绝骚。模型訓(xùn)練過程沒問題后再增加詞的個數(shù)耐版。
# 目的是得到一份字典(embeddings_index)含有1萬個詞，每個詞對應(yīng)屬于自己的300維向量
embeddings_index = {}
print('Indexing word vectors.')
path = '../word2vec_model'
model = gensim.models.Word2Vec.load(path)
word_vectors = model.wv
for word, vocab_obj in model.wv.vocab.items():
if int(vocab_obj.index) < MAX_NB_WORDS:
embeddings_index[word] = word_vectors[word]
del model, word_vectors # 刪掉gensim模型釋放內(nèi)存
print('Found %s word vectors.' % len(embeddings_index))
# print out:
# Indexing word vectors.
# Found 10000 word vectors.
step 2 獲取訓(xùn)練文本和對應(yīng)的標(biāo)簽
我的訓(xùn)練數(shù)據(jù)保存成了csv文件压汪，有三列 content, channel_id, name粪牲，其中的name與channel_id是一一對應(yīng)的。content已經(jīng)提前分好詞止剖。
print('Processing text dataset')
texts = [] # list of text samples
labels = [] # list of label ids
labels_index = {} # label與name的對應(yīng)關(guān)系
# 讀取數(shù)據(jù)
path = '../content.csv'
contents = pd.read_csv(path)
contents = contents.dropna()
# 提取文本內(nèi)容與label
texts = contents['content'].values.tolist()
labels = contents['channel_id'].map(int)
labels = labels.values.tolist()
# 獲得label與name的對應(yīng)關(guān)系
tem_labels_index = contents.groupby(['name', 'channel_id']).size().reset_index()
tem_labels_index = tem_labels_index[['channel_id', 'name']].values.tolist()
for idx, name in tem_labels_index:
labels_index[name] = idx
del contents, tem_labels_index
print('Found %s texts.' % len(texts))
# print out
# Processing text dataset
# Found 57867 texts.
step 3
文本準(zhǔn)備腺阳，keras相關(guān)函數(shù)在keras 文檔 Text Preprocessing 部分 可以找到
tokenizer = Tokenizer(num_words=MAX_NB_WORDS) # 傳入我們詞向量的字典
tokenizer.fit_on_texts(texts) # 傳入我們的訓(xùn)練數(shù)據(jù)落君，得到訓(xùn)練數(shù)據(jù)中出現(xiàn)的詞的字典
sequences = tokenizer.texts_to_sequences(texts) # 根據(jù)訓(xùn)練數(shù)據(jù)中出現(xiàn)的詞的字典，將訓(xùn)練數(shù)據(jù)轉(zhuǎn)換為sequences
word_index = tokenizer.word_index
print('Found %s unique tokens.' % len(word_index))
data = pad_sequences(sequences, maxlen=MAX_SEQUENCE_LENGTH) # 限制每篇文章的長度
labels = to_categorical(np.asarray(labels)) # label one hot表示
print('Shape of data tensor:', data.shape)
print('Shape of label tensor:', labels.shape)
# print out
# Found 379653 unique tokens.
# Shape of data tensor: (57867, 1000)
# Shape of label tensor: (57867, 26) # 我的文本類別有26類
step 4 準(zhǔn)備訓(xùn)練集與測試集
# 打亂文章順序
indices = np.arange(data.shape[0])
np.random.shuffle(indices)
data = data[indices]
labels = labels[indices]
num_validation_samples = int(VALIDATION_SPLIT * data.shape[0])
# 切割數(shù)據(jù)
x_train = data[:-num_validation_samples]
y_train = labels[:-num_validation_samples]
x_val = data[-num_validation_samples:]
y_val = labels[-num_validation_samples:]
step 5 準(zhǔn)備embedding layer
num_words = min(MAX_NB_WORDS, len(word_index)) # 對比詞向量字典中包含詞的個數(shù)與文本數(shù)據(jù)所有詞的個數(shù)亭引，取小
embedding_matrix = np.zeros((num_words, EMBEDDING_DIM))
for word, i in word_index.items():
if i >= MAX_NB_WORDS:
continue
embedding_vector = embeddings_index.get(word)
if embedding_vector is not None:
# 文本數(shù)據(jù)中的詞在詞向量字典中沒有绎速，向量為取0；如果有則取詞向量中該詞的向量
embedding_matrix[i] = embedding_vector
# 將預(yù)訓(xùn)練好的詞向量加載如embedding layer
# 我們設(shè)置 trainable = False焙蚓，代表詞向量不作為參數(shù)進(jìn)行更新
embedding_layer = Embedding(num_words,
EMBEDDING_DIM,
weights=[embedding_matrix],
input_length=MAX_SEQUENCE_LENGTH,
trainable=False)
step 6 訓(xùn)練模型
做了那么多準(zhǔn)備朝氓，我們終于可以訓(xùn)練模型啦！
keras 文檔 pooling 部分
keras 文檔 convolutional 部分
# 訓(xùn)練 1D 卷積神經(jīng)網(wǎng)絡(luò) 使用 Maxpooling1D
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
x = Conv1D(filters=128, kernel_size=5, activation='relu')(embedded_sequences)
x = MaxPooling1D((pool_size=5)(x)
x = Conv1D(filters=128, kernel_size=5, activation='relu')(x)
x = MaxPooling1D((pool_size=5)(x)
x = Conv1D(filters=128, kernel_size=5,, activation='relu')(x)
x = MaxPooling1D((pool_size=35)(x)
x = Flatten()(x)
x = Dense(128, activation='relu')(x)
preds = Dense(len(labels_index), activation='softmax')(x)
model = Model(sequence_input, preds)
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['acc'])
# 如果希望短一些時間可以主届，epochs調(diào)小
model.fit(x_train, y_train,
batch_size=128,
epochs=50,
validation_data=(x_val, y_val))
準(zhǔn)確率變化
總結(jié)
訓(xùn)練集準(zhǔn)確率92.29%左右赵哲，測試集準(zhǔn)確率74.96%左右，說明模型可能過擬合了君丁。沒關(guān)系枫夺，我們已經(jīng)實現(xiàn)了目標(biāo)。整個流程跑通了绘闷。為了提高準(zhǔn)確率橡庞，可以嘗試：
1、增加文章數(shù)量印蔗，這次測試我用的文章不多
2扒最、文章類別均衡些，這次我用的文章類別嚴(yán)重有偏华嘹，某些類別文章特別多
3吧趣、嘗試dropout和Batch normalization控制過擬合
4、嘗試改變網(wǎng)絡(luò)結(jié)構(gòu)
