關(guān)鍵詞：
- word embedding
- cnn
- glove:大神預(yù)先做好的詞向量(就是每個(gè)單詞用固定維數(shù)的向量表示)
- 20_newsgroup:需要進(jìn)行分類(lèi)的文本(training data and testing data)
過(guò)程
- 將所有的新聞樣本轉(zhuǎn)化為詞索引序列
- 生成一個(gè)glove詞向量矩陣
- 將詞向量矩陣載入Keras Embedding層,設(shè)置該層的權(quán)重不可再訓(xùn)練
- Keras Embedding層之后連接一個(gè)1D的卷積層，并用一個(gè)softmax全連接輸出新聞?lì)悇e
原理分析
Get training data and testing data
texts = [] # texts表示的是一系列文件文字集(X)
labels_index = {} # 這個(gè)字典用于講文件夾名（新聞?lì)悇e）和label_id對(duì)應(yīng)起來(lái)
labels = [] # labels表示每個(gè)text所屬類(lèi)別(y)
TEXT_DATA_DIR = "text_data/20_newsgroup"
for name in sorted(os.listdir(TEXT_DATA_DIR)):
path = os.path.join(TEXT_DATA_DIR, name)
print('\n遍歷文件夾 %s :' % name)
if os.path.isdir(path):
labels_id = len(labels_index)
labels_index[name] = labels_id
for fname in sorted(os.listdir(path)):
if fname.isdigit():
print("%s \t" % fname, end="")
fpath = os.path.join(path, fname)
f = open(fpath, encoding='latin1')
texts.append(f.read())
f.close()
labels.append(labels_id)
# 得到的每個(gè)text表示的是一個(gè)文件里的所有內(nèi)容鞠眉，而不是指單獨(dú)一個(gè)單詞
print('Found %s texts.' % len(texts))
>>> Found 19997 texts.
每個(gè)text包含個(gè)數(shù)不等的word,并且有對(duì)應(yīng)的新聞?lì)悇e(labels_id),labels_index就是連接這個(gè)兩個(gè)關(guān)系的字典
keras.preprocessing.text.textTokenizer
文本預(yù)處理
MAX_NUM_WORDS = 6000
tokenizer = Tokenizer(num_words=MAX_NUM_WORDS)
tokenizer.fit_on_texts(texts)
sequences = tokenizer.texts_to_sequences(texts)
# 字典草巡，key = sequences of text, values = labels_id
word_index = tokenizer.word_index
print('Found %s unique tokens.' % len(texts))
print('Found %s words.' % len(word_index))
MAX_SEQUENCE_LENGTH = 500 # 每個(gè)text中取多少word, （多剪少補(bǔ)）
data = pad_sequences(sequences, maxlen=MAX_SEQUENCE_LENGTH)
labels = to_categorical(np.array(labels))
print('Shape of data tensor:', data.shape)
print('Shape of label tensor:', labels.shape)
Tokenizer類(lèi)講每個(gè)text變成一個(gè)sequence百宇，每個(gè)word對(duì)應(yīng)sequence中的元素庆锦。
輸出結(jié)果
將有序數(shù)據(jù)隨機(jī)化用于訓(xùn)練和驗(yàn)證：比例(8:2)
VALIDATION_SPLIT = 0.2
indices = np.arange(data.shape[0])
np.random.shuffle(indices)
data = data[indices]
labels = labels[indices]
nb_validation_samples = int(VALIDATION_SPLIT * data.shape[0])
x_train = data[:-nb_validation_samples]
y_train = labels[:-nb_validation_samples]
x_val = data[-nb_validation_samples:]
y_val = labels[-nb_validation_samples:]
獲取Glove詞向量
# glove字典辫继，每個(gè)單詞對(duì)應(yīng)一個(gè)100維的向量
def get_glove_dict(glove_dir):
embeddings_index = {}
f = open(os.path.join(glove_dir, 'glove.6B.100d.txt'))
for line in f:
values = line.split()
word = values[0]
coefs = np.asarray(values[1:], dtype='float32')
embeddings_index[word] = coefs
f.close()
print('Found %s word vectors.' % len(embeddings_index))
return embeddings_index
輸出:
>>> Found 400000 word vectors.
詞向量嵌入Embedding
GLOVE_DIR = 'text_data/glove.6B'
EMBEDDING_DIM = 100
embedding_matrix = np.zeros((len(word_index) + 1, EMBEDDING_DIM))
embedding_index = get_glove_dict(GLOVE_DIR)
for word, i in word_index.items():
embedding_vector = embedding_index.get(word)
if embedding_vector is not None:
# words not found in embedding index will be all-zeros.
embedding_matrix[i] = embedding_vector
embedding_layer = Embedding(len(word_index) + 1,
EMBEDDING_DIM,
weights=[embedding_matrix],
input_length=MAX_SEQUENCE_LENGTH,
trainable=False)
分析：
這里我們獲取到一個(gè)400000x100的詞向量硼婿，看來(lái)英語(yǔ)單詞大概有400000個(gè)锌半。
training_data是19997x500，每個(gè)元素表示一個(gè)word寇漫，我們需要為每個(gè)word再詞向量中找到它對(duì)應(yīng)的向量刊殉，這就是word embedding需要做的工作，可以節(jié)省大量計(jì)算時(shí)間州胳。
embedding_vector = embedding_index.get(word)
就是獲取traing_data中出現(xiàn)的word的vector
輸入卷積層
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
x = Conv1D(128, 5, activation='relu')(embedded_sequences)
x = MaxPooling1D(5)(x)
x = Conv1D(128, 5, activation='relu')(x)
x = MaxPooling1D(5)(x)
x = Conv1D(128, 5, activation='relu')(x)
x = MaxPooling1D(15)(x) # global max pooling
x = Flatten()(x)
x = Dense(128, activation='relu')(x)
preds = Dense(len(labels_index), activation='softmax')(x)
model = Model(sequence_input, preds)
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['acc'])
# happy learning!
model.fit(x_train, y_train, validation_data=(x_val, y_val),
epochs=5, batch_size=128)
Result：
loss: 0.1446 - acc: 0.9448 - val_loss: 0.1871 - val_acc: 0.9325
還是不錯(cuò)的
參考：
https://github.com/MoyanZitto/keras-cn/blob/master/docs/legacy/blog/word_embedding.md
