RNN循環(huán)神經(jīng)網(wǎng)絡(luò)

用RNN(循環(huán)神經(jīng)網(wǎng)絡(luò))實現(xiàn)連續(xù)數(shù)據(jù)的預(yù)測(以股票預(yù)測為例)

回顧卷積神經(jīng)網(wǎng)絡(luò)

卷積就是特征提取器，就是C(卷積)B(批標(biāo)準(zhǔn)化)A(激活)P(池化)D(隨機(jī)丟棄)

卷積神經(jīng)網(wǎng)絡(luò)：借助卷積核提取空間特征后酌壕，送入全連接網(wǎng)絡(luò)软瞎，實現(xiàn)離散數(shù)據(jù)的分類

循環(huán)神經(jīng)網(wǎng)絡(luò)

循環(huán)核

參數(shù)時間共享扎阶，循環(huán)層提取時間信息

前向傳播時：記憶體內(nèi)存儲的狀態(tài)信息ht秘蛔，在每個時刻都被刷新然痊，三個參數(shù)矩陣wxh饼丘、whh笼恰、why自始自終都是固定不變的冤荆。

反向傳播時：三個參數(shù)矩陣wxh朴则、whh、why被梯度下降法更新

y_t=softmax(h_t w_{hy}+by),h_t=tanh(x_t w_{xh}+h_{t-1} w_{hh}+bh)

循環(huán)核時間步展開

按照時間步展開，就是把循環(huán)核按照時間軸方向展開乌妒，每個時刻記憶體狀態(tài)信息ht被更新汹想，記憶體周圍的參數(shù)矩陣wxh、whh和why是固定不變的撤蚊，我們訓(xùn)練優(yōu)化的就是這些參數(shù)矩陣古掏，訓(xùn)練完成后，使用效果最好的參數(shù)矩陣侦啸，執(zhí)行前向傳播槽唾，輸出預(yù)測結(jié)果，其實光涂，這和我們?nèi)祟惖念A(yù)測是一致的庞萍，你腦中的記憶體都會根據(jù)當(dāng)前的輸入而更新。當(dāng)前的預(yù)測推理忘闻，是根據(jù)你以往的知識積累钝计，用固化下來的參數(shù)矩陣進(jìn)行的推理判斷

循環(huán)神經(jīng)網(wǎng)絡(luò)：就是借助循環(huán)核提取時間特征后，再將提取到的時間特征信息送入全連接網(wǎng)絡(luò)齐佳，實現(xiàn)連續(xù)數(shù)據(jù)的預(yù)測私恬。

$y_t$

是整個循環(huán)網(wǎng)絡(luò)的末層，從公式來看炼吴，就是一個全連接網(wǎng)絡(luò)本鸣，借助全連接網(wǎng)絡(luò)，實現(xiàn)連續(xù)數(shù)據(jù)預(yù)測

循環(huán)計算層

每個循環(huán)核構(gòu)成一層循環(huán)計算層硅蹦，循環(huán)計算層的層數(shù)是向輸出方向增長的荣德，每個循環(huán)計算層中的每個每個循環(huán)核中記憶體的個數(shù)是根據(jù)你的需求任意指定的

TF描述循環(huán)計算層(帶大家計算循環(huán)計算層的前向傳播)

tf.keras.layers.SimpleRNN(記憶體個數(shù),activation='激活函數(shù)',return_sequences=是否每個時刻輸出ht到下一層)

activation='激活函數(shù)'(不寫，默認(rèn)使用tanh)

return_sequences=True 各時間步輸出ht

return_sequences=False 僅最后時間步輸出ht(默認(rèn))

例：SimpleRNN(3,return_sequences=True)

此api對于送入循環(huán)層的數(shù)據(jù)維度是有要求的提针，要求送入循環(huán)層的數(shù)據(jù)是三維的命爬。第一維是送入樣本的總數(shù)量，第二維是循環(huán)核按時間步展開的步數(shù)辐脖，第三維是每個時間步輸入特征的個數(shù)

循環(huán)計算過程

字母預(yù)測:輸入a預(yù)測出b饲宛，輸入b預(yù)測出c，輸入c預(yù)測出d嗜价，輸入d預(yù)測出e艇抠，輸入e預(yù)測出a

使用獨(dú)熱碼對a、b久锥、c家淤、d、e編碼為10000瑟由、01000絮重、00100、00010、00001

實踐：ABCDE字母預(yù)測

One-hot
Embedding

實踐：股票預(yù)測

RNN
LSTM
GRU

import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Dense, SimpleRNN
import matplotlib.pyplot as plt
import os

input_word = "abcde"
w_to_id = {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4}  # 單詞映射到數(shù)值id的詞典
id_to_onehot = {0: [1., 0., 0., 0., 0.], 1: [0., 1., 0., 0., 0.], 2: [0., 0., 1., 0., 0.], 3: [0., 0., 0., 1., 0.],
                4: [0., 0., 0., 0., 1.]}  # id編碼為one-hot

x_train = [id_to_onehot[w_to_id['a']], id_to_onehot[w_to_id['b']], id_to_onehot[w_to_id['c']],
           id_to_onehot[w_to_id['d']], id_to_onehot[w_to_id['e']]]
y_train = [w_to_id['b'], w_to_id['c'], w_to_id['d'], w_to_id['e'], w_to_id['a']]

np.random.seed(7)
np.random.shuffle(x_train)
np.random.seed(7)
np.random.shuffle(y_train)
tf.random.set_seed(7)

# 使x_train符合SimpleRNN輸入要求：[送入樣本數(shù)青伤， 循環(huán)核時間展開步數(shù)督怜， 每個時間步輸入特征個數(shù)]。
# 此處整個數(shù)據(jù)集送入狠角，送入樣本數(shù)為len(x_train)号杠；輸入1個字母出結(jié)果，循環(huán)核時間展開步數(shù)為1; 表示為獨(dú)熱碼有5個輸入特征丰歌，每個時間步輸入特征個數(shù)為5
x_train = np.reshape(x_train, (len(x_train), 1, 5))
y_train = np.array(y_train)

model = tf.keras.Sequential([
    SimpleRNN(3),
    Dense(5, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(0.01),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

checkpoint_save_path = "./checkpoint/rnn_onehot_1pre1.ckpt"

if os.path.exists(checkpoint_save_path + '.index'):
    print('-------------load the model-----------------')
    model.load_weights(checkpoint_save_path)

cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
                                                 save_weights_only=True,
                                                 save_best_only=True,
                                                 monitor='loss')  # 由于fit沒有給出測試集姨蟋，不計算測試集準(zhǔn)確率，根據(jù)loss立帖，保存最優(yōu)模型

history = model.fit(x_train, y_train, batch_size=32, epochs=100, callbacks=[cp_callback])

model.summary()

# print(model.trainable_variables)
file = open('./weights.txt', 'w')  # 參數(shù)提取
for v in model.trainable_variables:
    file.write(str(v.name) + '\n')
    file.write(str(v.shape) + '\n')
    file.write(str(v.numpy()) + '\n')
file.close()

###############################################    show   ###############################################

# 顯示訓(xùn)練集和驗證集的acc和loss曲線
acc = history.history['sparse_categorical_accuracy']
loss = history.history['loss']

plt.subplot(1, 2, 1)
plt.plot(acc, label='Training Accuracy')
plt.title('Training Accuracy')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(loss, label='Training Loss')
plt.title('Training Loss')
plt.legend()
plt.show()

############### predict #############

preNum = int(input("input the number of test alphabet:"))
for i in range(preNum):
    alphabet1 = input("input test alphabet:")
    alphabet = [id_to_onehot[w_to_id[alphabet1]]]
    # 使alphabet符合SimpleRNN輸入要求：[送入樣本數(shù)眼溶， 循環(huán)核時間展開步數(shù)， 每個時間步輸入特征個數(shù)]厘惦。此處驗證效果送入了1個樣本偷仿，送入樣本數(shù)為1哩簿；輸入1個字母出結(jié)果宵蕉，所以循環(huán)核時間展開步數(shù)為1; 表示為獨(dú)熱碼有5個輸入特征，每個時間步輸入特征個數(shù)為5
    alphabet = np.reshape(alphabet, (1, 1, 5))
    result = model.predict(alphabet)
    pred = tf.argmax(result, axis=1)
    pred = int(pred)
    tf.print(alphabet1 + '->' + input_word[pred])

-------------load the model-----------------
Epoch 1/100
1/1 [==============================] - 0s 65ms/step - loss: 0.0397 - sparse_categorical_accuracy: 1.0000
Epoch 2/100
1/1 [==============================] - 0s 57ms/step - loss: 0.0395 - sparse_categorical_accuracy: 1.0000
Epoch 3/100
1/1 [==============================] - 0s 56ms/step - loss: 0.0393 - sparse_categorical_accuracy: 1.0000
Epoch 4/100
1/1 [==============================] - 0s 53ms/step - loss: 0.0391 - sparse_categorical_accuracy: 1.0000
Epoch 95/100
1/1 [==============================] - 0s 59ms/step - loss: 0.0251 - sparse_categorical_accuracy: 1.0000
Epoch 96/100
1/1 [==============================] - 0s 56ms/step - loss: 0.0250 - sparse_categorical_accuracy: 1.0000
Epoch 97/100
1/1 [==============================] - 0s 57ms/step - loss: 0.0249 - sparse_categorical_accuracy: 1.0000
Epoch 98/100
1/1 [==============================] - 0s 56ms/step - loss: 0.0248 - sparse_categorical_accuracy: 1.0000
Epoch 99/100
1/1 [==============================] - 0s 51ms/step - loss: 0.0247 - sparse_categorical_accuracy: 1.0000
Epoch 100/100
1/1 [==============================] - 0s 64ms/step - loss: 0.0246 - sparse_categorical_accuracy: 1.0000
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn (SimpleRNN)       multiple                  27        
_________________________________________________________________
dense (Dense)                multiple                  20        
=================================================================
Total params: 47
Trainable params: 47
Non-trainable params: 0
_________________________________________________________________

output_1_1.png

input the number of test alphabet:5
input test alphabet:a
a->b
input test alphabet:b
b->c
input test alphabet:d
d->e
input test alphabet:c
c->d
input test alphabet:a
a->b

循環(huán)計算過程

輸入abcd輸出e节榜，輸入bcde輸出a羡玛，輸入cdea輸出b，輸入deab輸出c宗苍，輸入eabc輸出d

連續(xù)輸入四個字母稼稿，預(yù)測下一個字母為例描述循環(huán)核按時間展開后，循環(huán)計算過程讳窟。
下面為代碼

import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Dense, SimpleRNN
import matplotlib.pyplot as plt
import os

input_word = "abcde"
w_to_id = {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4}  # 單詞映射到數(shù)值id的詞典
id_to_onehot = {0: [1., 0., 0., 0., 0.], 1: [0., 1., 0., 0., 0.], 2: [0., 0., 1., 0., 0.], 3: [0., 0., 0., 1., 0.],
                4: [0., 0., 0., 0., 1.]}  # id編碼為one-hot

x_train = [
    [id_to_onehot[w_to_id['a']], id_to_onehot[w_to_id['b']], id_to_onehot[w_to_id['c']], id_to_onehot[w_to_id['d']]],
    [id_to_onehot[w_to_id['b']], id_to_onehot[w_to_id['c']], id_to_onehot[w_to_id['d']], id_to_onehot[w_to_id['e']]],
    [id_to_onehot[w_to_id['c']], id_to_onehot[w_to_id['d']], id_to_onehot[w_to_id['e']], id_to_onehot[w_to_id['a']]],
    [id_to_onehot[w_to_id['d']], id_to_onehot[w_to_id['e']], id_to_onehot[w_to_id['a']], id_to_onehot[w_to_id['b']]],
    [id_to_onehot[w_to_id['e']], id_to_onehot[w_to_id['a']], id_to_onehot[w_to_id['b']], id_to_onehot[w_to_id['c']]],
]
y_train = [w_to_id['e'], w_to_id['a'], w_to_id['b'], w_to_id['c'], w_to_id['d']]

np.random.seed(7)
np.random.shuffle(x_train)
np.random.seed(7)
np.random.shuffle(y_train)
tf.random.set_seed(7)

# 使x_train符合SimpleRNN輸入要求：[送入樣本數(shù)让歼， 循環(huán)核時間展開步數(shù)， 每個時間步輸入特征個數(shù)]丽啡。
# 此處整個數(shù)據(jù)集送入谋右，送入樣本數(shù)為len(x_train)；輸入4個字母出結(jié)果补箍，循環(huán)核時間展開步數(shù)為4; 表示為獨(dú)熱碼有5個輸入特征改执，每個時間步輸入特征個數(shù)為5
x_train = np.reshape(x_train, (len(x_train), 4, 5))
y_train = np.array(y_train)

model = tf.keras.Sequential([
    SimpleRNN(3),
    Dense(5, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(0.01),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

checkpoint_save_path = "./checkpoint/rnn_onehot_4pre1.ckpt"

if os.path.exists(checkpoint_save_path + '.index'):
    print('-------------load the model-----------------')
    model.load_weights(checkpoint_save_path)

cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
                                                 save_weights_only=True,
                                                 save_best_only=True,
                                                 monitor='loss')  # 由于fit沒有給出測試集，不計算測試集準(zhǔn)確率坑雅，根據(jù)loss辈挂，保存最優(yōu)模型

history = model.fit(x_train, y_train, batch_size=32, epochs=100, callbacks=[cp_callback])

model.summary()

# print(model.trainable_variables)
file = open('./weights.txt', 'w')  # 參數(shù)提取
for v in model.trainable_variables:
    file.write(str(v.name) + '\n')
    file.write(str(v.shape) + '\n')
    file.write(str(v.numpy()) + '\n')
file.close()

###############################################    show   ###############################################

# 顯示訓(xùn)練集和驗證集的acc和loss曲線
acc = history.history['sparse_categorical_accuracy']
loss = history.history['loss']

plt.subplot(1, 2, 1)
plt.plot(acc, label='Training Accuracy')
plt.title('Training Accuracy')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(loss, label='Training Loss')
plt.title('Training Loss')
plt.legend()
plt.show()

############### predict #############

preNum = int(input("input the number of test alphabet:"))
for i in range(preNum):
    alphabet1 = input("input test alphabet:")
    alphabet = [id_to_onehot[w_to_id[a]] for a in alphabet1]
    # 使alphabet符合SimpleRNN輸入要求：[送入樣本數(shù)， 循環(huán)核時間展開步數(shù)裹粤， 每個時間步輸入特征個數(shù)]终蒂。此處驗證效果送入了1個樣本，送入樣本數(shù)為1；輸入4個字母出結(jié)果拇泣，所以循環(huán)核時間展開步數(shù)為4; 表示為獨(dú)熱碼有5個輸入特征悉尾，每個時間步輸入特征個數(shù)為5
    alphabet = np.reshape(alphabet, (1, 4, 5))
    result = model.predict(alphabet)
    pred = tf.argmax(result, axis=1)
    pred = int(pred)
    tf.print(alphabet1 + '->' + input_word[pred])

-------------load the model-----------------
Epoch 1/100
1/1 [==============================] - 0s 65ms/step - loss: 0.2530 - sparse_categorical_accuracy: 1.0000
Epoch 2/100
1/1 [==============================] - 0s 53ms/step - loss: 0.2490 - sparse_categorical_accuracy: 1.0000
Epoch 3/100
1/1 [==============================] - 0s 51ms/step - loss: 0.2451 - sparse_categorical_accuracy: 1.0000
Epoch 4/100
1/1 [==============================] - 0s 57ms/step - loss: 0.2412 - sparse_categorical_accuracy: 1.0000
Epoch 5/100
1/1 [==============================] - 0s 65ms/step - loss: 0.2375 - sparse_categorical_accuracy: 1.0000
Epoch 96/100
1/1 [==============================] - 0s 75ms/step - loss: 0.0873 - sparse_categorical_accuracy: 1.0000
Epoch 97/100
1/1 [==============================] - 0s 59ms/step - loss: 0.0866 - sparse_categorical_accuracy: 1.0000
Epoch 98/100
1/1 [==============================] - 0s 55ms/step - loss: 0.0859 - sparse_categorical_accuracy: 1.0000
Epoch 99/100
1/1 [==============================] - 0s 57ms/step - loss: 0.0852 - sparse_categorical_accuracy: 1.0000
Epoch 100/100
1/1 [==============================] - 0s 56ms/step - loss: 0.0846 - sparse_categorical_accuracy: 1.0000
Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn_2 (SimpleRNN)     multiple                  27        
_________________________________________________________________
dense_2 (Dense)              multiple                  20        
=================================================================
Total params: 47
Trainable params: 47
Non-trainable params: 0
_________________________________________________________________

output_3_1.png

input the number of test alphabet:5
input test alphabet:abcd
abcd->e
input test alphabet:bcda
bcda->b
input test alphabet:cdab
cdab->c
input test alphabet:dabc
dabc->d
input test alphabet:eabc
eabc->d

Embedding--一種編碼方法

獨(dú)熱碼：數(shù)據(jù)量大過于稀疏，映射之間是獨(dú)立的挫酿，沒有表現(xiàn)出關(guān)聯(lián)性

Embedding：是一種單詞編碼方法构眯，用低維向量實現(xiàn)了編碼。這種編碼通過神經(jīng)網(wǎng)絡(luò)訓(xùn)練優(yōu)化早龟，能表達(dá)出單詞間的相關(guān)性

tf.keras.layers.Embedding(詞匯表大小惫霸，編碼維度)

編碼維度就是用幾個數(shù)字表達(dá)一個單詞，對1-100進(jìn)行編碼葱弟，[4]編碼為[0.25,0.1,0.11]

例：tf.keras.layers.Embedding(100,3)

入Embedding時壹店，x_train維度：[送入樣本數(shù)，循環(huán)核時間展開步數(shù)]

# Embedding編碼實現(xiàn)輸入一個字母預(yù)測一個字母例子
import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Dense, SimpleRNN, Embedding
import matplotlib.pyplot as plt
import os

input_word = "abcde"
w_to_id = {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4}  # 單詞映射到數(shù)值id的詞典

x_train = [w_to_id['a'], w_to_id['b'], w_to_id['c'], w_to_id['d'], w_to_id['e']]
y_train = [w_to_id['b'], w_to_id['c'], w_to_id['d'], w_to_id['e'], w_to_id['a']]

np.random.seed(7)
np.random.shuffle(x_train)
np.random.seed(7)
np.random.shuffle(y_train)
tf.random.set_seed(7)

# 使x_train符合Embedding輸入要求：[送入樣本數(shù)芝加， 循環(huán)核時間展開步數(shù)] 硅卢，
# 此處整個數(shù)據(jù)集送入所以送入，送入樣本數(shù)為len(x_train)藏杖；輸入1個字母出結(jié)果将塑，循環(huán)核時間展開步數(shù)為1。
x_train = np.reshape(x_train, (len(x_train), 1))
y_train = np.array(y_train)

model = tf.keras.Sequential([
    Embedding(5, 2),
    SimpleRNN(3),
    Dense(5, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(0.01),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

checkpoint_save_path = "./checkpoint/run_embedding_1pre1.ckpt"

if os.path.exists(checkpoint_save_path + '.index'):
    print('-------------load the model-----------------')
    model.load_weights(checkpoint_save_path)

cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
                                                 save_weights_only=True,
                                                 save_best_only=True,
                                                 monitor='loss')  # 由于fit沒有給出測試集蝌麸，不計算測試集準(zhǔn)確率点寥，根據(jù)loss，保存最優(yōu)模型

history = model.fit(x_train, y_train, batch_size=32, epochs=100, callbacks=[cp_callback])

model.summary()

# print(model.trainable_variables)
file = open('./weights.txt', 'w')  # 參數(shù)提取
for v in model.trainable_variables:
    file.write(str(v.name) + '\n')
    file.write(str(v.shape) + '\n')
    file.write(str(v.numpy()) + '\n')
file.close()

###############################################    show   ###############################################

# 顯示訓(xùn)練集和驗證集的acc和loss曲線
acc = history.history['sparse_categorical_accuracy']
loss = history.history['loss']

plt.subplot(1, 2, 1)
plt.plot(acc, label='Training Accuracy')
plt.title('Training Accuracy')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(loss, label='Training Loss')
plt.title('Training Loss')
plt.legend()
plt.show()

############### predict #############

preNum = int(input("input the number of test alphabet:"))
for i in range(preNum):
    alphabet1 = input("input test alphabet:")
    alphabet = [w_to_id[alphabet1]]
    # 使alphabet符合Embedding輸入要求：[送入樣本數(shù)来吩， 循環(huán)核時間展開步數(shù)]敢辩。
    # 此處驗證效果送入了1個樣本，送入樣本數(shù)為1弟疆；輸入1個字母出結(jié)果戚长，循環(huán)核時間展開步數(shù)為1。
    alphabet = np.reshape(alphabet, (1, 1))
    result = model.predict(alphabet)
    pred = tf.argmax(result, axis=1)
    pred = int(pred)
    tf.print(alphabet1 + '->' + input_word[pred])

-------------load the model-----------------
Epoch 1/100
1/1 [==============================] - 0s 59ms/step - loss: 0.5450 - sparse_categorical_accuracy: 1.0000
Epoch 2/100
1/1 [==============================] - 0s 55ms/step - loss: 0.5380 - sparse_categorical_accuracy: 1.0000
Epoch 3/100
1/1 [==============================] - 0s 48ms/step - loss: 0.5312 - sparse_categorical_accuracy: 1.0000
Epoch 96/100
1/1 [==============================] - 0s 57ms/step - loss: 0.2340 - sparse_categorical_accuracy: 1.0000
Epoch 97/100
1/1 [==============================] - 0s 58ms/step - loss: 0.2324 - sparse_categorical_accuracy: 1.0000
Epoch 98/100
1/1 [==============================] - 0s 54ms/step - loss: 0.2308 - sparse_categorical_accuracy: 1.0000
Epoch 99/100
1/1 [==============================] - 0s 70ms/step - loss: 0.2292 - sparse_categorical_accuracy: 1.0000
Epoch 100/100
1/1 [==============================] - 0s 61ms/step - loss: 0.2275 - sparse_categorical_accuracy: 1.0000
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding (Embedding)        (None, None, 2)           10        
_________________________________________________________________
simple_rnn (SimpleRNN)       (None, 3)                 18        
_________________________________________________________________
dense (Dense)                (None, 5)                 20        
=================================================================
Total params: 48
Trainable params: 48
Non-trainable params: 0
_________________________________________________________________

output_5_1.png

input the number of test alphabet:5
input test alphabet:a
a->b
input test alphabet:c
c->d
input test alphabet:d
d->e
input test alphabet:e
e->a
input test alphabet:b
b->c

# Embedding編碼實現(xiàn)輸入四個字母預(yù)測一個字母例子
import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Dense, SimpleRNN, Embedding
import matplotlib.pyplot as plt
import os

input_word = "abcdefghijklmnopqrstuvwxyz"
w_to_id = {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4,
           'f': 5, 'g': 6, 'h': 7, 'i': 8, 'j': 9,
           'k': 10, 'l': 11, 'm': 12, 'n': 13, 'o': 14,
           'p': 15, 'q': 16, 'r': 17, 's': 18, 't': 19,
           'u': 20, 'v': 21, 'w': 22, 'x': 23, 'y': 24, 'z': 25}  # 單詞映射到數(shù)值id的詞典

training_set_scaled = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
                       11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
                       21, 22, 23, 24, 25]

x_train = []
y_train = []

for i in range(4, 26):
    x_train.append(training_set_scaled[i - 4:i])
    y_train.append(training_set_scaled[i])

np.random.seed(7)
np.random.shuffle(x_train)
np.random.seed(7)
np.random.shuffle(y_train)
tf.random.set_seed(7)

# 使x_train符合Embedding輸入要求：[送入樣本數(shù)怠苔， 循環(huán)核時間展開步數(shù)] 同廉，
# 此處整個數(shù)據(jù)集送入所以送入，送入樣本數(shù)為len(x_train)嘀略；輸入4個字母出結(jié)果恤溶，循環(huán)核時間展開步數(shù)為4。
x_train = np.reshape(x_train, (len(x_train), 4))
y_train = np.array(y_train)

model = tf.keras.Sequential([
    Embedding(26, 2),
    SimpleRNN(10),
    Dense(26, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(0.01),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

checkpoint_save_path = "./checkpoint/rnn_embedding_4pre1.ckpt"

if os.path.exists(checkpoint_save_path + '.index'):
    print('-------------load the model-----------------')
    model.load_weights(checkpoint_save_path)

cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
                                                 save_weights_only=True,
                                                 save_best_only=True,
                                                 monitor='loss')  # 由于fit沒有給出測試集帜羊，不計算測試集準(zhǔn)確率咒程，根據(jù)loss，保存最優(yōu)模型

history = model.fit(x_train, y_train, batch_size=32, epochs=100, callbacks=[cp_callback])

model.summary()

file = open('./weights.txt', 'w')  # 參數(shù)提取
for v in model.trainable_variables:
    file.write(str(v.name) + '\n')
    file.write(str(v.shape) + '\n')
    file.write(str(v.numpy()) + '\n')
file.close()

###############################################    show   ###############################################

# 顯示訓(xùn)練集和驗證集的acc和loss曲線
acc = history.history['sparse_categorical_accuracy']
loss = history.history['loss']

plt.subplot(1, 2, 1)
plt.plot(acc, label='Training Accuracy')
plt.title('Training Accuracy')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(loss, label='Training Loss')
plt.title('Training Loss')
plt.legend()
plt.show()

################# predict ##################

preNum = int(input("input the number of test alphabet:"))
for i in range(preNum):
    alphabet1 = input("input test alphabet:")
    alphabet = [w_to_id[a] for a in alphabet1]
    # 使alphabet符合Embedding輸入要求：[送入樣本數(shù)讼育， 時間展開步數(shù)]帐姻。
    # 此處驗證效果送入了1個樣本稠集，送入樣本數(shù)為1；輸入4個字母出結(jié)果饥瓷，循環(huán)核時間展開步數(shù)為4剥纷。
    alphabet = np.reshape(alphabet, (1, 4))
    result = model.predict([alphabet])
    pred = tf.argmax(result, axis=1)
    pred = int(pred)
    tf.print(alphabet1 + '->' + input_word[pred])

Epoch 1/100
1/1 [==============================] - 0s 60ms/step - loss: 3.2579 - sparse_categorical_accuracy: 0.0455
Epoch 2/100
1/1 [==============================] - 0s 56ms/step - loss: 3.2364 - sparse_categorical_accuracy: 0.0455
Epoch 3/100
1/1 [==============================] - 0s 51ms/step - loss: 3.2139 - sparse_categorical_accuracy: 0.0909
Epoch 4/100
1/1 [==============================] - 0s 54ms/step - loss: 3.1891 - sparse_categorical_accuracy: 0.0455
Epoch 5/100
1/1 [==============================] - 0s 53ms/step - loss: 3.1619 - sparse_categorical_accuracy: 0.0455
Epoch 96/100
1/1 [==============================] - 0s 80ms/step - loss: 0.2158 - sparse_categorical_accuracy: 1.0000
Epoch 97/100
1/1 [==============================] - 0s 74ms/step - loss: 0.2103 - sparse_categorical_accuracy: 1.0000
Epoch 98/100
1/1 [==============================] - 0s 66ms/step - loss: 0.2050 - sparse_categorical_accuracy: 1.0000
Epoch 99/100
1/1 [==============================] - 0s 74ms/step - loss: 0.1998 - sparse_categorical_accuracy: 1.0000
Epoch 100/100
1/1 [==============================] - 0s 66ms/step - loss: 0.1948 - sparse_categorical_accuracy: 1.0000
Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, None, 2)           52        
_________________________________________________________________
simple_rnn_1 (SimpleRNN)     (None, 10)                130       
_________________________________________________________________
dense_1 (Dense)              (None, 26)                286       
=================================================================
Total params: 468
Trainable params: 468
Non-trainable params: 0
_________________________________________________________________

output_6_1.png

input the number of test alphabet:3
input test alphabet:abcd
abcd->e
input test alphabet:opqr
opqr->s
input test alphabet:wxyz
wxyz->o

RNN實現(xiàn)股票預(yù)測(使用60天的開盤價預(yù)測第61天的開盤價)

import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Dropout, Dense, SimpleRNN
import matplotlib.pyplot as plt
import os
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error, mean_absolute_error
import math

maotai = pd.read_csv('./SH600519.csv')  # 讀取股票文件

training_set = maotai.iloc[0:2426 - 300, 2:3].values  # 前(2426-300=2126)天的開盤價作為訓(xùn)練集,表格從0開始計數(shù)，2:3 是提取[2:3)列呢铆，前閉后開,故提取出C列開盤價
test_set = maotai.iloc[2426 - 300:, 2:3].values  # 后300天的開盤價作為測試集

# 歸一化
sc = MinMaxScaler(feature_range=(0, 1))  # 定義歸一化：歸一化到(0晦鞋，1)之間
training_set_scaled = sc.fit_transform(training_set)  # 求得訓(xùn)練集的最大值，最小值這些訓(xùn)練集固有的屬性棺克，并在訓(xùn)練集上進(jìn)行歸一化
test_set = sc.transform(test_set)  # 利用訓(xùn)練集的屬性對測試集進(jìn)行歸一化

x_train = []
y_train = []

x_test = []
y_test = []

# 測試集：csv表格中前2426-300=2126天數(shù)據(jù)
# 利用for循環(huán)悠垛，遍歷整個訓(xùn)練集，提取訓(xùn)練集中連續(xù)60天的開盤價作為輸入特征x_train娜谊，第61天的數(shù)據(jù)作為標(biāo)簽确买，for循環(huán)共構(gòu)建2426-300-60=2066組數(shù)據(jù)。
for i in range(60, len(training_set_scaled)):
    x_train.append(training_set_scaled[i - 60:i, 0])
    y_train.append(training_set_scaled[i, 0])
# 對訓(xùn)練集進(jìn)行打亂
np.random.seed(7)
np.random.shuffle(x_train)
np.random.seed(7)
np.random.shuffle(y_train)
tf.random.set_seed(7)
# 將訓(xùn)練集由list格式變?yōu)閍rray格式
x_train, y_train = np.array(x_train), np.array(y_train)

# 使x_train符合RNN輸入要求：[送入樣本數(shù)纱皆， 循環(huán)核時間展開步數(shù)湾趾， 每個時間步輸入特征個數(shù)]。
# 此處整個數(shù)據(jù)集送入派草，送入樣本數(shù)為x_train.shape[0]即2066組數(shù)據(jù)搀缠；輸入60個開盤價，預(yù)測出第61天的開盤價澳眷，循環(huán)核時間展開步數(shù)為60; 每個時間步送入的特征是某一天的開盤價胡嘿，只有1個數(shù)據(jù)，故每個時間步輸入特征個數(shù)為1
x_train = np.reshape(x_train, (x_train.shape[0], 60, 1))
# 測試集：csv表格中后300天數(shù)據(jù)
# 利用for循環(huán)钳踊，遍歷整個測試集，提取測試集中連續(xù)60天的開盤價作為輸入特征x_train勿侯，第61天的數(shù)據(jù)作為標(biāo)簽拓瞪，for循環(huán)共構(gòu)建300-60=240組數(shù)據(jù)。
for i in range(60, len(test_set)):
    x_test.append(test_set[i - 60:i, 0])
    y_test.append(test_set[i, 0])
# 測試集變array并reshape為符合RNN輸入要求：[送入樣本數(shù)助琐， 循環(huán)核時間展開步數(shù)祭埂， 每個時間步輸入特征個數(shù)]
x_test, y_test = np.array(x_test), np.array(y_test)
x_test = np.reshape(x_test, (x_test.shape[0], 60, 1))

model = tf.keras.Sequential([
    SimpleRNN(80, return_sequences=True),
    Dropout(0.2),
    SimpleRNN(100),
    Dropout(0.2),
    Dense(1)
])

model.compile(optimizer=tf.keras.optimizers.Adam(0.001),
              loss='mean_squared_error')  # 損失函數(shù)用均方誤差
# 該應(yīng)用只觀測loss數(shù)值，不觀測準(zhǔn)確率兵钮，所以刪去metrics選項蛆橡，一會在每個epoch迭代顯示時只顯示loss值

checkpoint_save_path = "./checkpoint/rnn_stock.ckpt"

if os.path.exists(checkpoint_save_path + '.index'):
    print('-------------load the model-----------------')
    model.load_weights(checkpoint_save_path)

cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
                                                 save_weights_only=True,
                                                 save_best_only=True,
                                                 monitor='val_loss')

history = model.fit(x_train, y_train, batch_size=64, epochs=50, validation_data=(x_test, y_test), validation_freq=1,
                    callbacks=[cp_callback])

model.summary()

file = open('./weights.txt', 'w')  # 參數(shù)提取
for v in model.trainable_variables:
    file.write(str(v.name) + '\n')
    file.write(str(v.shape) + '\n')
    file.write(str(v.numpy()) + '\n')
file.close()

loss = history.history['loss']
val_loss = history.history['val_loss']

plt.plot(loss, label='Training Loss')
plt.plot(val_loss, label='Validation Loss')
plt.title('Training and Validation Loss')
plt.legend()
plt.show()

################## predict ######################
# 測試集輸入模型進(jìn)行預(yù)測
predicted_stock_price = model.predict(x_test)
# 對預(yù)測數(shù)據(jù)還原---從（0，1）反歸一化到原始范圍
predicted_stock_price = sc.inverse_transform(predicted_stock_price)
# 對真實數(shù)據(jù)還原---從（0掘譬，1）反歸一化到原始范圍
real_stock_price = sc.inverse_transform(test_set[60:])
# 畫出真實數(shù)據(jù)和預(yù)測數(shù)據(jù)的對比曲線
plt.plot(real_stock_price, color='red', label='MaoTai Stock Price')
plt.plot(predicted_stock_price, color='blue', label='Predicted MaoTai Stock Price')
plt.title('MaoTai Stock Price Prediction')
plt.xlabel('Time')
plt.ylabel('MaoTai Stock Price')
plt.legend()
plt.show()

##########evaluate##############
# calculate MSE 均方誤差 ---> E[(預(yù)測值-真實值)^2] (預(yù)測值減真實值求平方后求均值)
mse = mean_squared_error(predicted_stock_price, real_stock_price)
# calculate RMSE 均方根誤差--->sqrt[MSE]    (對均方誤差開方)
rmse = math.sqrt(mean_squared_error(predicted_stock_price, real_stock_price))
# calculate MAE 平均絕對誤差----->E[|預(yù)測值-真實值|](預(yù)測值減真實值求絕對值后求均值）
mae = mean_absolute_error(predicted_stock_price, real_stock_price)
print('均方誤差: %.6f' % mse)
print('均方根誤差: %.6f' % rmse)
print('平均絕對誤差: %.6f' % mae)

Epoch 1/50
33/33 [==============================] - 3s 82ms/step - loss: 0.1162 - val_loss: 0.0434
Epoch 2/50
33/33 [==============================] - 2s 75ms/step - loss: 0.0262 - val_loss: 0.0044
Epoch 46/50
33/33 [==============================] - 3s 106ms/step - loss: 0.0012 - val_loss: 0.0011
Epoch 47/50
33/33 [==============================] - 3s 98ms/step - loss: 0.0013 - val_loss: 0.0098
Epoch 48/50
33/33 [==============================] - 3s 105ms/step - loss: 0.0013 - val_loss: 0.0010
Epoch 49/50
33/33 [==============================] - 3s 99ms/step - loss: 0.0012 - val_loss: 0.0042
Epoch 50/50
33/33 [==============================] - 3s 100ms/step - loss: 0.0011 - val_loss: 0.0081
Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn_2 (SimpleRNN)     multiple                  6560      
_________________________________________________________________
dropout (Dropout)            multiple                  0         
_________________________________________________________________
simple_rnn_3 (SimpleRNN)     multiple                  18100     
_________________________________________________________________
dropout_1 (Dropout)          multiple                  0         
_________________________________________________________________
dense_2 (Dense)              multiple                  101       
=================================================================
Total params: 24,761
Trainable params: 24,761
Non-trainable params: 0
_________________________________________________________________

output_8_1.png

output_8_2.png

均方誤差: 4064.391444
均方根誤差: 63.752580
平均絕對誤差: 59.361189

用LSTM實現(xiàn)股票預(yù)測

傳統(tǒng)循環(huán)網(wǎng)絡(luò)RNN可以通過記憶體實現(xiàn)短期記憶進(jìn)行連續(xù)數(shù)據(jù)的預(yù)測泰演，但是當(dāng)連續(xù)數(shù)據(jù)的序列變長時，會使展開時間步過長葱轩。在反向傳播更新參數(shù)時睦焕，梯度要按照時間步連續(xù)相乘藐握，會導(dǎo)致梯度消失。所以在1997年Hochreitere等人提出了長短期記憶網(wǎng)絡(luò)LSTM

LSTM計算過程

長短期記憶網(wǎng)絡(luò)引入了三個門限垃喊，輸入門

$i_t$

猾普、遺忘門
$ft$

、輸出門
$o_t$

本谜，引入了表征長期記憶的細(xì)胞態(tài)
$C_t$

初家，引入了等待存入長期記憶的候選態(tài)
$\widetilde{C}_t$

輸入門(門限)：

$i_t=\sigma(W_i\cdot [h_{t-1},x_t]+b_i)$

遺忘門(門限)：

$f_t=\sigma(W_f\cdot [h_{t-1},x_t]+b_f)$

輸出門(門限)：

$o_t=\sigma(W_o\cdot [h_{t-1},x_t]+b_o)$

細(xì)胞態(tài)(長期記憶)：

$C_t=f_t*C_{t-1}+i_t*\widetilde{C}_t$

記憶體(短期記憶)：

$h_t=o_t*tanh(C_t)$

候選態(tài)(歸納出的新知識)：

$\widetilde{C}_t=tanh(W_c\cdot[h_{t-1},x_t]+b_c)$

TF描述LSTM層

tf.keras.layers.LSTM(記憶體個數(shù),return_sequences=是否返回輸出)

return_sequences=True 各時間步輸出ht

return_sequences=False 僅最后時間步輸出ht(默認(rèn))

舉個例子

model=tf.keras.Sequential([
    LSTM(80,return_sequences=True),
    Dropout(0.2),
    LSTM(100),
    Dropout(0.2),
    Dense(1)
])

import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Dropout, Dense, LSTM
import matplotlib.pyplot as plt
import os
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error, mean_absolute_error
import math

maotai = pd.read_csv('./SH600519.csv')  # 讀取股票文件

training_set = maotai.iloc[0:2426 - 300, 2:3].values  # 前(2426-300=2126)天的開盤價作為訓(xùn)練集,表格從0開始計數(shù)，2:3 是提取[2:3)列乌助，前閉后開,故提取出C列開盤價
test_set = maotai.iloc[2426 - 300:, 2:3].values  # 后300天的開盤價作為測試集

# 歸一化
sc = MinMaxScaler(feature_range=(0, 1))  # 定義歸一化：歸一化到(0笤成，1)之間
training_set_scaled = sc.fit_transform(training_set)  # 求得訓(xùn)練集的最大值，最小值這些訓(xùn)練集固有的屬性眷茁，并在訓(xùn)練集上進(jìn)行歸一化
test_set = sc.transform(test_set)  # 利用訓(xùn)練集的屬性對測試集進(jìn)行歸一化

x_train = []
y_train = []

x_test = []
y_test = []

# 測試集：csv表格中前2426-300=2126天數(shù)據(jù)
# 利用for循環(huán)炕泳，遍歷整個訓(xùn)練集，提取訓(xùn)練集中連續(xù)60天的開盤價作為輸入特征x_train上祈，第61天的數(shù)據(jù)作為標(biāo)簽培遵，for循環(huán)共構(gòu)建2426-300-60=2066組數(shù)據(jù)。
for i in range(60, len(training_set_scaled)):
    x_train.append(training_set_scaled[i - 60:i, 0])
    y_train.append(training_set_scaled[i, 0])
# 對訓(xùn)練集進(jìn)行打亂
np.random.seed(7)
np.random.shuffle(x_train)
np.random.seed(7)
np.random.shuffle(y_train)
tf.random.set_seed(7)
# 將訓(xùn)練集由list格式變?yōu)閍rray格式
x_train, y_train = np.array(x_train), np.array(y_train)

# 使x_train符合RNN輸入要求：[送入樣本數(shù)登刺， 循環(huán)核時間展開步數(shù)籽腕， 每個時間步輸入特征個數(shù)]。
# 此處整個數(shù)據(jù)集送入纸俭，送入樣本數(shù)為x_train.shape[0]即2066組數(shù)據(jù)皇耗；輸入60個開盤價，預(yù)測出第61天的開盤價揍很，循環(huán)核時間展開步數(shù)為60; 每個時間步送入的特征是某一天的開盤價郎楼，只有1個數(shù)據(jù)，故每個時間步輸入特征個數(shù)為1
x_train = np.reshape(x_train, (x_train.shape[0], 60, 1))
# 測試集：csv表格中后300天數(shù)據(jù)
# 利用for循環(huán)窒悔，遍歷整個測試集呜袁，提取測試集中連續(xù)60天的開盤價作為輸入特征x_train，第61天的數(shù)據(jù)作為標(biāo)簽简珠，for循環(huán)共構(gòu)建300-60=240組數(shù)據(jù)阶界。
for i in range(60, len(test_set)):
    x_test.append(test_set[i - 60:i, 0])
    y_test.append(test_set[i, 0])
# 測試集變array并reshape為符合RNN輸入要求：[送入樣本數(shù)， 循環(huán)核時間展開步數(shù)聋庵， 每個時間步輸入特征個數(shù)]
x_test, y_test = np.array(x_test), np.array(y_test)
x_test = np.reshape(x_test, (x_test.shape[0], 60, 1))

model = tf.keras.Sequential([
    LSTM(80, return_sequences=True),
    Dropout(0.2),
    LSTM(100),
    Dropout(0.2),
    Dense(1)
])

model.compile(optimizer=tf.keras.optimizers.Adam(0.001),
              loss='mean_squared_error')  # 損失函數(shù)用均方誤差
# 該應(yīng)用只觀測loss數(shù)值膘融，不觀測準(zhǔn)確率，所以刪去metrics選項祭玉，一會在每個epoch迭代顯示時只顯示loss值

checkpoint_save_path = "./checkpoint/LSTM_stock.ckpt"

if os.path.exists(checkpoint_save_path + '.index'):
    print('-------------load the model-----------------')
    model.load_weights(checkpoint_save_path)

cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
                                                 save_weights_only=True,
                                                 save_best_only=True,
                                                 monitor='val_loss')

history = model.fit(x_train, y_train, batch_size=64, epochs=50, validation_data=(x_test, y_test), validation_freq=1,
                    callbacks=[cp_callback])

model.summary()

file = open('./weights.txt', 'w')  # 參數(shù)提取
for v in model.trainable_variables:
    file.write(str(v.name) + '\n')
    file.write(str(v.shape) + '\n')
    file.write(str(v.numpy()) + '\n')
file.close()

loss = history.history['loss']
val_loss = history.history['val_loss']

plt.plot(loss, label='Training Loss')
plt.plot(val_loss, label='Validation Loss')
plt.title('Training and Validation Loss')
plt.legend()
plt.show()

################## predict ######################
# 測試集輸入模型進(jìn)行預(yù)測
predicted_stock_price = model.predict(x_test)
# 對預(yù)測數(shù)據(jù)還原---從（0氧映，1）反歸一化到原始范圍
predicted_stock_price = sc.inverse_transform(predicted_stock_price)
# 對真實數(shù)據(jù)還原---從（0，1）反歸一化到原始范圍
real_stock_price = sc.inverse_transform(test_set[60:])
# 畫出真實數(shù)據(jù)和預(yù)測數(shù)據(jù)的對比曲線
plt.plot(real_stock_price, color='red', label='MaoTai Stock Price')
plt.plot(predicted_stock_price, color='blue', label='Predicted MaoTai Stock Price')
plt.title('MaoTai Stock Price Prediction')
plt.xlabel('Time')
plt.ylabel('MaoTai Stock Price')
plt.legend()
plt.show()

##########evaluate##############
# calculate MSE 均方誤差 ---> E[(預(yù)測值-真實值)^2] (預(yù)測值減真實值求平方后求均值)
mse = mean_squared_error(predicted_stock_price, real_stock_price)
# calculate RMSE 均方根誤差--->sqrt[MSE]    (對均方誤差開方)
rmse = math.sqrt(mean_squared_error(predicted_stock_price, real_stock_price))
# calculate MAE 平均絕對誤差----->E[|預(yù)測值-真實值|](預(yù)測值減真實值求絕對值后求均值）
mae = mean_absolute_error(predicted_stock_price, real_stock_price)
print('均方誤差: %.6f' % mse)
print('均方根誤差: %.6f' % rmse)
print('平均絕對誤差: %.6f' % mae)

Epoch 1/50
33/33 [==============================] - 1s 38ms/step - loss: 0.0135 - val_loss: 0.0493
Epoch 2/50
33/33 [==============================] - 1s 25ms/step - loss: 0.0013 - val_loss: 0.0140
Epoch 3/50
33/33 [==============================] - 1s 25ms/step - loss: 0.0011 - val_loss: 0.0070
Epoch 4/50
33/33 [==============================] - 1s 22ms/step - loss: 0.0013 - val_loss: 0.0204
Epoch 5/50
33/33 [==============================] - 1s 26ms/step - loss: 0.0011 - val_loss: 0.0066
Epoch 49/50
33/33 [==============================] - 1s 22ms/step - loss: 6.8864e-04 - val_loss: 0.0059
Epoch 50/50
33/33 [==============================] - 1s 22ms/step - loss: 5.6762e-04 - val_loss: 0.0061
Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lstm (LSTM)                  multiple                  26240     
_________________________________________________________________
dropout_2 (Dropout)          multiple                  0         
_________________________________________________________________
lstm_1 (LSTM)                multiple                  72400     
_________________________________________________________________
dropout_3 (Dropout)          multiple                  0         
_________________________________________________________________
dense_3 (Dense)              multiple                  101       
=================================================================
Total params: 98,741
Trainable params: 98,741
Non-trainable params: 0
_________________________________________________________________

![[圖片上傳中...(output_12_1.png-a5d73f-1601466475068-0)]

output_12_2.png

](https://upload-images.jianshu.io/upload_images/4469078-a2c3276490817af9.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

output_10_2.png

均方誤差: 3084.908334
均方根誤差: 55.541951
平均絕對誤差: 49.045563

用GRU實現(xiàn)股票預(yù)測

在2014年cho等人簡化了LSTM結(jié)構(gòu)攘宙，提出了GRU網(wǎng)絡(luò)屯耸，GRU使記憶體ht融合了長期記憶和短期記憶

更新門：

$z_t=\sigma(W_z\cdot [h_{t-1},x_t])$

重置門：

$r_t=\sigma(W_r\cdot [h_{t-1},x_t])$

記憶體：

$h_t=(1-z_t)*h_{t-1}+z_t*\widetilde{h}_t$

候選隱藏層：

$\widetilde{h}_t=tanh(W\cdot [r_t*h_{t-1},x_t])$

TF描述GRU層

tf.keras.layers.GRU(記憶體個數(shù)拐迁，return_sequences=是否返回輸出)

return_sequences=True 各時間步輸出ht,一般最后一層用false，中間層用true

return_sequences=False 僅最后時間步輸出ht(默認(rèn))

舉個例子

model=tf.keras.Sequential([
    GRU(80,return_sequences=True),
    Dropout(0.2),
    GRU(100),
    Dropout(0.2),
    Dense(1)
])

import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Dropout, Dense, GRU
import matplotlib.pyplot as plt
import os
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error, mean_absolute_error
import math

maotai = pd.read_csv('./SH600519.csv')  # 讀取股票文件

training_set = maotai.iloc[0:2426 - 300, 2:3].values  # 前(2426-300=2126)天的開盤價作為訓(xùn)練集,表格從0開始計數(shù)疗绣，2:3 是提取[2:3)列线召，前閉后開,故提取出C列開盤價
test_set = maotai.iloc[2426 - 300:, 2:3].values  # 后300天的開盤價作為測試集

# 歸一化
sc = MinMaxScaler(feature_range=(0, 1))  # 定義歸一化：歸一化到(0，1)之間
training_set_scaled = sc.fit_transform(training_set)  # 求得訓(xùn)練集的最大值多矮，最小值這些訓(xùn)練集固有的屬性缓淹，并在訓(xùn)練集上進(jìn)行歸一化
test_set = sc.transform(test_set)  # 利用訓(xùn)練集的屬性對測試集進(jìn)行歸一化

x_train = []
y_train = []

x_test = []
y_test = []

# 測試集：csv表格中前2426-300=2126天數(shù)據(jù)
# 利用for循環(huán)，遍歷整個訓(xùn)練集塔逃，提取訓(xùn)練集中連續(xù)60天的開盤價作為輸入特征x_train讯壶，第61天的數(shù)據(jù)作為標(biāo)簽，for循環(huán)共構(gòu)建2426-300-60=2066組數(shù)據(jù)湾盗。
for i in range(60, len(training_set_scaled)):
    x_train.append(training_set_scaled[i - 60:i, 0])
    y_train.append(training_set_scaled[i, 0])
# 對訓(xùn)練集進(jìn)行打亂
np.random.seed(7)
np.random.shuffle(x_train)
np.random.seed(7)
np.random.shuffle(y_train)
tf.random.set_seed(7)
# 將訓(xùn)練集由list格式變?yōu)閍rray格式
x_train, y_train = np.array(x_train), np.array(y_train)

# 使x_train符合RNN輸入要求：[送入樣本數(shù)伏蚊， 循環(huán)核時間展開步數(shù)， 每個時間步輸入特征個數(shù)]格粪。
# 此處整個數(shù)據(jù)集送入躏吊，送入樣本數(shù)為x_train.shape[0]即2066組數(shù)據(jù)；輸入60個開盤價帐萎，預(yù)測出第61天的開盤價比伏，循環(huán)核時間展開步數(shù)為60; 每個時間步送入的特征是某一天的開盤價，只有1個數(shù)據(jù)疆导，故每個時間步輸入特征個數(shù)為1
x_train = np.reshape(x_train, (x_train.shape[0], 60, 1))
# 測試集：csv表格中后300天數(shù)據(jù)
# 利用for循環(huán)赁项，遍歷整個測試集，提取測試集中連續(xù)60天的開盤價作為輸入特征x_train澈段，第61天的數(shù)據(jù)作為標(biāo)簽悠菜，for循環(huán)共構(gòu)建300-60=240組數(shù)據(jù)。
for i in range(60, len(test_set)):
    x_test.append(test_set[i - 60:i, 0])
    y_test.append(test_set[i, 0])
# 測試集變array并reshape為符合RNN輸入要求：[送入樣本數(shù)均蜜， 循環(huán)核時間展開步數(shù)李剖， 每個時間步輸入特征個數(shù)]
x_test, y_test = np.array(x_test), np.array(y_test)
x_test = np.reshape(x_test, (x_test.shape[0], 60, 1))

model = tf.keras.Sequential([
    GRU(80, return_sequences=True),
    Dropout(0.2),
    GRU(100),
    Dropout(0.2),
    Dense(1)
])

model.compile(optimizer=tf.keras.optimizers.Adam(0.001),
              loss='mean_squared_error')  # 損失函數(shù)用均方誤差
# 該應(yīng)用只觀測loss數(shù)值，不觀測準(zhǔn)確率囤耳，所以刪去metrics選項，一會在每個epoch迭代顯示時只顯示loss值

checkpoint_save_path = "./checkpoint/stock.ckpt"

if os.path.exists(checkpoint_save_path + '.index'):
    print('-------------load the model-----------------')
    model.load_weights(checkpoint_save_path)

cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
                                                 save_weights_only=True,
                                                 save_best_only=True,
                                                 monitor='val_loss')

history = model.fit(x_train, y_train, batch_size=64, epochs=50, validation_data=(x_test, y_test), validation_freq=1,
                    callbacks=[cp_callback])

model.summary()

file = open('./weights.txt', 'w')  # 參數(shù)提取
for v in model.trainable_variables:
    file.write(str(v.name) + '\n')
    file.write(str(v.shape) + '\n')
    file.write(str(v.numpy()) + '\n')
file.close()

loss = history.history['loss']
val_loss = history.history['val_loss']

plt.plot(loss, label='Training Loss')
plt.plot(val_loss, label='Validation Loss')
plt.title('Training and Validation Loss')
plt.legend()
plt.show()

################## predict ######################
# 測試集輸入模型進(jìn)行預(yù)測
predicted_stock_price = model.predict(x_test)
# 對預(yù)測數(shù)據(jù)還原---從（0偶芍，1）反歸一化到原始范圍
predicted_stock_price = sc.inverse_transform(predicted_stock_price)
# 對真實數(shù)據(jù)還原---從（0充择，1）反歸一化到原始范圍
real_stock_price = sc.inverse_transform(test_set[60:])
# 畫出真實數(shù)據(jù)和預(yù)測數(shù)據(jù)的對比曲線
plt.plot(real_stock_price, color='red', label='MaoTai Stock Price')
plt.plot(predicted_stock_price, color='blue', label='Predicted MaoTai Stock Price')
plt.title('MaoTai Stock Price Prediction')
plt.xlabel('Time')
plt.ylabel('MaoTai Stock Price')
plt.legend()
plt.show()

##########evaluate##############
# calculate MSE 均方誤差 ---> E[(預(yù)測值-真實值)^2] (預(yù)測值減真實值求平方后求均值)
mse = mean_squared_error(predicted_stock_price, real_stock_price)
# calculate RMSE 均方根誤差--->sqrt[MSE]    (對均方誤差開方)
rmse = math.sqrt(mean_squared_error(predicted_stock_price, real_stock_price))
# calculate MAE 平均絕對誤差----->E[|預(yù)測值-真實值|](預(yù)測值減真實值求絕對值后求均值）
mae = mean_absolute_error(predicted_stock_price, real_stock_price)
print('均方誤差: %.6f' % mse)
print('均方根誤差: %.6f' % rmse)
print('平均絕對誤差: %.6f' % mae)

Epoch 1/50
33/33 [==============================] - 10s 305ms/step - loss: 0.0197 - val_loss: 0.0133
Epoch 2/50
33/33 [==============================] - 11s 334ms/step - loss: 0.0015 - val_loss: 0.0044
Epoch 3/50
33/33 [==============================] - 14s 417ms/step - loss: 0.0011 - val_loss: 0.0029
Epoch 4/50
33/33 [==============================] - 12s 362ms/step - loss: 0.0011 - val_loss: 0.0027
Epoch 49/50
33/33 [==============================] - 10s 312ms/step - loss: 6.2348e-04 - val_loss: 0.0021
Epoch 50/50
33/33 [==============================] - 10s 311ms/step - loss: 5.1747e-04 - val_loss: 0.0035
Model: "sequential_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
gru (GRU)                    multiple                  19920     
_________________________________________________________________
dropout_4 (Dropout)          multiple                  0         
_________________________________________________________________
gru_1 (GRU)                  multiple                  54600     
_________________________________________________________________
dropout_5 (Dropout)          multiple                  0         
_________________________________________________________________
dense_4 (Dense)              multiple                  101       
=================================================================
Total params: 74,621
Trainable params: 74,621
Non-trainable params: 0
_________________________________________________________________

output_12_1.png

output_12_2.png

均方誤差: 1743.491148
均方根誤差: 41.755133
平均絕對誤差: 36.362541