title: pytorch中LSTM筆記
date: 2018-07-12 16:52:53
tags:
- torch項(xiàng)目
categories:
- pytorch

單向LSTM筆記

專業(yè)筆記見中文參考劫拗、英文參考

torch.nn.LSTM()輸入API

• 重要參數(shù)
  • input_size: 每一個(gè)時(shí)步(time_step)輸入到lstm單元的維度.(實(shí)際輸入的數(shù)據(jù)size為[batch_size, input_size])
  • hidden_size: 確定了隱含狀態(tài)hidden_state的維度. 可以簡(jiǎn)單的看成: 構(gòu)造了一個(gè)權(quán)重 , 隱含狀態(tài)

  • num_layers: 疊加的層數(shù)急凰。如圖所示num_layers為3
    圖 1.
• batch_first: 輸入數(shù)據(jù)的size為[batch_size, time_step, input_size]還是[time_step, batch_size, input_size]

示例代碼

使用單向LSTM進(jìn)行MNIST分類

import torch
import torch.utils.data as Data
import torchvision
import matplotlib.pyplot as plt
import numpy as np
BATCH_SIZE = 50

class RNN(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.rnn=torch.nn.LSTM(
            input_size=28,
            hidden_size=64,
            num_layers=1,
            batch_first=True
        )
        self.out=torch.nn.Linear(in_features=64,out_features=10)

    def forward(self,x):
        # 以下關(guān)于shape的注釋只針對(duì)單向
        # output: [batch_size, time_step, hidden_size]
        # h_n: [num_layers,batch_size, hidden_size] # 雖然LSTM的batch_first為True,但是h_n/c_n的第一維還是num_layers
        # c_n: 同h_n
        output,(h_n,c_n)=self.rnn(x)
        print(output.size())
        # output_in_last_timestep=output[:,-1,:] # 也是可以的
        output_in_last_timestep=h_n[-1,:,:]
        # print(output_in_last_timestep.equal(output[:,-1,:])) #ture
        x=self.out(output_in_last_timestep)
        return x

if __name__ == "__main__":
    # 1. 加載數(shù)據(jù)
    training_dataset = torchvision.datasets.MNIST("./mnist", train=True,
                                                  transform=torchvision.transforms.ToTensor(), download=True)
    dataloader = Data.DataLoader(dataset=training_dataset,
                                          batch_size=BATCH_SIZE, shuffle=True, num_workers=2)
    # showSample(dataloader)
    test_data=torchvision.datasets.MNIST(root="./mnist",train=False,
                                        transform=torchvision.transforms.ToTensor(),download=False)
    test_dataloader=Data.DataLoader(
        dataset=test_data,batch_size=1000,shuffle=False,num_workers=2)
    testdata_iter=iter(test_dataloader)
    test_x,test_y=testdata_iter.next()
    test_x=test_x.view(-1,28,28)
    # 2. 網(wǎng)絡(luò)搭建
    net=RNN()
    # 3. 訓(xùn)練
    # 3. 網(wǎng)絡(luò)的訓(xùn)練（和之前CNN訓(xùn)練的代碼基本一樣）
    optimizer=torch.optim.Adam(net.parameters(),lr=0.001)
    loss_F=torch.nn.CrossEntropyLoss()
    for epoch in range(1): # 數(shù)據(jù)集只迭代一次
        for step, input_data in enumerate(dataloader):
            x,y=input_data
            pred=net(x.view(-1,28,28))
    
            loss=loss_F(pred,y) # 計(jì)算loss
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            if step%50==49: # 每50步洪规，計(jì)算精度
                with torch.no_grad():
                    test_pred=net(test_x)
                    prob=torch.nn.functional.softmax(test_pred,dim=1)
                    pred_cls=torch.argmax(prob,dim=1)
                    acc=(pred_cls==test_y).sum().numpy()/pred_cls.size()[0]
                    print(f"{epoch}-{step}: accuracy:{acc}")

LSTM輸出API

由上面代碼可以看到輸出為：output,(h_n,c_n)=self.rnn(x)

image

• output: 如果num_layer為3翅阵，則output只記錄最后一層(即调炬，第三層)的輸出

  • 對(duì)應(yīng)圖中向上的各個(gè)time_step的乔夯，也即output
  • 其size根據(jù)batch_first而不同库说〔卣叮可能是[batch_size, time_step, hidden_size]或[time_step, batch_size, hidden_size]

• h_n: 各個(gè)層的最后一個(gè)時(shí)步的隱含狀態(tài)h.

  • size為[num_layers,batch_size, hidden_size]
  • 對(duì)應(yīng)圖中向右的. 可以看出對(duì)于單層單向的LSTM他匪， 其h_n最后一層輸出h_n[-1,:,:]菇存，和output最后一個(gè)時(shí)步的輸出output[:,-1,:]相等。在示例代碼中print(h_n[-1,:,:].equal(output[:,-1,:]))會(huì)打印True

• c_n: 各個(gè)層的最后一個(gè)時(shí)步的隱含狀態(tài)C

  • c_n可以看成另一個(gè)隱含狀態(tài)邦蜜，size和h_n相同

參考文獻(xiàn)

• https://medium.com/mlreview/understanding-lstm-and-its-diagrams-37e2f46f1714