Pytorch 分類問題

train loss 不斷下降捍壤，test loss不斷下降续誉，說明網(wǎng)絡仍在學習;

train loss 不斷下降，test loss趨于不變扛吞，說明網(wǎng)絡過擬合;

train loss 趨于不變，test loss不斷下降荆责，說明數(shù)據(jù)集100%有問題;

train loss 趨于不變滥比，test loss趨于不變，說明學習遇到瓶頸做院，需要減小學習率或批量數(shù)目;

train loss 不斷上升盲泛，test loss不斷上升，說明網(wǎng)絡結構設計不當键耕，訓練超參數(shù)設置不當寺滚，數(shù)據(jù)集經(jīng)過清洗等問題。

今天跑程序屈雄，首先數(shù)據(jù)集選擇BPSK QPSK 8QAM 16QAM村视，測試集每種調(diào)制方式190個，驗證集每種調(diào)制方式10個

第一次訓練

首先用ResNet18酒奶，測試得到的結果

Accuracy of? BPSK : 20 %

Accuracy of? QPSK : 10 %

Accuracy of? 8QAM : 70 %

Accuracy of 16QAM : 20 %

分析蚁孔，有可能的原因是數(shù)據(jù)集是存在頻率為0的時候的圖像，也許頻率為零初的噪聲干擾了效果

第二次訓練

生成了一個VGG11模型惋嚎，保存在VGG11_Cyclic_BPSK_QPSK_8QAM_16QAM.pkl杠氢，結果為

Accuracy of? BPSK : 40 %

Accuracy of? QPSK : 40 %

Accuracy of? 8QAM : 30 %

Accuracy of 16QAM : 30 %

平均的正確率為35%

這個結果還可以提升，提升的方式有：

1. 增加iteration次數(shù)

2. 改變學習速率

3. 增加VGGNet的層數(shù)

首先看第一種情況另伍，讓iteration有10次鼻百，那么訓練階段的loss function最后是0.013，此時得到的模型為VGG11_Cyclic_BPSK_QPSK_8QAM_16QAM.pkl摆尝，得到的結果

Accuracy of? BPSK : 50 %

Accuracy of? QPSK : 50 %

Accuracy of? 8QAM : 10 %

Accuracy of 16QAM :? 0 %

平均正確率為27%温艇，降了，

batch size變?yōu)?结榄，那么正確率

Accuracy of? BPSK : 20 %

Accuracy of? QPSK : 40 %

Accuracy of? 8QAM :? 0 %

Accuracy of 16QAM : 10 %

那么是不是可以在學習的過程中降低學習速率中贝。

需要在不同的loss function的情況下，有不同的learning rate臼朗，這是因為

接下來在訓練數(shù)據(jù)的時候邻寿，發(fā)現(xiàn)loss function的值在不斷降低蝎土，一直能降到小于0.01，直到第500個數(shù)據(jù)绣否，loss function又會升到2以上誊涯。這是由于每一類的數(shù)據(jù)一共有500個，這說明數(shù)據(jù)不打亂數(shù)據(jù)的話蒜撮，分類器的loss function會受影響暴构。模型保存為VGG11_Cyclic_BPSK_QPSK_8QAM_16QAM_3.pkl

[10, 950] loss: 0.048

('Learning rate is', 1e-06)

[10,? 1000] loss: 1.798

('Learning rate is', 1e-05)

[10,? 1050] loss: 2.041

('Learning rate is', 1e-05)

[10,? 1100] loss: 1.369

('Learning rate is', 1e-05)

[10,? 1150] loss: 1.073

('Learning rate is', 1e-05)

[10,? 1200] loss: 0.836

('Learning rate is', 1e-05)

[10,? 1250] loss: 0.572

('Learning rate is', 1e-06)

[10,? 1300] loss: 0.315

('Learning rate is', 1e-06)

[10,? 1350] loss: 0.154

('Learning rate is', 1e-06)

[10,? 1400] loss: 0.086

('Learning rate is', 1e-06)

[10,? 1450] loss: 0.056

('Learning rate is', 1e-06)

[10,? 1500] loss: 2.172

('Learning rate is', 1e-05)

[10,? 1550] loss: 1.758

('Learning rate is', 1e-05)

[10,? 1600] loss: 1.339

('Learning rate is', 1e-05)

[10,? 1650] loss: 1.118

('Learning rate is', 1e-05)

[10,? 1700] loss: 0.873

('Learning rate is', 1e-05)

[10,? 1750] loss: 0.629

('Learning rate is', 1e-06)

[10,? 1800] loss: 0.362

('Learning rate is', 1e-06)

[10,? 1850] loss: 0.179

('Learning rate is', 1e-06)

[10,? 1900] loss: 0.100

('Learning rate is', 1e-06)

[10,? 1950] loss: 0.058

('Learning rate is', 1e-06)

所以接下來我打亂數(shù)據(jù)來看，打亂數(shù)據(jù)的方法是

trainloader = torch.utils.data.DataLoader(train_loader, batch_size=define_batch_size, shuffle=False)

里段磨，shuffle=True

現(xiàn)在出現(xiàn)的情況是取逾，同一種調(diào)制方式的圖片剛剛訓練好，或者過擬合了苹支，又被新的其他類別的信號給破壞了砾隅。

梯度下降，紅色的箭頭示意超出最優(yōu)點的梯度

第三次訓練

shuffle設置為True以后债蜜，發(fā)現(xiàn)training loss趨于不變

[1, 50] loss: 1.392

('Learning rate is', 1e-05)

[1,? 100] loss: 1.412

('Learning rate is', 1e-05)

[1,? 150] loss: 1.412

('Learning rate is', 1e-05)

[1,? 200] loss: 1.409

('Learning rate is', 1e-05)

[1,? 250] loss: 1.391

('Learning rate is', 1e-05)

[1,? 300] loss: 1.413

('Learning rate is', 1e-05)

[1,? 350] loss: 1.395

('Learning rate is', 1e-05)

[1,? 400] loss: 1.424

('Learning rate is', 1e-05)

[1,? 450] loss: 1.406

('Learning rate is', 1e-05)

[1,? 500] loss: 1.371

('Learning rate is', 1e-05)

[1,? 550] loss: 1.391

('Learning rate is', 1e-05)

[1,? 600] loss: 1.411

('Learning rate is', 1e-05)

[1,? 650] loss: 1.409

('Learning rate is', 1e-05)

[1,? 700] loss: 1.377

('Learning rate is', 1e-05)

[1,? 750] loss: 1.396

('Learning rate is', 1e-05)

[1,? 800] loss: 1.403

('Learning rate is', 1e-05)

[1,? 850] loss: 1.402

('Learning rate is', 1e-05)

[1,? 900] loss: 1.387

('Learning rate is', 1e-05)

[1,? 950] loss: 1.389

('Learning rate is', 1e-05)

[1,? 1000] loss: 1.394

('Learning rate is', 1e-05)

[1,? 1050] loss: 1.385

('Learning rate is', 1e-05)

[1,? 1100] loss: 1.349

('Learning rate is', 1e-05)

[1,? 1150] loss: 1.384

('Learning rate is', 1e-05)

[1,? 1200] loss: 1.393

('Learning rate is', 1e-05)

[1,? 1250] loss: 1.394

('Learning rate is', 1e-05)

[1,? 1300] loss: 1.385

('Learning rate is', 1e-05)

[1,? 1350] loss: 1.386

('Learning rate is', 1e-05)

[1,? 1400] loss: 1.388

('Learning rate is', 1e-05)

[1,? 1450] loss: 1.387

('Learning rate is', 1e-05)

[1,? 1500] loss: 1.375

('Learning rate is', 1e-05)

[1,? 1550] loss: 1.392

('Learning rate is', 1e-05)

[1,? 1600] loss: 1.411

('Learning rate is', 1e-05)

[1,? 1650] loss: 1.389

('Learning rate is', 1e-05)

[1,? 1700] loss: 1.388

('Learning rate is', 1e-05)

[1,? 1750] loss: 1.385

('Learning rate is', 1e-05)

[1,? 1800] loss: 1.400

('Learning rate is', 1e-05)

[1,? 1850] loss: 1.390

('Learning rate is', 1e-05)

[1,? 1900] loss: 1.408

('Learning rate is', 1e-05)

[1,? 1950] loss: 1.375

('Learning rate is', 1e-05)

看來learning rate調(diào)小一些

這一次learning rate調(diào)整的時候是這樣晴埂，首先在loss高于0.8的時候，learning rate是1e-7寻定，loss介于0.8到0.02之間的時候是1e-8儒洛，但是這次loss降到0.9到1之間的時候就降不下去了，這說明loss在小于1的時候狼速，學習速率還要再降一降琅锻。

這次得到的模型保存為

VGG11_Cyclic_BPSK_QPSK_8QAM_16QAM_4.pkl

Accuracy of BPSK : 40 %

Accuracy of? QPSK : 30 %

Accuracy of? 8QAM : 10 %

Accuracy of 16QAM : 40 %

第四次訓練

這次選擇的是ResNet50，設定是：

loss>1, learning rate=1e-7

0.7<loss<1, learning rate=1e-9

0.1<loss<0.7, learning rate=1e-12

loss<0.1, learning rate=1e-14

這次loss function開始在0.5左右徘徊唐含，比VGGNet效果好浅浮，既然loss function在0.5左右徘徊，

[10, 1450] loss: 0.481

('Learning rate is', 1e-12)

[10,? 1500] loss: 0.541

('Learning rate is', 1e-12)

[10,? 1550] loss: 0.501

('Learning rate is', 1e-12)

[10,? 1600] loss: 0.576

('Learning rate is', 1e-12)

[10,? 1650] loss: 0.543

('Learning rate is', 1e-12)

[10,? 1700] loss: 0.473

('Learning rate is', 1e-12)

[10,? 1750] loss: 0.562

('Learning rate is', 1e-12)

[10,? 1800] loss: 0.579

('Learning rate is', 1e-12)

[10,? 1850] loss: 0.504

('Learning rate is', 1e-12)

[10,? 1900] loss: 0.490

('Learning rate is', 1e-12)

[10,? 1950] loss: 0.451

('Learning rate is', 1e-12)

那么就要在0.5左右再設置一個門限捷枯，當loss 低于0.5的時候滚秩，learning rate還要再降，而且0.7有點低淮捆，可以設定為0.8郁油，保存的模型是

現(xiàn)在的設定是

######################################################################

# Define the learning rate

change_learning_rate_mark1 = 2.2

learning_rate1 = 1e-4

change_learning_rate_mark2 = 2.0

learning_rate2 = 9e-5

change_learning_rate_mark3 = 1.8

learning_rate3 = 8e-5

change_learning_rate_mark4 = 1.6

learning_rate4 = 7e-5

change_learning_rate_mark5 = 1.4

learning_rate5 = 6e-5

change_learning_rate_mark6 = 1.2

learning_rate6 = 5e-5

change_learning_rate_mark7 = 1

learning_rate7 = 4e-5

change_learning_rate_mark8 = 0.8

learning_rate8 = 3e-5

change_learning_rate_mark9 = 0.6

learning_rate9 = 2e-5

change_learning_rate_mark10 = 0.4

learning_rate10 = 1e-5

change_learning_rate_mark11 = 0.2

learning_rate11 = 9e-6

change_learning_rate_mark12 = 0.08

learning_rate12 = 8e-6

learning_rate13 = 7e-6

stop_loss_function = 0.001

# ==========================================================

最后loss function到了0.3左右

('Learning rate is', 9e-06)

[10,? 1600] loss: 0.374

('Learning rate is', 9e-06)

[10,? 1650] loss: 0.451

('Learning rate is', 1e-05)

[10,? 1700] loss: 0.478

('Learning rate is', 1e-05)

[10,? 1750] loss: 0.417

('Learning rate is', 1e-05)

[10,? 1800] loss: 0.344

('Learning rate is', 9e-06)

[10,? 1850] loss: 0.301

('Learning rate is', 9e-06)

[10,? 1900] loss: 0.354

('Learning rate is', 9e-06)

[10,? 1950] loss: 0.317

('Learning rate is', 9e-06)

Finished Training

得到結果

Accuracy of BPSK : 47 %

Accuracy of? QPSK : 38 %

Accuracy of? 8QAM : 42 %

Accuracy of 16QAM : 52 %

第五次訓練

接下來試一試ResNet152，這次的learning rate按照x^{-2}來遞減

######################################################################

# Define the learning rate

change_learning_rate_mark1 =2.2

learning_rate1 =1e-3

change_learning_rate_mark2 =2.0

learning_rate2 =5e-4

change_learning_rate_mark3 =1.8

learning_rate3 =1.1e-4

change_learning_rate_mark4 =1.6

learning_rate4 =0.0625e-5

change_learning_rate_mark5 =1.4

learning_rate5 =0.04e-5

change_learning_rate_mark6 =1.2

learning_rate6 =0.0278e-5

change_learning_rate_mark7 =1

learning_rate7 =0.0204e-5

change_learning_rate_mark8 =0.8

learning_rate8 =0.0156e-5

change_learning_rate_mark9 =0.6

learning_rate9 =0.0123e-5

change_learning_rate_mark10 =0.4

learning_rate10 =1e-7

change_learning_rate_mark11 =0.2

learning_rate11 =0.0083e-6

change_learning_rate_mark12 =0.08

learning_rate12 =0.0069e-6

learning_rate13 =0.0059e-6

stop_loss_function =0.001

# ==========================================================

在0.8的地方還是降不下去了攀痊，所以loss改成x^{-4}來遞減桐腌，得到的結果

[10, 1450] loss: 1.561