說明
此代碼在matlab上搭建了簡單的生成對抗性網(wǎng)絡(luò)聋亡，用來生成手寫數(shù)字圖像汰蓉。
網(wǎng)絡(luò)中生成器和鑒別器的隱藏層均為2層珊佣，且都是全連接層康铭，是一個比較簡單的網(wǎng)絡(luò)結(jié)構(gòu)惯退。主要用來說明怎么在matlab上搭建GAN（Generative Adversarial Net）網(wǎng)絡(luò)。
網(wǎng)絡(luò)模型
如圖1所示从藤，是生成器網(wǎng)絡(luò)模型催跪，一個輸入層，兩個全連接層夷野。輸入的數(shù)據(jù)是100×1的噪聲懊蒸，輸出是784×1的向量，將輸出進行reshape之后悯搔，就可以得到一張28×28的手寫數(shù)字圖像榛鼎。
圖1 生成器
如圖2所示，是將生成器和鑒別器連接起來之后的模型。這里主要想說明一點者娱，在進行網(wǎng)絡(luò)參數(shù)更新的時候，為了得到生成器參數(shù)的偏導(dǎo)數(shù)苏揣，
bp過程需要鑒別器黄鳍，再傳到生成器。
到這里就產(chǎn)生了一個疑問：不是說更新生成器的時候平匈，鑒別網(wǎng)絡(luò)的參數(shù)需要固定住不變嗎框沟，如果bp過程需要經(jīng)過鑒別網(wǎng)絡(luò)，那應(yīng)該怎么保持鑒別網(wǎng)絡(luò)的參數(shù)不變呢增炭？
其實bp的時候忍燥，只是算出來了各個網(wǎng)絡(luò)層參數(shù)對于loss的偏導(dǎo)數(shù)，在求生成器的參數(shù)的偏導(dǎo)數(shù)的時候隙姿，鑒別網(wǎng)絡(luò)的參數(shù)的偏導(dǎo)數(shù)也被求出來了梅垄。但是求出來了偏導(dǎo)數(shù)，不一定就要對網(wǎng)絡(luò)進行更新输玷。也就是求出來了網(wǎng)絡(luò)loss對生成器和鑒別器的偏導(dǎo)數(shù)队丝，但是只使用到了生成器的偏導(dǎo)數(shù)來更新生成器。
圖2 將generator和discriminator看成一個整體
More
這個是我寫的欲鹏、在
matlab上實現(xiàn)GAN網(wǎng)絡(luò)的另外一份代碼机久，里面的網(wǎng)絡(luò)模型使用到了卷積、反卷積等赔嚎。
GAN 網(wǎng)絡(luò)matlab實現(xiàn)在
matlab平臺上膘盖，使用MatConvNet搭建GAN網(wǎng)絡(luò)的例子
使用MatConvNet搭建GAN網(wǎng)絡(luò)mnist_uint8.mat 下載地址
實例
實例1
??????代碼在github：gan_adam.m
這里使用上面提到的網(wǎng)絡(luò)結(jié)構(gòu)來生成手寫數(shù)字圖片，使用到了Adam算法作為優(yōu)化器來更新GAN網(wǎng)絡(luò)尤误。
clear;
clc;
% -----------加載數(shù)據(jù)
load('mnist_uint8', 'train_x');
train_x = double(reshape(train_x, 60000, 28, 28))/255;
train_x = permute(train_x,[1,3,2]);
train_x = reshape(train_x, 60000, 784);
% -----------------定義模型
generator = nnsetup([100, 512, 784]);
discriminator = nnsetup([784, 512, 1]);
% -----------開始訓(xùn)練
batch_size = 60;
epoch = 100;
images_num = 60000;
batch_num = ceil(images_num / batch_size);
learning_rate = 0.001;
for e=1:epoch
kk = randperm(images_num);
for t=1:batch_num
% 準(zhǔn)備數(shù)據(jù)
images_real = train_x(kk((t - 1) * batch_size + 1:t * batch_size), :, :);
noise = unifrnd(-1, 1, batch_size, 100);
% 開始訓(xùn)練
% -----------更新generator侠畔，固定discriminator
generator = nnff(generator, noise);
images_fake = generator.layers{generator.layers_count}.a;
discriminator = nnff(discriminator, images_fake);
logits_fake = discriminator.layers{discriminator.layers_count}.z;
discriminator = nnbp_d(discriminator, logits_fake, ones(batch_size, 1));
generator = nnbp_g(generator, discriminator);
generator = nnapplygrade(generator, learning_rate);
% -----------更新discriminator，固定generator
generator = nnff(generator, noise);
images_fake = generator.layers{generator.layers_count}.a;
images = [images_fake;images_real];
discriminator = nnff(discriminator, images);
logits = discriminator.layers{discriminator.layers_count}.z;
labels = [zeros(batch_size,1);ones(batch_size,1)];
discriminator = nnbp_d(discriminator, logits, labels);
discriminator = nnapplygrade(discriminator, learning_rate);
% ----------------輸出loss
if t == batch_num
c_loss = sigmoid_cross_entropy(logits(1:batch_size), ones(batch_size, 1));
d_loss = sigmoid_cross_entropy(logits, labels);
fprintf('c_loss:"%f",d_loss:"%f"\n',c_loss, d_loss);
end
if t == batch_num
path = ['./pics/epoch_',int2str(e),'_t_',int2str(t),'.png'];
save_images(images_fake, [4, 4], path);
fprintf('save_sample:%s\n', path);
end
end
end
% sigmoid激活函數(shù)
function output = sigmoid(x)
output =1./(1+exp(-x));
end
% relu
function output = relu(x)
output = max(x, 0);
end
% relu對x的導(dǎo)數(shù)
function output = delta_relu(x)
output = max(x,0);
output(output>0) = 1;
end
% 交叉熵?fù)p失函數(shù)袄膏，此處的logits是未經(jīng)過sigmoid激活的
% https://www.tensorflow.org/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits
function result = sigmoid_cross_entropy(logits, labels)
result = max(logits, 0) - logits .* labels + log(1 + exp(-abs(logits)));
result = mean(result);
end
% sigmoid_cross_entropy對logits的導(dǎo)數(shù)践图，此處的logits是未經(jīng)過sigmoid激活的
function result = delta_sigmoid_cross_entropy(logits, labels)
temp1 = max(logits, 0);
temp1(temp1>0) = 1;
temp2 = logits;
temp2(temp2>0) = -1;
temp2(temp2<0) = 1;
result = temp1 - labels + exp(-abs(logits))./(1+exp(-abs(logits))) .* temp2;
end
% 根據(jù)所給的結(jié)構(gòu)建立網(wǎng)絡(luò)
function nn = nnsetup(architecture)
nn.architecture = architecture;
nn.layers_count = numel(nn.architecture);
% t,beta1,beta2,epsilon,nn.layers{i}.w_m,nn.layers{i}.w_v,nn.layers{i}.b_m,nn.layers{i}.b_v是應(yīng)用adam算法更新網(wǎng)絡(luò)所需的變量
nn.t = 0;
nn.beta1 = 0.9;
nn.beta2 = 0.999;
nn.epsilon = 10^(-8);
% 假設(shè)結(jié)構(gòu)為[100, 512, 784]，則有3層沉馆，輸入層100码党，兩個隱藏層：100*512，512*784, 輸出為最后一層的a值（激活值）
for i = 2 : nn.layers_count
nn.layers{i}.w = normrnd(0, 0.02, nn.architecture(i-1), nn.architecture(i));
nn.layers{i}.b = normrnd(0, 0.02, 1, nn.architecture(i));
nn.layers{i}.w_m = 0;
nn.layers{i}.w_v = 0;
nn.layers{i}.b_m = 0;
nn.layers{i}.b_v = 0;
end
end
% 前向傳遞
function nn = nnff(nn, x)
nn.layers{1}.a = x;
for i = 2 : nn.layers_count
input = nn.layers{i-1}.a;
w = nn.layers{i}.w;
b = nn.layers{i}.b;
nn.layers{i}.z = input*w + repmat(b, size(input, 1), 1);
if i ~= nn.layers_count
nn.layers{i}.a = relu(nn.layers{i}.z);
else
nn.layers{i}.a = sigmoid(nn.layers{i}.z);
end
end
end
% discriminator的bp斥黑，下面的bp涉及到對各個參數(shù)的求導(dǎo)
% 如果更改網(wǎng)絡(luò)結(jié)構(gòu)（激活函數(shù)等）則涉及到bp的更改揖盘，更改weights，biases的個數(shù)則不需要更改bp
% 為了更新w,b锌奴，就是要求最終的loss對w兽狭，b的偏導(dǎo)數(shù)，殘差就是在求w，b偏導(dǎo)數(shù)的中間計算過程的結(jié)果
function nn = nnbp_d(nn, y_h, y)
% d表示殘差箕慧，殘差就是最終的loss對各層未激活值（z）的偏導(dǎo)服球，偏導(dǎo)數(shù)的計算需要采用鏈?zhǔn)角髮?dǎo)法則-自己手動推出來
n = nn.layers_count;
% 最后一層的殘差
nn.layers{n}.d = delta_sigmoid_cross_entropy(y_h, y);
for i = n-1:-1:2
d = nn.layers{i+1}.d;
w = nn.layers{i+1}.w;
z = nn.layers{i}.z;
% 每一層的殘差是對每一層的未激活值求偏導(dǎo)數(shù)，所以是后一層的殘差乘上w,再乘上對激活值對未激活值的偏導(dǎo)數(shù)
nn.layers{i}.d = d*w' .* delta_relu(z);
end
% 求出各層的殘差之后颠焦，就可以根據(jù)殘差求出最終loss對weights和biases的偏導(dǎo)數(shù)
for i = 2:n
d = nn.layers{i}.d;
a = nn.layers{i-1}.a;
% dw是對每層的weights進行偏導(dǎo)數(shù)的求解
nn.layers{i}.dw = a'*d / size(d, 1);
nn.layers{i}.db = mean(d, 1);
end
end
% generator的bp
function g_net = nnbp_g(g_net, d_net)
n = g_net.layers_count;
a = g_net.layers{n}.a;
% generator的loss是由label_fake得到的斩熊，(images_fake過discriminator得到label_fake)
% 對g進行bp的時候，可以將g和d看成是一個整體
% g最后一層的殘差等于d第2層的殘差乘上(a .* (a_o))
g_net.layers{n}.d = d_net.layers{2}.d * d_net.layers{2}.w' .* (a .* (1-a));
for i = n-1:-1:2
d = g_net.layers{i+1}.d;
w = g_net.layers{i+1}.w;
z = g_net.layers{i}.z;
% 每一層的殘差是對每一層的未激活值求偏導(dǎo)數(shù)伐庭，所以是后一層的殘差乘上w,再乘上對激活值對未激活值的偏導(dǎo)數(shù)
g_net.layers{i}.d = d*w' .* delta_relu(z);
end
% 求出各層的殘差之后粉渠，就可以根據(jù)殘差求出最終loss對weights和biases的偏導(dǎo)數(shù)
for i = 2:n
d = g_net.layers{i}.d;
a = g_net.layers{i-1}.a;
% dw是對每層的weights進行偏導(dǎo)數(shù)的求解
g_net.layers{i}.dw = a'*d / size(d, 1);
g_net.layers{i}.db = mean(d, 1);
end
end
% 應(yīng)用梯度
% 使用adam算法更新變量，可以參考：
% https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer
function nn = nnapplygrade(nn, learning_rate)
n = nn.layers_count;
nn.t = nn.t+1;
beta1 = nn.beta1;
beta2 = nn.beta2;
lr = learning_rate * sqrt(1-nn.beta2^nn.t) / (1-nn.beta1^nn.t);
for i = 2:n
dw = nn.layers{i}.dw;
db = nn.layers{i}.db;
% 下面的6行代碼是使用adam更新weights與biases
nn.layers{i}.w_m = beta1 * nn.layers{i}.w_m + (1-beta1) * dw;
nn.layers{i}.w_v = beta2 * nn.layers{i}.w_v + (1-beta2) * (dw.*dw);
nn.layers{i}.w = nn.layers{i}.w - lr * nn.layers{i}.w_m ./ (sqrt(nn.layers{i}.w_v) + nn.epsilon);
nn.layers{i}.b_m = beta1 * nn.layers{i}.b_m + (1-beta1) * db;
nn.layers{i}.b_v = beta2 * nn.layers{i}.b_v + (1-beta2) * (db.*db);
nn.layers{i}.b = nn.layers{i}.b - lr * nn.layers{i}.b_m ./ (sqrt(nn.layers{i}.b_v) + nn.epsilon);
end
end
% 保存圖片圾另，便于觀察generator生成的images_fake
function save_images(images, count, path)
n = size(images, 1);
row = count(1);
col = count(2);
I = zeros(row*28, col*28);
for i = 1:row
for j = 1:col
r_s = (i-1)*28+1;
c_s = (j-1)*28+1;
index = (i-1)*col + j;
pic = reshape(images(index, :), 28, 28);
I(r_s:r_s+27, c_s:c_s+27) = pic;
end
end
imwrite(I, path);
end
結(jié)果
epoch_5_t_1000.png
epoch_13_t_1000.png
實例2霸株，Mini-batch Gradient Descent
??????代碼在github：gan_mbgd.m
這里使用的網(wǎng)絡(luò)結(jié)構(gòu)與實例1的一樣，只是換了一種優(yōu)化器集乔。使用Mini-batch Gradient Descent算法更新GAN網(wǎng)絡(luò)去件，下面是部分代碼。
% 應(yīng)用梯度
function nn = nnapplygrade(nn, learning_rate)
n = nn.layers_count;
for i = 2:n
dw = nn.layers{i}.dw;
db = nn.layers{i}.db;
nn.layers{i}.w = nn.layers{i}.w - learning_rate * dw;
nn.layers{i}.b = nn.layers{i}.b - learning_rate * db;
end
end
結(jié)果：
epoch_11_t_1000.png
epoch_13_t_1000.png
