粒子群優(yōu)化算法（PSO）是一種基于種群的全局優(yōu)化算法，用于尋找函數(shù)的全局最優(yōu)解饥脑。它基于模擬鳥群或魚群尋找食物的行為的思想阿蝶。群體中的每個粒子代表一個候選解，并根據(jù)自身的經(jīng)驗和相鄰的經(jīng)驗更新其在搜索空間中的位置柬甥。表現(xiàn)最佳的粒子饮六，即全局最佳粒子，用于指導整個群體的搜索苛蒲。
反向傳播算法（BP）是一種用于訓練人工神經(jīng)網(wǎng)絡(luò)的監(jiān)督學習算法卤橄。它是一種基于梯度的優(yōu)化方法，用于調(diào)整網(wǎng)絡(luò)的權(quán)重臂外，以最小化預(yù)測輸出與實際目標之間的誤差窟扑。誤差被計算并通過網(wǎng)絡(luò)反向傳播喇颁，從而使得權(quán)重可以在下一次迭代中被更新，以減少誤差嚎货。BP被用于各種應(yīng)用橘霎，例如圖像識別、自然語言處理和控制系統(tǒng)殖属。
下面將使用sk包的PSO來優(yōu)化BP神經(jīng)網(wǎng)絡(luò)姐叁。
1. 加載包
import numpy as np
from sko.PSO import PSO
import matplotlib.pyplot as plt
2.定義激活函數(shù)
import numpy as np
from sko.PSO import PSO
import matplotlib.pyplot as plt
3.定義BP神經(jīng)網(wǎng)絡(luò)
class BP_Neural_Network:
def __init__(self, input_size, hidden_size, output_size):
self.input_size = input_size
self.hidden_size = hidden_size
self.output_size = output_size
# Initialize the weight matrices
self.W1 = np.random.randn(self.input_size, self.hidden_size)
self.W2 = np.random.randn(self.hidden_size, self.output_size)
def forward(self, X):
self.z = np.dot(X, self.W1)
self.z2 = sigmoid(self.z)
self.z3 = np.dot(self.z2, self.W2)
o = sigmoid(self.z3)
return o
def backward(self, X, y, o):
self.o_error = y - o
self.o_delta = self.o_error * sigmoid_derivative(o)
self.z2_error = self.o_delta.dot(self.W2.T)
self.z2_delta = self.z2_error * sigmoid_derivative(self.z2)
self.W1 += X.T.dot(self.z2_delta)
self.W2 += self.z2.T.dot(self.o_delta)
def train(self, X, y):
o = self.forward(X)
self.backward(X, y, o)
def predict(self, X):
return self.forward(X)
4.定義優(yōu)化目標
def obj_func(weights):
nn.W1 = weights[0:3 * 2].reshape((3, 2))
nn.W2 = weights[3 * 2:].reshape((2, 1))
y_pred = nn.predict(X_train)
return np.mean(np.power(y_train - y_pred, 2))
5.初始化設(shè)置
nn = BP_Neural_Network(input_size=3, hidden_size=2, output_size=1)
X_train = np.array([[0, 0, 1], [1, 1, 1], [1, 0, 1], [0, 1, 1]])
y_train = np.array([[0], [1], [1], [0]])
pso = PSO(func=obj_func, n_dim=3 * 2 + 2, lb=[-10, -10, -10, -10, -10, -10, -10, -10], ub=[10, 10, 10, 10, 10, 10, 10, 10], pop=30, max_iter=100)
6.運行PSO獲取最佳權(quán)重偏置
pso.run()
nn.W1 = pso.gbest_x[0:3 * 2].reshape((3, 2))
nn.W2 = pso.gbest_x[3 * 2:].reshape((2, 1))
7.預(yù)測
nn.W1 = pso.gbest_x[0:3 * 2].reshape((3, 2))
nn.W2 = pso.gbest_x[3 * 2:].reshape((2, 1))
Predicted outputs: [[5.19002550e-05]
[9.99948100e-01]
[9.99948101e-01]
[5.19000440e-05]]
8.畫圖展示
plt.plot(pso.gbest_y_hist)
plt.xlabel('Iteration')
plt.ylabel('Mean Squared Error')
plt.show()
E1D92759A61042E395925A86FCC461F3.jpg
