1.3 邏輯回歸

將線性回歸的模型改一改荷并，就可以用于二分類。邏輯回歸擬合樣本屬于某個(gè)分類目胡，也就是樣本為正樣本的概率锯七。

操作步驟

導(dǎo)入所需的包。

import tensorflow as tf
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
import sklearn.datasets as ds
import sklearn.model_selection as ms

導(dǎo)入數(shù)據(jù)誉己，并進(jìn)行預(yù)處理眉尸。我們使用鳶尾花數(shù)據(jù)集所有樣本，根據(jù)萼片長(zhǎng)度和花瓣長(zhǎng)度預(yù)測(cè)樣本是不是山鳶尾（第一種）巫延。

iris = ds.load_iris()

x_ = iris.data[:, [0, 2]]
y_ = (iris.target == 0).astype(int)
y_ = np.expand_dims(y_ , 1)

x_train, x_test, y_train, y_test = \
    ms.train_test_split(x_, y_, train_size=0.7, test_size=0.3)

定義超參數(shù)效五。

n_input = 2
n_epoch = 2000
lr = 0.05
threshold = 0.5

搭建模型炉峰。

x = tf.placeholder(tf.float64, [None, n_input])
y = tf.placeholder(tf.float64, [None, 1])
w = tf.Variable(np.random.rand(n_input, 1))
b = tf.Variable(np.random.rand(1, 1))
z = x @ w + b
a = tf.sigmoid(z)

定義損失疼阔、優(yōu)化操作戒劫、和準(zhǔn)確率度量指標(biāo)半夷。分類問(wèn)題有很多指標(biāo)，這里只展示一種迅细。

我們使用交叉熵?fù)p失函數(shù)巫橄，如下。

它的意思是茵典，對(duì)于正樣本湘换，y 為 1，損失變?yōu)?code>-log(a)统阿，輸出會(huì)盡可能接近一彩倚。對(duì)于負(fù)樣本，y為 0扶平，損失變?yōu)?code>-log(1 - a)帆离，輸出會(huì)盡可能接近零〗岢危總之哥谷，它使輸出盡可能接近真實(shí)標(biāo)簽。

loss = - tf.reduce_mean(y * tf.log(a) + (1 - y) * tf.log(1 - a))
op = tf.train.AdamOptimizer(lr).minimize(loss)

y_hat = tf.to_double(a > threshold)
acc = tf.reduce_mean(tf.to_double(tf.equal(y_hat, y)))

使用訓(xùn)練集訓(xùn)練模型麻献。

losses = []
accs = []

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    saver = tf.train.Saver(max_to_keep=1)
    
    for e in range(n_epoch):
        _, loss_ = sess.run([op, loss], feed_dict={x: x_train, y: y_train})
        losses.append(loss_)

使用測(cè)試集計(jì)算準(zhǔn)確率们妥。

        acc_ = sess.run(acc, feed_dict={x: x_test, y: y_test})
        accs.append(acc_)

每一百步打印損失和度量值。

        if e % 100 == 0:
            print(f'epoch: {e}, loss: {loss_}, acc: {acc_}')
            saver.save(sess,'logit/logit', global_step=e)

得到?jīng)Q策邊界：

    x_plt = x_[:, 0]
    y_plt = x_[:, 1]
    c_plt = y_.ravel()
    x_min = x_plt.min() - 1
    x_max = x_plt.max() + 1
    y_min = y_plt.min() - 1
    y_max = y_plt.max() + 1
    x_rng = np.arange(x_min, x_max, 0.05)
    y_rng = np.arange(y_min, y_max, 0.05)
    x_rng, y_rng = np.meshgrid(x_rng, y_rng)
    model_input = np.asarray([x_rng.ravel(), y_rng.ravel()]).T
    model_output = sess.run(y_hat, feed_dict={x: model_input}).astype(int)
    c_rng = model_output.reshape(x_rng.shape)

輸出：

epoch: 0, loss: 3.935746371309244, acc: 0.3333333333333333
epoch: 100, loss: 0.1969325408656252, acc: 1.0
epoch: 200, loss: 0.08548362243852041, acc: 1.0
epoch: 300, loss: 0.050833687966014396, acc: 1.0
epoch: 400, loss: 0.034929315249291375, acc: 1.0
epoch: 500, loss: 0.026013692651528184, acc: 1.0
epoch: 600, loss: 0.02038864243607467, acc: 1.0
epoch: 700, loss: 0.016552042129938136, acc: 1.0
epoch: 800, loss: 0.013786692432697542, acc: 1.0
epoch: 900, loss: 0.011709709551073783, acc: 1.0
epoch: 1000, loss: 0.010099234422592073, acc: 1.0
epoch: 1100, loss: 0.008818382202721829, acc: 1.0
epoch: 1200, loss: 0.007778392815694136, acc: 1.0
epoch: 1300, loss: 0.0069193419951217704, acc: 1.0
epoch: 1400, loss: 0.0061993983430654875, acc: 1.0
epoch: 1500, loss: 0.00558852696047961, acc: 1.0
epoch: 1600, loss: 0.005064638072189167, acc: 1.0
epoch: 1700, loss: 0.00461114435393481, acc: 1.0
epoch: 1800, loss: 0.004215362417896155, acc: 1.0
epoch: 1900, loss: 0.003867437954560204, acc: 1.0

繪制整個(gè)數(shù)據(jù)集以及決策邊界赎瑰。

plt.figure()
cmap = mpl.colors.ListedColormap(['r', 'b'])
plt.scatter(x_plt, y_plt, c=c_plt, cmap=cmap)
plt.contourf(x_rng, y_rng, c_rng, alpha=0.2, linewidth=5, cmap=cmap)
plt.title('Data and Model')
plt.xlabel('Petal Length (cm)')
plt.ylabel('Sepal Length (cm)')
plt.show()

https://github.com/wizardforcel/how2tf/raw/master/img/1-3-1.png

繪制訓(xùn)練集上的損失王悍。

plt.figure()
plt.plot(losses)
plt.title('Loss on Training Set')
plt.xlabel('#epoch')
plt.ylabel('Cross Entropy')
plt.show()

https://github.com/wizardforcel/how2tf/raw/master/img/1-3-2.png

繪制測(cè)試集上的準(zhǔn)確率。

plt.figure()
plt.plot(accs)
plt.title('Accurary on Testing Set')
plt.xlabel('#epoch')
plt.ylabel('Accurary')
plt.show()

https://github.com/wizardforcel/how2tf/raw/master/img/1-3-3.png

擴(kuò)展閱讀

• 斯坦福 CS229 筆記：六餐曼、邏輯回歸