Sec. 3: THE LINEAR EXPLANATION OF ADVERSARIAL EXAMPLES
Consider the dot product between a weight vector and an adversarial example : The adversarial perturbation causes the activation to grow by . We can maximize this increase subject to the max norm constraint on by assigning .
這里漱牵，如果，怎能滿(mǎn)足黔夭？從下文來(lái)看應(yīng)該是.
If has dimensions and the average magnitude of an element of the weight vector is , then the activation will grow by .
注：什么是Max norm constraints钦椭？下面是來(lái)自CS231n課程的答案:
Max norm constraints. Another form of regularization is to enforce an absolute upper bound on the magnitude of the weight vector for every neuron and use projected gradient descent to enforce the constraint. In practice, this corresponds to performing the parameter update as normal, and then enforcing the constraint by clamping the weight vector of every neuron to satisfy . Typical values of are on orders of 3 or 4. Some people report improvements when using this form of regularization. One of its appealing properties is that network cannot “explode” even when the learning rates are set too high because the updates are always bounded.
Sec 4: LINEAR PERTURBATION OF NON-LINEAR MODELS
Let be the parameters of a model, the input to the model, the targets associated with (for machine learning tasks that have targets) and be the cost used to train the neural network. We can linearize the cost function around the current value of , obtaining an optimal max-norm constrained perturbation of 這里，有如下幾個(gè)問(wèn)題：
，怎能滿(mǎn)足
黔夭？從下文來(lái)看應(yīng)該是
.
has
dimensions and the average magnitude of an element of the weight vector is
, then the activation will grow by
.
由于須滿(mǎn)足
的最大值最大只能為
了一膨；同時(shí)，為了最大化
，需要保證
of every neuron to satisfy
. Typical values of
are on orders of 3 or 4. Some people report improvements when using this form of regularization. One of its appealing properties is that network cannot “explode” even when the learning rates are set too high because the updates are always bounded.
求導(dǎo)呢碑诉？因?yàn)槲覀円獢_動(dòng)的是
彪腔，這里為什么是取
的
