Dense參數(shù)介紹
Dense調(diào)用命令：
class Dense(
units,
activation=None,
use_bias=True,
kernel_initializer='glorot_uniform',
bias_initializer='zeros',
kernel_regularizer=None,
bias_regularizer=None,
activity_regularizer=None,
kernel_constraint=None,
bias_constraint=None,
**kwargs)
我們常用到的參數(shù)有
- units:設(shè)置該層節(jié)點(diǎn)數(shù)，也可以看成對(duì)下一層的輸出披坏。
- activation：激活函數(shù)，在這一層輸出的時(shí)候是否需要激活函數(shù)
- use_bias：偏置碎节，默認(rèn)帶有偏置。
其他參數(shù)的含義：
- kernel_initializer:權(quán)重初始化方法
- bias_initializer:偏置值初始化方法
- kernel_regularizer:權(quán)重規(guī)范化函數(shù)
- bias_regularizer:偏置值規(guī)范化方法
- activity_regularizer:輸出的規(guī)范化方法
- kernel_constraint:權(quán)重變化限制函數(shù)
- bias_constraint:偏置值變化限制函數(shù)
units用法
在參數(shù)介紹的時(shí)候就說了units是該層節(jié)點(diǎn)數(shù)，只能是正整數(shù)魁索。用法如下：
model.add(layers.Dense(units=10))
這條命令只是定義了這一層有10個(gè)節(jié)點(diǎn)迈勋，沒有激活函數(shù)，但是有偏置（偏置是默認(rèn)的）瘫想。但是在第一層需要注意一下仗阅，第一層需要定義輸入數(shù)據(jù)形狀。（x是輸入的數(shù)據(jù)）
model.add(layers.Dense(units=10,input_shape = (x.shape[1],)))
activation用法
這個(gè)參數(shù)是用于做非線性變換的国夜。如果是單層網(wǎng)絡(luò)减噪，那么它輸出就是A*X+b的線性輸出。而activation很多這里介紹一些常用的：
- relu：
keras.activations.relu(x, alpha=0.0, max_value=None, threshold=0.0)整流線性單元车吹。使用默認(rèn)值時(shí)筹裕，它返回逐元素的 max(x, 0)否則，它遵循：- if x >= max_value：f(x) = max_value
- elif threshold <= x < max_value：f(x) = x
- else：f(x) = alpha * (x - threshold)
relu激活函數(shù)
model.add(layers.Dense(units=10,activation='relu'))
-
sigmoid：Sigmoid函數(shù)是一個(gè)在生物學(xué)中常見的S型函數(shù)礼搁，將變量映射到0,1之間饶碘。logistic回歸就是用sigmoid函數(shù)輸出概率值。
sigmoid激活函數(shù)
model.add(layers.Dense(units=10,activation='sigmoid'))
-
tanh：雙曲正切激活函數(shù)馒吴，這個(gè)與sigmoid的區(qū)別是壓縮到了[-1扎运，1].
image.png
model.add(layers.Dense(units=10,activation='tanh'))
除了上面的三種激活函數(shù)還有：
- elu：指數(shù)線性單元。
- selu：可伸縮的指數(shù)線性單元（SELU）饮戳。
- softplus：Softplus 激活函數(shù)豪治。
- softsign：Softsign 激活函數(shù)。
- hard_sigmoid：Hard sigmoid 激活函數(shù)扯罐，計(jì)算速度比 sigmoid 激活函數(shù)更快负拟。
- exponential：自然數(shù)指數(shù)激活函數(shù)。
- linear：線性激活函數(shù)（即不做任何改變）
這個(gè)是keras.activation的中文API：Keras 中文文檔
具體的各個(gè)激活函數(shù)的用法可以參考一下歹河。下面是activation在keras的源代碼有興趣可以研究一下
"""Built-in activation functions.
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import six
import warnings
from . import backend as K
from .utils.generic_utils import deserialize_keras_object
from .engine import Layer
def softmax(x, axis=-1):
"""Softmax activation function.
# Arguments
x: Input tensor.
axis: Integer, axis along which the softmax normalization is applied.
# Returns
Tensor, output of softmax transformation.
# Raises
ValueError: In case `dim(x) == 1`.
"""
ndim = K.ndim(x)
if ndim == 2:
return K.softmax(x)
elif ndim > 2:
e = K.exp(x - K.max(x, axis=axis, keepdims=True))
s = K.sum(e, axis=axis, keepdims=True)
return e / s
else:
raise ValueError('Cannot apply softmax to a tensor that is 1D. '
'Received input: %s' % x)
def elu(x, alpha=1.0):
"""Exponential linear unit.
# Arguments
x: Input tensor.
alpha: A scalar, slope of negative section.
# Returns
The exponential linear activation: `x` if `x > 0` and
`alpha * (exp(x)-1)` if `x < 0`.
# References
- [Fast and Accurate Deep Network Learning by Exponential
Linear Units (ELUs)](https://arxiv.org/abs/1511.07289)
"""
return K.elu(x, alpha)
def selu(x):
"""Scaled Exponential Linear Unit (SELU).
SELU is equal to: `scale * elu(x, alpha)`, where alpha and scale
are predefined constants. The values of `alpha` and `scale` are
chosen so that the mean and variance of the inputs are preserved
between two consecutive layers as long as the weights are initialized
correctly (see `lecun_normal` initialization) and the number of inputs
is "large enough" (see references for more information).
# Arguments
x: A tensor or variable to compute the activation function for.
# Returns
The scaled exponential unit activation: `scale * elu(x, alpha)`.
# Note
- To be used together with the initialization "lecun_normal".
- To be used together with the dropout variant "AlphaDropout".
# References
- [Self-Normalizing Neural Networks](https://arxiv.org/abs/1706.02515)
"""
alpha = 1.6732632423543772848170429916717
scale = 1.0507009873554804934193349852946
return scale * K.elu(x, alpha)
def softplus(x):
"""Softplus activation function.
# Arguments
x: Input tensor.
# Returns
The softplus activation: `log(exp(x) + 1)`.
"""
return K.softplus(x)
def softsign(x):
"""Softsign activation function.
# Arguments
x: Input tensor.
# Returns
The softsign activation: `x / (abs(x) + 1)`.
"""
return K.softsign(x)
def relu(x, alpha=0., max_value=None, threshold=0.):
"""Rectified Linear Unit.
With default values, it returns element-wise `max(x, 0)`.
Otherwise, it follows:
`f(x) = max_value` for `x >= max_value`,
`f(x) = x` for `threshold <= x < max_value`,
`f(x) = alpha * (x - threshold)` otherwise.
# Arguments
x: Input tensor.
alpha: float. Slope of the negative part. Defaults to zero.
max_value: float. Saturation threshold.
threshold: float. Threshold value for thresholded activation.
# Returns
A tensor.
"""
return K.relu(x, alpha=alpha, max_value=max_value, threshold=threshold)
def tanh(x):
"""Hyperbolic tangent activation function.
# Arguments
x: Input tensor.
# Returns
The hyperbolic activation:
`tanh(x) = (exp(x) - exp(-x)) / (exp(x) + exp(-x))`
"""
return K.tanh(x)
def sigmoid(x):
"""Sigmoid activation function.
# Arguments
x: Input tensor.
# Returns
The sigmoid activation: `1 / (1 + exp(-x))`.
"""
return K.sigmoid(x)
def hard_sigmoid(x):
"""Hard sigmoid activation function.
Faster to compute than sigmoid activation.
# Arguments
x: Input tensor.
# Returns
Hard sigmoid activation:
- `0` if `x < -2.5`
- `1` if `x > 2.5`
- `0.2 * x + 0.5` if `-2.5 <= x <= 2.5`.
"""
return K.hard_sigmoid(x)
def exponential(x):
"""Exponential (base e) activation function.
# Arguments
x: Input tensor.
# Returns
Exponential activation: `exp(x)`.
"""
return K.exp(x)
def linear(x):
"""Linear (i.e. identity) activation function.
# Arguments
x: Input tensor.
# Returns
Input tensor, unchanged.
"""
return x
def serialize(activation):
return activation.__name__
def deserialize(name, custom_objects=None):
return deserialize_keras_object(
name,
module_objects=globals(),
custom_objects=custom_objects,
printable_module_name='activation function')
def get(identifier):
"""Get the `identifier` activation function.
# Arguments
identifier: None or str, name of the function.
# Returns
The activation function, `linear` if `identifier` is None.
# Raises
ValueError if unknown identifier
"""
if identifier is None:
return linear
if isinstance(identifier, six.string_types):
identifier = str(identifier)
return deserialize(identifier)
elif callable(identifier):
if isinstance(identifier, Layer):
warnings.warn(
'Do not pass a layer instance (such as {identifier}) as the '
'activation argument of another layer. Instead, advanced '
'activation layers should be used just like any other '
'layer in a model.'.format(
identifier=identifier.__class__.__name__))
return identifier
else:
raise ValueError('Could not interpret '
'activation function identifier:', identifier)
