• Dataset transformations| 數(shù)據(jù)轉(zhuǎn)換

• Combining estimators|組合學(xué)習(xí)器
• Feature extration|特征提取
• Preprocessing data|數(shù)據(jù)預(yù)處理

<p id='1'>1 Dataset transformations</p>

scikit-learn provides a library of transformers, which may clean (see Preprocessing data), reduce (see Unsupervised dimensionality reduction), expand (see Kernel Approximation) or generate (see Feature extraction) feature representations.

scikit-learn 提供了數(shù)據(jù)轉(zhuǎn)換的模塊，包括數(shù)據(jù)清理残炮、降維、擴(kuò)展和特征提取傀蚌。

Like other estimators, these are represented by classes with fit method, which learns model parameters (e.g. mean and standard deviation for normalization) from a training set, and a transform method which applies this transformation model to unseen data. fit_transform may be more convenient and efficient for modelling and transforming the training data simultaneously.

scikit-learn模塊有3種通用的方法：fit(X,y=None)、transform(X)膀估、fit_transform(X)谎亩、inverse_transform(newX)。fit用來訓(xùn)練模型轴或；transform在訓(xùn)練后用來降維；fit_transform先用訓(xùn)練模型仰禀，然后返回降維后的X照雁；inverse_transform用來將降維后的數(shù)據(jù)轉(zhuǎn)換成原始數(shù)據(jù)。

<p id='1.1'>1.1 combining estimators</p>

• <p id='1.1.1'>1.1.1 Pipeline:chaining estimators</p>

Pipeline 模塊是用來組合一系列估計(jì)器的答恶。對固定的一系列操作非常便利饺蚊，如：同時(shí)結(jié)合特征選擇、數(shù)據(jù)標(biāo)準(zhǔn)化悬嗓、分類污呼。

• Usage|使用
  代碼：

from sklearn.pipeline import Pipeline  
from sklearn.svm import SVC 
from sklearn.decomposition import PCA
from sklearn.pipeline import make_pipeline
#define estimators
#the arg is a list of (key,value) pairs,where the key is a string you want to give this step and value is an estimators object
estimators=[('reduce_dim',PCA()),('svm',SVC())]  
#combine estimators
clf1=Pipeline(estimators)
clf2=make_pipeline(PCA(),SVC())  #use func make_pipeline() can do the same thing
print(clf1,'\n',clf2) 

輸出：

Pipeline(steps=[('reduce_dim', PCA(copy=True, n_components=None, whiten=False)), ('svm',           SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False))]) 
 Pipeline(steps=[('pca', PCA(copy=True, n_components=None, whiten=False)), ('svc', SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False))])

可以通過set_params()方法設(shè)置學(xué)習(xí)器的屬性，參數(shù)形式為<estimator>_<parameter>

clf.set_params(svm__C=10)

上面的方法在網(wǎng)格搜索時(shí)很重要：

from sklearn.grid_search import GridSearchCV
params = dict(reduce_dim__n_components=[2, 5, 10],svm__C=[0.1, 10, 100])
grid_search = GridSearchCV(clf, param_grid=params)

上面的例子相當(dāng)于把pipeline生成的學(xué)習(xí)器作為一個(gè)普通的學(xué)習(xí)器包竹，參數(shù)形式為<estimator>_<parameter>燕酷。

• Note|說明
  1.可以使用dir()函數(shù)查看clf的所有屬性和方法籍凝。例如step屬性就是每個(gè)操作步驟的屬性。
  如

('reduce_dim', PCA(copy=True, n_components=None, whiten=False))

2.調(diào)用pipeline生成的學(xué)習(xí)器的fit方法相當(dāng)于依次調(diào)用其包含的所有學(xué)習(xí)器的方法苗缩，transform輸入然后把結(jié)果扔向下一步驟饵蒂。pipeline生成的學(xué)習(xí)器有著它包含的學(xué)習(xí)器的所有方法。如果最后一個(gè)學(xué)習(xí)器是分類酱讶，那么生成的學(xué)習(xí)器就是分類退盯，如果最后一個(gè)是transform，那么生成的學(xué)習(xí)器就是transform浴麻，依次類推得问。

• <p id='1.1.2'> 1.1.2 FeatureUnion: composite feature spaces</p>

與pipeline不同的是FeatureUnion只組合transformer，它們也可以結(jié)合成更復(fù)雜的模型软免。

FeatureUnion combines several transformer objects into a new transformer that combines their output. AFeatureUnion takes a list of transformer objects. During fitting, each of these is fit to the data independently. For transforming data, the transformers are applied in parallel, and the sample vectors they output are concatenated end-to-end into larger vectors.

• Usage|使用
  代碼：

from sklearn.pipeline import FeatureUnion   
from sklearn.decomposition import PCA
from sklearn.decomposition import KernelPCA
from sklearn.pipeline import make_union
#define transformers
#the arg is a list of (key,value) pairs,where the key is a string you want to give this step and value is an transformer object
estimators=[('linear_pca)',PCA()),('Kernel_pca',KernelPCA())]  
#combine transformers
clf1=FeatureUnion(estimators)
clf2=make_union(PCA(),KernelPCA())
print(clf1,'\n',clf2) 
print(dir(clf1))

輸出：

FeatureUnion(n_jobs=1,
       transformer_list=[('linear_pca)', PCA(copy=True, n_components=None, whiten=False)), ('Kernel_pca', KernelPCA(alpha=1.0, coef0=1, degree=3, eigen_solver='auto',
     fit_inverse_transform=False, gamma=None, kernel='linear',
     kernel_params=None, max_iter=None, n_components=None,
     remove_zero_eig=False, tol=0))],
       transformer_weights=None) 
 FeatureUnion(n_jobs=1,
       transformer_list=[('pca', PCA(copy=True, n_components=None, whiten=False)), ('kernelpca', KernelPCA(alpha=1.0, coef0=1, degree=3, eigen_solver='auto',
     fit_inverse_transform=False, gamma=None, kernel='linear',
     kernel_params=None, max_iter=None, n_components=None,
     remove_zero_eig=False, tol=0))],
       transformer_weights=None)

可以看出FeatureUnion的用法與pipeline一致

• Note|說明

(A [FeatureUnion
](http://scikit- learn.org/stable/modules/generated/sklearn.pipeline.FeatureUnion.html#sklearn.pipeline.FeatureUn ion) has no way of checking whether two transformers might produce identical features. It only produces a union when the feature sets are disjoint, and making sure they are is the caller’s responsibility.)

Here is a example python source code:[feature_stacker.py](http://scikit-learn.org/stable/_downloads/feature_stacker.py)

<p id='1.2'>1.2 Feature extraction</p>

The sklearn.feature_extraction
module can be used to extract features in a format supported by machine learning algorithms from datasets consisting of formats such as text and image.

skilearn.feature_extraction模塊是用機(jī)器學(xué)習(xí)算法所支持的數(shù)據(jù)格式來提取數(shù)據(jù)，如將text和image信息轉(zhuǎn)換成dataset焚挠。
Note:
Feature extraction（特征提雀嘞簟）與Feature selection（特征選擇）不同，前者是用來將非數(shù)值的數(shù)據(jù)轉(zhuǎn)換成數(shù)值的數(shù)據(jù)蝌衔，后者是用機(jī)器學(xué)習(xí)的方法對特征進(jìn)行學(xué)習(xí)（如PCA降維）榛泛。

• <p id='1.2.1'>1.2.1 Loading features from dicts</p>

The class DictVectorizer
can be used to convert feature arrays represented as lists of standard Python dict
objects to the NumPy/SciPy representation used by scikit-learn estimators.
Dictvectorizer類用來將python內(nèi)置的dict類型轉(zhuǎn)換成數(shù)值型的array。dict類型的好處是在存儲(chǔ)稀疏數(shù)據(jù)時(shí)不用存儲(chǔ)無用的值噩斟。

代碼：

measurements=[{'city': 'Dubai', 'temperature': 33.}
,{'city': 'London', 'temperature':12.}
,{'city':'San Fransisco','temperature':18.},]
from sklearn.feature_extraction import DictVectorizer
vec=DictVectorizer()
x=vec.fit_transform(measurements).toarray()
print(x)
print(vec.get_feature_names())```
輸出：

[[ 1. 0. 0. 33.]
[ 0. 1. 0. 12.]
[ 0. 0. 1. 18.]]
['city=Dubai', 'city=London', 'city=San Fransisco', 'temperature']
[Finished in 0.8s]

* ###<p id='1.2.2'>1.2.2 Feature hashing</p>
* ###<p id='1.2.3'>1.2.3 Text feature extraction</p>
* ###<p id='1.2.4'>1.2.4 Image feature extraction</p>
以上三小節(jié)暫未考慮（設(shè)計(jì)到語言處理及圖像處理）[見官方文檔][官方文檔]
[官方文檔]: http://scikit-learn.org/stable/data_transforms.html

##<p id='1.3'>1.3 Preprogressing data</p>
>The sklearn.preprocessing
 package provides several common utility functions and transformer classes to change raw feature vectors into a representation that is more suitable for the downstream estimators

sklearn.preprogressing模塊提供了幾種常見的數(shù)據(jù)轉(zhuǎn)換曹锨，如標(biāo)準(zhǔn)化、歸一化等剃允。
* ###<p id='1.3.1'>1.3.1 Standardization, or mean removal and variance scaling</p>
>**Standardization** of datasets is a **common requirement for many machine learning estimators** implemented in the scikit; they might behave badly if the individual features do not more or less look like standard normally distributed data: Gaussian with **zero mean and unit variance**.

 很多學(xué)習(xí)算法都要求事先對數(shù)據(jù)進(jìn)行標(biāo)準(zhǔn)化沛简，如果不是像標(biāo)準(zhǔn)正太分布一樣0均值1方差就可能會(huì)有很差的表現(xiàn)。

 * Usage|用法

 代碼：
```python
from sklearn import preprocessing
import numpy as np
X = np.array([[1.,-1., 2.], [2.,0.,0.], [0.,1.,-1.]])
Y=X
Y_scaled = preprocessing.scale(Y)
y_mean=Y_scaled.mean(axis=0) #If 0, independently standardize each feature, otherwise (if 1) standardize each sample|axis=0 時(shí)求每個(gè)特征的均值斥废，axis=1時(shí)求每個(gè)樣本的均值
y_std=Y_scaled.std(axis=0)
print(Y_scaled)
scaler= preprocessing.StandardScaler().fit(Y)#用StandardScaler類也能完成同樣的功能
print(scaler.transform(Y))

輸出：

[[ 0.         -1.22474487  1.33630621]
 [ 1.22474487  0.         -0.26726124]
 [-1.22474487  1.22474487 -1.06904497]]
[[ 0.         -1.22474487  1.33630621]
 [ 1.22474487  0.         -0.26726124]
 [-1.22474487  1.22474487 -1.06904497]]
[Finished in 1.4s]

• Note|說明
  1.func scale
  2.class StandardScaler
  3.StandardScaler 是一種Transformer方法椒楣，可以讓pipeline來使用。
  MinMaxScaler （min-max標(biāo)準(zhǔn)化[0,1]）類和MaxAbsScaler（[-1,1]）類是另外兩個(gè)標(biāo)準(zhǔn)化的方式牡肉，用法和StandardScaler類似捧灰。
  4.處理稀疏數(shù)據(jù)時(shí)用MinMax和MaxAbs很合適
  5.魯棒的數(shù)據(jù)標(biāo)準(zhǔn)化方法（適用于離群點(diǎn)很多的數(shù)據(jù)處理）：

the median and the interquartile range often give better results

用中位數(shù)代替均值（使均值為0），用上四分位數(shù)-下四分位數(shù)代替方差（IQR為1统锤？）毛俏。

• <p id='1.3.2'>1.3.2 Impution of missing values|缺失值的處理</p>

• Usage
  代碼：

import scipy.sparse as sp
from sklearn.preprocessing import Imputer
X=sp.csc_matrix([[1,2],[0,3],[7,6]])
imp=preprocessing.Imputer(missing_value=0,strategy='mean',axis=0)
imp.fit(X)
X_test=sp.csc_matrix([[0, 2], [6, 0], [7, 6]])
print(X_test)
print(imp.transform(X_test))

輸出：

  (1, 0)    6
  (2, 0)    7
  (0, 1)    2
  (2, 1)    6
[[ 4.          2.        ]
 [ 6.          3.66666675]
 [ 7.          6.        ]]
[Finished in 0.6s]

• Note
  1.scipy.sparse是用來存儲(chǔ)稀疏矩陣的
  2.Imputer可以用來處理scipy.sparse稀疏矩陣

• <p id='1.3.3'>1.3.3 Generating polynomial features</p>

• Usage
  代碼：

import numpy as np
from sklearn.preprocessing import PolynomialFeatures
X=np.arange(6).reshape(3,2)
print(X)
poly=PolynomialFeatures(2)
print(poly.fit_transform(X))

輸出：

[[0 1]
 [2 3]
 [4 5]]
[[  1.   0.   1.   0.   0.   1.]
 [  1.   2.   3.   4.   6.   9.]
 [  1.   4.   5.  16.  20.  25.]]
[Finished in 0.8s]

• Note
  生成多項(xiàng)式特征用在多項(xiàng)式回歸中以及多項(xiàng)式核方法中 。

• <p id='1.3.4'>1.3.4 Custom transformers</p>

這是用來構(gòu)造transform方法的函數(shù)

• Usage:
  代碼：

import numpy as np
from sklearn.preprocessing import FunctionTransformer
transformer = FunctionTransformer(np.log1p)
x=np.array([[0,1],[2,3]])
print(transformer.transform(x))

輸出：

[[ 0.          0.69314718]
 [ 1.09861229  1.38629436]]
[Finished in 0.8s]

• Note

For a full code example that demonstrates using a FunctionTransformer
to do custom feature selection, see Using FunctionTransformer to select columns