主要內(nèi)容：
- Pandas中的常用統(tǒng)計(jì)量
- 頻數(shù)
- 數(shù)據(jù)分割
- 增量百分比
- Covariance協(xié)方差
- Correlation相關(guān)
- 秩
Pandas中的常用統(tǒng)計(jì)量
Method Description
-----------------------------------------------------------
count() Number of non-null observations
sum() Sum of values
mean() Mean of values
median() Arithmetic median of values
min() Minimum
max() Maximum
std() Bessel-corrected sample standard deviation
var() Unbiased variance
skew() Sample skewness (3rd moment)
kurt() Sample kurtosis (4th moment)
quantile() Sample quantile (value at %)
apply() Generic apply
cov() Unbiased covariance (binary)
corr() Correlation (binary)
使用方法很簡單啊肄程，dataframe和Series都可以用明也。默認(rèn)計(jì)算的是列的統(tǒng)計(jì)量。比如：
df.mean() #按列計(jì)算均值
df.mean(1) #按行計(jì)算均值
df.T #也可以用轉(zhuǎn)置函數(shù)操控行列骤宣。
另外可以用describe()計(jì)算常用的幾個(gè)統(tǒng)計(jì)量。
df.describe() #按列計(jì)算count, mean, std, min, 25% 50% 75%, max
太簡單序愚，不舉例了憔披。
頻數(shù)
對于離散分布的數(shù)據(jù)，頻數(shù)是一個(gè)很重要的統(tǒng)計(jì)量。
In [13]: data = np.random.randint(0, 7, size=50)
In [14]: data
Out[14]:
array([0, 1, 6, 2, 4, 2, 4, 2, 5, 2, 0, 2, 0, 2, 2, 3, 4, 6, 6, 1, 6, 3, 6,
5, 0, 0, 6, 1, 5, 6, 2, 6, 0, 3, 3, 6, 4, 6, 0, 0, 0, 4, 0, 6, 5, 6,
5, 6, 3, 1])
In [15]: s = pd.Series(data)
In [16]: s.value_counts()
Out[16]:
6 13
0 10
2 8
5 5
4 5
3 5
1 4
dtype: int64
# 還可以用以下方式計(jì)算頻率芬膝。
# numpy array也可以用此方法計(jì)算望门。
In [17]: pd.value_counts(data)
In [18]: pd.value_counts(s)
# 結(jié)果跟s.value_counts()一樣，不再貼出來了锰霜。
可以用眾數(shù)mode的方法選取出現(xiàn)最多的值筹误。
In [20]: s.mode()
Out[20]:
0 6
dtype: int32
數(shù)據(jù)分割
.cut(arr, n) 分成n部分，qcut(arr, [])百分等級(jí)癣缅〕簦看例子。
In [25]: arr = np.random.randn(20)
In [26]: factor = pd.cut(arr, 4) #四等分
In [27]: factor #結(jié)果標(biāo)出來每個(gè)數(shù)據(jù)屬于哪個(gè)范圍
Out[27]:
[(-0.356, 0.695], (-0.356, 0.695], (-0.356, 0.695], (-0.356, 0.695], (-0.356, 0.695], ..., (-1.412, -0.356], (-0.356, 0.695], (-1.412, -0.356], (-1.412, -0.356], (0.695, 1.747]]
Length: 20
Categories (4, object): [(-1.412, -0.356] < (-0.356, 0.695] < (0.695, 1.747] < (1.747, 2.799]]
In [28]: factor = pd.cut(arr, [-5, -1, 0, 1, 5])
In [29]: factor
Out[29]:
[(-1, 0], (0, 1], (0, 1], (0, 1], (0, 1], ..., (-1, 0], (0, 1], (-1, 0], (-5, -1], (0, 1]]
Length: 20
Categories (4, object): [(-5, -1] < (-1, 0] < (0, 1] < (1, 5]]
# 計(jì)算每個(gè)部分的個(gè)數(shù)
In [30]: pd.value_counts(factor)
Out[30]:
(0, 1] 7
(-1, 0] 7
(1, 5] 3
(-5, -1] 3
dtype: int64
In [31]: factor2 = pd.qcut(arr, [0, .25, .5, .75, 1])
In [32]: factor2
Out[32]:
[(-0.392, 0.0332], (0.0332, 0.663], (0.0332, 0.663], (0.0332, 0.663], (0.0332, 0.663], ..., [-1.408, -0.392], (0.0332, 0.663], [-1.408, -0.392], [-1.408, -0.392], (0.663, 2.799]]
Length: 20
Categories (4, object): [[-1.408, -0.392] < (-0.392, 0.0332] < (0.0332, 0.663] < (0.663, 2.799]]
增量百分比
增量百分比（percentage increase）：（新值-原來的值）/原來的值
In [19]: ser = pd.Series(np.random.randn(8))
In [20]: ser
Out[20]:
0 -0.299098
1 -0.083835
2 -0.899787
3 1.331207
4 -0.931809
5 0.363284
6 -1.324239
7 -0.187943
dtype: float64
In [21]: ser.pct_change()
Out[21]:
0 NaN
1 -0.719708
2 9.732820
3 -2.479470
4 -1.699973
5 -1.389869
6 -4.645190
7 -0.858075
dtype: float64
In [22]: ser.pct_change(periods=3)
Out[22]:
0 NaN
1 NaN
2 NaN
3 -5.450733
4 10.114794
5 -1.403744
6 -1.994765
7 -0.798303
dtype: float64
Covariance協(xié)方差
In [5]: s1 = pd.Series(np.random.randn(1000))
In [6]: s2 = pd.Series(np.random.randn(1000))
In [7]: s1.cov(s2)
In [8]: frame = pd.DataFrame(np.random.randn(1000, 5), columns=['a', 'b', 'c', 'd', 'e'])
In [9]: frame.cov()
Correlation相關(guān)
可用的方法：pearson（默認(rèn)）友存、kendall祷膳、spearman
In [3]: frame = pd.DataFrame(np.random.randn(1000, 5), columns=['a', 'b', 'c', 'd', 'e'])
In [4]: frame.ix[::2] = np.nan
In [5]: frame.head()
Out[5]:
a b c d e
0 NaN NaN NaN NaN NaN
1 1.323677 0.000110 -1.333564 0.099595 0.151210
2 NaN NaN NaN NaN NaN
3 -0.038058 -0.559302 1.011685 -0.796040 0.123981
4 NaN NaN NaN NaN NaN
#單列之間的相關(guān)，也就是series之間的相關(guān)屡立。
In [6]: frame['a'].corr(frame['b'])
Out[6]: 0.043408577177721466
#指定計(jì)算相關(guān)的方法直晨，結(jié)果會(huì)有不同。
In [7]: frame['a'].corr(frame['b'], method='spearman')
Out[7]: 0.045387541550166201
#整個(gè)數(shù)據(jù)庫的相關(guān)膨俐，生成一個(gè)矩陣勇皇，兩兩之間的相關(guān)系數(shù)。
In [8]: In [19]: frame.corr()
Out[8]:
a b c d e
a 1.000000 0.043409 -0.044429 -0.062085 0.059482
b 0.043409 1.000000 -0.068241 0.061163 -0.019741
c -0.044429 -0.068241 1.000000 -0.028021 -0.066897
d -0.062085 0.061163 -0.028021 1.000000 -0.034094
e 0.059482 -0.019741 -0.066897 -0.034094 1.000000
可以設(shè)置min_periods參數(shù)吟策，用來指定所需要的非空數(shù)據(jù)點(diǎn)的閾值儒士，如果非空數(shù)據(jù)點(diǎn)小于閾值結(jié)果為NA。在下一篇關(guān)于rolling的介紹中會(huì)用到檩坚，這里不詳細(xì)說了着撩。
另一個(gè)計(jì)算相關(guān)的方法corrwith()
In [25]: index = ['a', 'b', 'c', 'd', 'e']
In [26]: columns = ['one', 'two', 'three', 'four']
In [27]: df1 = pd.DataFrame(np.random.randn(5, 4), index=index, columns=columns)
In [28]: df2 = pd.DataFrame(np.random.randn(4, 4), index=index[:4], columns=columns)
In [29]: df1.corrwith(df2)
Out[29]:
one -0.125501
two -0.493244
three 0.344056
four 0.004183
dtype: float64
In [30]: df2.corrwith(df1, axis=1)
Out[30]:
a -0.675817
b 0.458296
c 0.190809
d -0.186275
e NaN
dtype: float64
秩
In [31]: s = pd.Series(np.random.np.random.randn(5), index=list('abcde'))
In [32]: s['d'] = s['b'] # so there's a tie
In [33]: s.rank()
Out[33]:
a 5.0
b 2.5
c 1.0
d 2.5
e 4.0
dtype: float64
In [29]: df = pd.DataFrame(np.random.np.random.randn(10, 6))
...: df.rank(1) # axis=1 對列rank，也就是求一行數(shù)據(jù)的秩
In [30]: df.rank(0) # 默認(rèn)是0
