PART I 決策樹 (Decision Tree)
決策樹基本知識(shí)
image.png
image.png
決策樹何時(shí)停止生長(zhǎng)：
(I) all leaf nodes are pure with entropy of zero;
(II) a prespecified minimum change in purity cannot be made with any splitting methods;
(III) the number of observations in the leaf node reaches the pre-specified minimum one.
- 加載并查看數(shù)據(jù)集
data(airquality)
str(airquality)
# 'data.frame': 153 obs. of 6 variables:
# $ Ozone : int 41 36 12 18 NA 28 23 19 8 NA ...
# $ Solar.R: int 190 118 149 313 NA NA 299 99 19 194 ...
# $ Wind : num 7.4 8 12.6 11.5 14.3 14.9 8.6 13.8 20.1 8.6 ...
# $ Temp : int 67 72 74 62 56 66 65 59 61 69 ...
# $ Month : int 5 5 5 5 5 5 5 5 5 5 ...
# $ Day : int 1 2 3 4 5 6 7 8 9 10 ...
- 插補(bǔ)
Ozone變量中的缺失值
set.seed(888)
airquality[is.na(airquality$Ozone),1] <- sample(airquality[!is.na(airquality$Ozone),1],37) #使用非缺失值進(jìn)行插補(bǔ)
summary(airquality$Ozone)
# Min. 1st Qu. Median Mean 3rd Qu. Max.
# 1.00 16.00 30.00 41.31 63.00 168.00
- 擬合模型
library(party)
airct <- ctree(Ozone ~ ., data = airquality,controls = ctree_control(maxsurrogate = 3))
airct
# Conditional inference tree with 4 terminal nodes
#
# Response: Ozone
# Inputs: Solar.R, Wind, Temp, Month, Day
# Number of observations: 153
#
# 1) Temp <= 76; criterion = 1, statistic = 47.479
# 2)* weights = 61
# 1) Temp > 76
# 3) Wind <= 6.3; criterion = 1, statistic = 24.235
# 4)* weights = 21
# 3) Wind > 6.3
# 5) Temp <= 84; criterion = 0.964, statistic = 7.182
# 6)* weights = 45
# 5) Temp > 84
# 7)* weights = 26
- 結(jié)果可視化
plot(airct)
image.png
查看具體每個(gè)節(jié)點(diǎn)的情況
plot(airct, inner_panel = node_boxplot, edge_panel = function(...) invisible(),tnex = 1)
image.png
inner <- nodes(airct, c(1,3,5,7))
layout(matrix(1:length(inner), ncol = length(inner)/2))
out <- sapply(inner, function(i) {
splitstat <- i$psplit$splitstatistic
x <- airquality[[i$psplit$variableName]][splitstat >0]
plot(x, splitstat[splitstat > 0],main =
paste("Node",i$nodeID), xlab = i$psplit$variableName,
ylab = "Statistic",ylim = c(0, 10), cex.axis = 1.2, cex.lab =
1.2,cex.main = 1.2)
abline(v = i$psplit$splitpoint, lty = 4)
})
image.png
連續(xù)型變量有多種分割方法，用一個(gè)統(tǒng)計(jì)量statistic可以描述每種分割方法的好差程度，statistic值越大說(shuō)明分割越好
- 決策樹的預(yù)測(cè)
ind <- sample(2, nrow(airquality), replace=TRUE, prob
= c(0.7,0.3))
newData <- airquality[ind==2,]
newpred <- predict(airct, newdata= newData)
plot(newpred,newData$Ozone,xlab="Ozone value predicted by decision tree",
ylab="Observed ozone value")
image.png
PART II 隨機(jī)森林 (Random Forest)
image.png
- 擬合隨機(jī)森林模型
aircf<-cforest(Ozone ~ ., data = airquality)
aircf
# Random Forest using Conditional Inference Trees
#
# Number of trees: 500
#
# Response: Ozone
# Inputs: Solar.R, Wind, Temp, Month, Day
# Number of observations: 153
- 評(píng)估預(yù)測(cè)效果
predforest <- predict(aircf, newdata= newData)
plot(predforest,newData$Ozone,ylab="Observed ozone value",
xlab="Predicted ozone value based on random forest")
image.png
PART III Model based recursive partitioning
airmod <- mob(Ozone ~Temp+Day|Solar.R+Wind+Month, data = airquality)
# Variables after symbol “|” are partitioning variables
plot(airmod)
image.png
airmod
# 1) Wind <= 6.3; criterion = 0.999, statistic = 25.078
# 2)* weights = 23
# Terminal node model
# Gaussian GLM with coefficients:
# (Intercept) Temp Day
# -96.4060 2.0736 -0.1832
#
# 1) Wind > 6.3
# 3)* weights = 123
# Terminal node model
# Gaussian GLM with coefficients:
# (Intercept) Temp Day
# -90.0534 1.5811 0.2236
#
coef(airmod)
# (Intercept) Temp Day
# 2 -96.40597 2.073623 -0.1831996
# 3 -90.05342 1.581143 0.2235920
sctest(airmod,node = 1)
# Solar.R Wind Month
# statistic 6.0347325 25.077906381 22.955024196
# p.value 0.9661175 0.001153747 0.003099691
參考資料
- 章仲恒教授丁香園課程：決策樹與隨機(jī)森林
- Annals of Translational Medicine: Big-data Clinical Trial Column
- Zhongheng Zhang Decision tree modeling using R
