先來(lái)一個(gè)簡(jiǎn)單的測(cè)試
# curl -XPOST "http://192.168.9.155:9200/_analyze?analyzer=standard&pretty" -d 'PHP是世界上最好的語(yǔ)言' //_analyze表示分析分詞拭卿;analyzer=standard蟹肘,表示分詞方式standard; -d表示測(cè)試的一段文字
測(cè)試結(jié)果
{
"tokens" : [
{
"token" : "php",
"start_offset" : 0,
"end_offset" : 3,
"type" : "<ALPHANUM>",
"position" : 0
},
{
"token" : "是",
"start_offset" : 3,
"end_offset" : 4,
"type" : "<IDEOGRAPHIC>",
"position" : 1
},
{
"token" : "世",
"start_offset" : 4,
"end_offset" : 5,
"type" : "<IDEOGRAPHIC>",
"position" : 2
},
{
"token" : "界",
"start_offset" : 5,
"end_offset" : 6,
"type" : "<IDEOGRAPHIC>",
"position" : 3
},
{
"token" : "上",
"start_offset" : 6,
"end_offset" : 7,
"type" : "<IDEOGRAPHIC>",
"position" : 4
},
{
"token" : "最",
"start_offset" : 7,
"end_offset" : 8,
"type" : "<IDEOGRAPHIC>",
"position" : 5
},
{
"token" : "好",
"start_offset" : 8,
"end_offset" : 9,
"type" : "<IDEOGRAPHIC>",
"position" : 6
},
{
"token" : "的",
"start_offset" : 9,
"end_offset" : 10,
"type" : "<IDEOGRAPHIC>",
"position" : 7
},
{
"token" : "語(yǔ)",
"start_offset" : 10,
"end_offset" : 11,
"type" : "<IDEOGRAPHIC>",
"position" : 8
},
{
"token" : "言",
"start_offset" : 11,
"end_offset" : 12,
"type" : "<IDEOGRAPHIC>",
"position" : 9
}
]
}
接下來(lái)使用我們的IK
ik 帶有兩個(gè)分詞器
ik_max_word :會(huì)將文本做最細(xì)粒度的拆分完沪;盡可能多的拆分出詞語(yǔ)驯击,拼接各種可能的組合 屋谭。
ik_smart:會(huì)做最粗粒度的拆分;已被分出的詞語(yǔ)將不會(huì)再次被其它詞語(yǔ)占有 转晰。
curl -XPOST "http://192.168.9.155:9200/_analyze?analyzer=ik_smart&pretty" -d 'PHP是世界上最好的語(yǔ)言' //ik_smart方式
{
"tokens" : [
{
"token" : "php",
"start_offset" : 0,
"end_offset" : 3,
"type" : "ENGLISH",
"position" : 0
},
{
"token" : "世界上",
"start_offset" : 4,
"end_offset" : 7,
"type" : "CN_WORD",
"position" : 1
},
{
"token" : "最好",
"start_offset" : 7,
"end_offset" : 9,
"type" : "CN_WORD",
"position" : 2
},
{
"token" : "語(yǔ)言",
"start_offset" : 10,
"end_offset" : 12,
"type" : "CN_WORD",
"position" : 3
}
]
}
curl -XPOST "http://192.168.9.155:9200/_analyze?analyzer=ik_max_word&pretty" -d 'PHP是世界上最好的語(yǔ)言' //ik_max_word方式
{
"tokens" : [
{
"token" : "php",
"start_offset" : 0,
"end_offset" : 3,
"type" : "ENGLISH",
"position" : 0
},
{
"token" : "世界上",
"start_offset" : 4,
"end_offset" : 7,
"type" : "CN_WORD",
"position" : 1
},
{
"token" : "世界",
"start_offset" : 4,
"end_offset" : 6,
"type" : "CN_WORD",
"position" : 2
},
{
"token" : "上",
"start_offset" : 6,
"end_offset" : 7,
"type" : "CN_CHAR",
"position" : 3
},
{
"token" : "最好",
"start_offset" : 7,
"end_offset" : 9,
"type" : "CN_WORD",
"position" : 4
},
{
"token" : "語(yǔ)言",
"start_offset" : 10,
"end_offset" : 12,
"type" : "CN_WORD",
"position" : 5
}
]
}
區(qū)別很明顯~