查詢和過濾的區(qū)別
ES提供基于JSON的完整DSL來定義查詢憔儿,查詢DSL包括兩種子句:
葉查詢子句:在特定的字段上查找特定的值,比如match分预、term或者range查詢趴梢,這些查詢可以自己使用
復合查詢子句:包含其他葉查詢或復合查詢子句,以合理的方式結(jié)合多條查詢(比如bool或dis_max查詢)秀睛,或者改變查詢行為(比如not或constant_score查詢)
查詢(query):用于檢查內(nèi)容與條件是否匹配尔当,并且計算_score元字段表示匹配度,查詢的結(jié)構(gòu)中以query參數(shù)開始來執(zhí)行內(nèi)容查詢
過濾(filter):不計算匹配得分蹂安,只是簡單地決定文檔是否匹配椭迎,內(nèi)容過濾主要用于過濾結(jié)構(gòu)化的數(shù)據(jù)
使用過濾往往會被ElasticSearch自動緩存來提高性能
查詢子句也可以傳遞filter查詢,比如bool查詢內(nèi)的filter田盈、constant_score查詢內(nèi)的filter等
舉個查詢子句的例子畜号,查詢會匹配符合下列所有文檔
1、title字段包含單詞 "重生甜俏妻逆襲"
2允瞧、author字段包含"簾半卷"
3简软、amount字段為100
4、ConsumeTime字段包含從2018-01-01至今的日期
請求: POST http://127.0.0.1:9200/xy_order/_search
{
"query":{
"bool":{
"must":[
{"match":{"title":"重生甜俏妻逆襲"}},
{"match":{"author":"簾半卷"}}
],
"filter":[
{"term":"amount":100},
{"range","ConsumeTime":{"gte":"2018-01-01"}}
]
}
}
}
query參數(shù)表示內(nèi)容查詢瓷式,內(nèi)容查詢中使用的bool和match子句替饿,用于計算每個文檔的匹配得分。
filter參數(shù)表示內(nèi)容過濾贸典,內(nèi)容過濾中使用的term和range子句视卢,會過濾掉不匹配的文檔,并且不影響計算文檔匹配得分廊驼。
全文搜索
標準查詢:接受文本/數(shù)字/日期的查詢据过,分析參數(shù)并組成查詢條件惋砂,例如:
{
"match":{
"message":"this is a test"
}
}
注意message是字段名,可以用任何字段的名(包括_all)來替換绳锅。
有三種類型的match查詢:布爾(bool)西饵、短語(phrase)和短語前綴(phrase_prefix),除此之外還有多段查詢鳞芙、Lucene語法查詢眷柔、簡化查詢
布爾查詢
默認的標準查詢類型,分析文本并且組成一個布爾類型原朝。operator參數(shù)可以設置為or或者and來控制布爾子句(默認為or)驯嘱,用于匹配的should子句的最小數(shù)量可以使用minimun_should_match參數(shù)來設置
可以設置analyzer來控制在文本上執(zhí)行分析過程的分詞器。默認是字段映射中明確定義或者默認的搜索分詞器
lenient參數(shù)可以設置為誒true來忽略數(shù)據(jù)類型匹配出錯造成的異常喳坠,例如嘗試通過文本查詢字符串來查詢數(shù)字類型字段默認為false
短語查詢
短語查詢分析文本并創(chuàng)建短語查詢鞠评,例如:
GET xy_order/_search
{
"query": {
"match_phrase":{
"cbid":"8228721504057103"
}
}
}
短語查詢根據(jù)一個可配置的slope匹配索引詞
可以設置analyzer來控制將要在文本上執(zhí)行分詞的分詞器,默認是字段映射中定義的分析器或者默認的搜索分析器壕鹉,例如:
GET xy_order/_search
{
"query": {
"match_phrase":{
"cbid":{
"query":"8228721504057103","analyzer": "standard"
}
}
}
}
短語前綴查詢
可以對文本最后一個字段進行前綴匹配剃幌,例如:
GET xy_order/_search
{
"query": {
"match_phrase_prefix":{"ServerIp":"10.236"}
}
}
也可以接受max_expansions參數(shù),可以控制最后索引詞會擴展多少前綴晾浴,推薦設置為一個可以接受的值來控制查詢的執(zhí)行時間负乡。例如:
GET xy_order/_search
{
"query": {
"match_phrase_prefix":{"ServerIp":{"query":"10.236", "max_expansions": 10}}
}
}
多字段查詢
在標準查詢的基礎上,支持多字段查詢:
GET bookcenter/_search
{
"query": {
"multi_match": {
"query": "少年",
"fields": ["desc","content"]
}
}
}
字段可以通過通配符指定:
GET bookcenter/_search
{
"query": {
"multi_match": {
"query": "法師",
"fields": ["desc","*tent"]
}
}
}
結(jié)果:
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.9635944,
"hits": [
{
"_index": "bookcenter",
"_type": "qidian",
"_id": "3",
"_score": 0.9635944,
"_source": {
"title": "全職法師",
"author": "亂",
"desc": "心潮澎湃怠肋,無限幻想敬鬓,迎風揮擊千層浪淹朋,少年不敗熱血笙各!",
"category": "東方玄幻",
"content": "一覺醒來,世界大變熟悉的高中傳授的是魔法础芍,告訴大家要成為一名出色的魔法師杈抢。居住的都市之外游蕩著襲擊人類的魔物妖獸,虎視眈眈.崇尚科學的世界變成了崇尚魔法仑性,偏偏有著一樣以學渣看待自己的老師惶楼,一樣目光異樣的同學,一樣社會底層掙扎的爸爸诊杆,一樣純美卻不能走路的非血緣妹妹……不過歼捐,莫凡發(fā)現(xiàn)絕大多數(shù)人都只能夠主修一系魔法,自己卻是全系全能法師晨汹!"
}
}
]
}
}
多匹配查詢內(nèi)部執(zhí)行方式取決于type參數(shù)豹储,可以設置的值如下所示:
best_fields————(默認)查找匹配任何字段的文檔,但是使用最佳匹配字段的_score
most_fields————查找匹配任何字段的文檔淘这,結(jié)合每個字段的_score
cross_fields————用相同的分析器處理字段剥扣,把這些字段當做一個大字段巩剖,查找任何字段的每個單詞
phrase————在每個字段上運行短語匹配查詢,結(jié)合每個字段的_score
phrase_prefix————在每個字段上運行短語前綴匹配查詢钠怯,結(jié)合每個字段的_score
字段查詢
全文文本查詢在執(zhí)行之前會分析查詢字符串佳魔,而字段查詢只針對存儲在反向索引中的精確索引詞
這種查詢通常用于結(jié)構(gòu)化數(shù)據(jù),例如數(shù)字晦炊、日期和枚舉鞠鲜,而不是全文文本字段
1、單字段查詢
根據(jù)指定字段中包含的指定內(nèi)容查詢文檔
GET xy_order/_search
{
"query": {
"term": {
"cbid": {
"value": "4513669803378203"
}
}
}
}
結(jié)果:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "xy_order",
"_type": "order",
"_id": "4513669803378203",
"_score": 1,
"_source": {
"YwGuid": 800159902781,
"AppId": 36,
"AreaId": 1,
"AppType": 111,
"cbid": 4513669803378203,
"ItemId": 25419177209132988,
"Uuid": 3279,
"Amount": 3,
"iscount": 1,
"valueofAmount": 3,
"OrderId": "9bb713f10b8669dc5cfc4fcb6e732c00",
"Subject": "爆萌小仙:撲倒冰山冷上神",
"DeviceUid": "171840941",
"UserIp": "10.62.21.173",
"ServerIp": "10.226.143.222",
"ConsumeTime": 1514736047000,
"CreateTime": 0
}
}
]
}
}
多字段查詢
過濾文檔断国,文檔字段匹配任何提供的索引詞镊尺,例如:
GET xy_order/_search
{
"query": {
"constant_score": {
"filter": {"terms": {
"cbid": [
"4513669803378203",
"8228721504057103"
]
}},
"boost": 1.2
}
}
}
結(jié)果:
{
"took": 6,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1.2,
"hits": [
{
"_index": "xy_order",
"_type": "order",
"_id": "4513669803378203",
"_score": 1.2,
"_source": {
"YwGuid": 800159902781,
"AppId": 36,
"AreaId": 1,
"AppType": 111,
"cbid": 4513669803378203,
"ItemId": 25419177209132988,
"Uuid": 3279,
"Amount": 3,
"iscount": 1,
"valueofAmount": 3,
"OrderId": "9bb713f10b8669dc5cfc4fcb6e732c00",
"Subject": "爆萌小仙:撲倒冰山冷上神",
"DeviceUid": "171840941",
"UserIp": "10.62.21.173",
"ServerIp": "10.226.143.222",
"ConsumeTime": 1514736047000,
"CreateTime": 0
}
},
{
"_index": "xy_order",
"_type": "order",
"_id": "8228721504057103",
"_score": 1.2,
"_source": {
"YwGuid": 1039914285,
"AppId": 36,
"AreaId": 1,
"AppType": 111,
"cbid": 8228721504057103,
"ItemId": 25430287753426924,
"Uuid": 437,
"Amount": 5,
"iscount": 1,
"valueofAmount": 5,
"OrderId": "e0c07410ed29dcb26ef3d0c021a4e20b",
"Subject": "重生甜俏妻逆襲",
"DeviceUid": "183635853",
"UserIp": "10.242.15.141",
"ServerIp": "10.236.16.173",
"ConsumeTime": 1514822297000,
"CreateTime": 0
}
}
]
}
}
范圍查詢
根據(jù)字段包含的值(日期、數(shù)字或字符串)范圍查找文檔
例如查詢amount在10到20之間的文檔(數(shù)據(jù)量有點大并思,限制一下結(jié)果數(shù))
GET xy_order/_search
{
"query": {
"range": {
"Amount": {
"gte": 10,
"lte": 20
}
}
},
"from": 0
, "size": 2
}
結(jié)果
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 755,
"max_score": 1,
"hits": [
{
"_index": "xy_order",
"_type": "order",
"_id": "5392592004069701",
"_score": 1,
"_source": {
"YwGuid": 2351842566,
"AppId": 36,
"AreaId": 3,
"AppType": 111,
"cbid": 5392592004069701,
"ItemId": 14475631957226180,
"Uuid": 78,
"Amount": 10,
"iscount": 0,
"valueofAmount": 0,
"OrderId": "16fa7c7f5fa91624ced9a050ba34dd99",
"Subject": "風流太子穿越妃",
"DeviceUid": "868090023415690",
"UserIp": "223.89.154.49",
"ServerIp": "101.226.103.77",
"ConsumeTime": 1514737983000,
"CreateTime": 0
}
},
{
"_index": "xy_order",
"_type": "order",
"_id": "7519820503924203",
"_score": 1,
"_source": {
"YwGuid": 493458659,
"AppId": 36,
"AreaId": 3,
"AppType": 111,
"cbid": 7519820503924203,
"ItemId": 25418846504780308,
"Uuid": 466,
"Amount": 10,
"iscount": 1,
"valueofAmount": 5,
"OrderId": "8e69a792d8ac936adfff67fc3fb28da3",
"Subject": "網(wǎng)游之召喚王",
"DeviceUid": "865121031827908",
"UserIp": "125.69.125.205",
"ServerIp": "101.226.103.77",
"ConsumeTime": 1514738295000,
"CreateTime": 0
}
}
]
}
}
范圍查詢接受的參數(shù)如下所示:
gte:大于或等于
gt: 大于
lte: 小于或等于
lt: 小于
boost: 設置查詢的加權(quán)值庐氮,默認為1.0
復合查詢
常數(shù)得分查詢
這個查詢包含另一個查詢,并且僅返回過濾查詢中任何常數(shù)得分等于查詢加權(quán)的文檔
GET xy_order/_search
{
"query": {
"constant_score": {
"filter": {"term": {
"Amount": 100
}},
"boost": 1.0
}
}
, "from": 0
, "size": 2
}
結(jié)果:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 1,
"hits": [
{
"_index": "xy_order",
"_type": "order",
"_id": "6338525304454703",
"_score": 1,
"_source": {
"YwGuid": 120005257736,
"AppId": 36,
"AreaId": 3,
"AppType": 135,
"cbid": 6338525304454703,
"ItemId": 6338525304454703,
"Uuid": 0,
"Amount": 100,
"iscount": 0,
"valueofAmount": 100,
"OrderId": "9bf4eb7f138f54605386442f1798b1a6",
"Subject": "貞觀小農(nóng)民",
"DeviceUid": "866486024724605",
"UserIp": "220.166.141.46",
"ServerIp": "101.226.103.77",
"ConsumeTime": 1514739482000,
"CreateTime": 0
}
},
{
"_index": "xy_order",
"_type": "order",
"_id": "8542213204453803",
"_score": 1,
"_source": {
"YwGuid": 120002974277,
"AppId": 36,
"AreaId": 3,
"AppType": 135,
"cbid": 8542213204453803,
"ItemId": 8542213204453803,
"Uuid": 0,
"Amount": 100,
"iscount": 0,
"valueofAmount": 100,
"OrderId": "53d39f8542717ea87d7949b9a905e067",
"Subject": "重生農(nóng)女:妙手空間獵世子",
"DeviceUid": "868256021922124",
"UserIp": "223.81.196.70",
"ServerIp": "101.226.103.77",
"ConsumeTime": 1514784659000,
"CreateTime": 0
}
}
]
}
}
布爾查詢
獲取匹配其他查詢的布爾值的文檔宋彼,布爾查詢對應Luence的BooleanQuery弄砍,基于一個或多個布爾子句的使用,每個子句都有一類事件:
1输涕、must————必須出現(xiàn)在匹配文檔中音婶,并且會影響匹配得分
2、filter————必須出現(xiàn)在匹配文檔中莱坎,匹配得分將會被忽略
3衣式、should————應該出現(xiàn)在匹配文檔中,在布爾查詢中如果沒有must或filter子句檐什,文檔必須匹配一個或多個should子句碴卧。應該匹配的should子句的最小值可以通過minimum_should_match參數(shù)進行設置
4、must_not————必須不出現(xiàn)在匹配文檔中
布爾查詢也支持disable_coord參數(shù)(默認為false)
布爾查詢采取匹配越多越好的方式乃正,所以每個匹配的must或should子句的得分會被加在一起住册,每個文檔提供最終的_score,例如:
GET xy_order/_search
{
"query": {
"bool": {
"must": [
{"term": {
"ServerIp": {
"value": "101.226.103.77"
}
}}
]
, "must_not": [
{"term": {
"AreaId": {
"value": 5
}
}}
]
, "filter": {
"range": {
"Amount": {
"gte": 10,
"lte": 200
}
}
}
}
}
, "from": 0
, "size": 2
}
結(jié)果:
{
"took": 12,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 332,
"max_score": 0.4822159,
"hits": [
{
"_index": "xy_order",
"_type": "order",
"_id": "25382584000584301",
"_score": 0.4822159,
"_source": {
"YwGuid": 801025145875,
"AppId": 36,
"AreaId": 4,
"AppType": 111,
"cbid": 25382584000584300,
"ItemId": 16574121561281348,
"Uuid": 533,
"Amount": 14,
"iscount": 1,
"valueofAmount": 14,
"OrderId": "f58ac0fbfdd35209801dd961e3ded854",
"Subject": "重生之嫡女謀嫁",
"DeviceUid": "ea4ce27785e403d7e847710ec5aaad36",
"UserIp": "68.43.179.51",
"ServerIp": "101.226.103.77",
"ConsumeTime": 1514736119000,
"CreateTime": 0
}
},
{
"_index": "xy_order",
"_type": "order",
"_id": "25113666000167002",
"_score": 0.4822159,
"_source": {
"YwGuid": 800218465088,
"AppId": 36,
"AreaId": 3,
"AppType": 111,
"cbid": 25113666000167000,
"ItemId": 11386956139303862,
"Uuid": 286,
"Amount": 10,
"iscount": 1,
"valueofAmount": 10,
"OrderId": "0937b1098aa84b5ead4e4ab8f794a2d6",
"Subject": "邪王寵妻:醫(yī)妃休想出墻",
"DeviceUid": "862823036270694",
"UserIp": "61.158.149.245",
"ServerIp": "101.226.103.77",
"ConsumeTime": 1514736190000,
"CreateTime": 0
}
}
]
}
}
最大值獲取查詢
這個查詢通過執(zhí)行自己的子查詢生成文檔的并集,并且用文檔執(zhí)行任何子查詢的最大匹配得分作為文檔得分
查詢對應Luence的DisjunctionMaxQuery
GET xy_order/_search
{
"query": {
"dis_max": {
"tie_breaker": 0.7,
"boost": 1.2,
"queries": [
{"term": {
"Amount": {
"value": 100
}
}},
{"term": {
"AreaId": {
"value": 5
}
}}
]
}
}
, "from": 0
, "size": 5
}
boosting查詢
可以用來有效降級匹配給出的查詢結(jié)果瓮具,不像布爾查詢的NOT子句荧飞,boosting查詢?nèi)稳贿x擇包括不合需要的索引詞的文檔。但降低了它們的整體得分:
GET xy_order/_search
{
"query": {
"boosting": {
"positive": {
"term": {
"AreaId": {
"value": "1"
}
}
},
"negative": {
"term": {
"cbid": {
"value": "8824770204890803"
}
}
}
, "negative_boost": 0.2
}
}
, "from": 0
, "size": 2
}
ElasticSearch高亮顯示
ElasticSearch中的高亮顯示是來源于Lucene的功能名党,允許在一個或者多個字段上突出顯示搜索內(nèi)容叹阔,Lucene支持三種高亮顯示方式highlighter
、fast-vector-highlighter
传睹、positings-highlighter
耳幢,第一種是默認的標準類型。例如:
GET qidian_book/abstract/_search
{
"query": {
"term": {
"desc": {
"value": "mofashi"
}
}
},
"highlight": {
"fields": {"desc": {}}
}
}
結(jié)果:
{
"took": 39,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.3973951,
"hits": [
{
"_index": "qidian_book",
"_type": "abstract",
"_id": "1",
"_score": 0.3973951,
"_source": {
"title": "dao mu bi ji",
"author": "tang jia san shao",
"desc": "yi jiao xing lai,shi jie da bian shu xi de gao zhong chuan shou de shi mo fa, gao su da jia yao cheng wei yi ming chu se de mofashi, ju zhu de du shi zhi wai you dan zhe xi ji ren lei de mowu yao sou, zi ji yao cheng wei quan xiao zhu mu de mofashi"
},
"highlight": {
"desc": [
"da bian shu xi de gao zhong chuan shou de shi mo fa, gao su da jia yao cheng wei yi ming chu se de <em>mofashi</em>",
"de du shi zhi wai you dan zhe xi ji ren lei de mowu yao sou, zi ji yao cheng wei quan xiao zhu mu de <em>mofashi</em>"
]
}
}
]
}
}
結(jié)果中有高亮顯示的內(nèi)容<em>mofashi</em>,為了執(zhí)行高亮顯示蒋歌,該字段必須有實際的內(nèi)容帅掘,并且這個字段必須進行存儲委煤,就是在字段映射中store的值必須為true,不能只在內(nèi)存中修档。系統(tǒng)會自動加載_source字段并匹配相關(guān)的列
fast-vector-highlighter
highlighter
是普通的高亮顯示,而fast-vector-highlighter
高亮顯示更加有特點碧绞,如下所示:
1、快吱窝,特別是內(nèi)容大的字段讥邻,比如大于1MB
2、可定制的boundary_chars
,boundary_max_scan
和fragment_offset
3院峡、可以設置term_vector
的值為with_positions_offset
兴使,增加索引的大小
4、可以將多個字段的匹配組合成一個結(jié)果
5照激、可以權(quán)重匹配分配在不同的位置上