2021年初報(bào)名了阿里云舉辦的elasticsearch 百人大作戰(zhàn)柒昏,共同完成《ELK操作手冊(cè)》的編寫奉呛,有幸參與到了基礎(chǔ)能力編寫的部分-search基本操作渴丸,現(xiàn)整理部分內(nèi)容展現(xiàn)給大家參考學(xué)習(xí)植康。
業(yè)務(wù)背景
在2B行業(yè)效扫,對(duì)商品的搜索展示是有一定業(yè)務(wù)要求的,例如:存在合作關(guān)系的買家和供應(yīng)商才能看到供應(yīng)商店鋪的商品席覆,不存在合作關(guān)系的買家則不展示商品史辙,另外,有些商品對(duì)客戶甲展示一種價(jià)格佩伤,對(duì)另外一些客戶則展示另外一種價(jià)格髓霞,從而區(qū)分不同的會(huì)員、分組對(duì)商品價(jià)格的區(qū)別畦戒。一句話總結(jié):2B行業(yè)的商品銷售具有一定封閉性、特殊性结序。后續(xù)例子均在此背景下展開描述障斋,以方便大家更加貼近業(yè)務(wù)場(chǎng)景來(lái)熟悉elastic search 對(duì)文檔、索引徐鹤、查詢的一系列操作垃环。
定義mapping
商品字段描述如下:
goodsName: 商品名稱
skuCode:商品sku編碼
brandName:商品品牌名稱
channelType:渠道類型
shopCode: 店鋪編碼
publicPrice:售賣價(jià)格(基礎(chǔ)價(jià),對(duì)所有人開放價(jià)格)
closeUserCode:封閉會(huì)員編碼
groupPrice:分組價(jià)格返敬,其中使用嵌套類型存儲(chǔ)
boxLevelPrice:分組價(jià)格
level:分組級(jí)別
定義商品mapping
PUT my_goods_20210423
{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 1
}
},
"mappings": {
"properties": {
"goodsName": {
"type": "text",
"analyzer": "ik_smart"
},
"skuCode": {
"type": "keyword"
},
"brandName": {
"type": "keyword"
},
"channelType": {
"type": "keyword"
},
"shopCode": {
"type": "keyword"
},
"publicPrice": {
"type": "float"
},
"closeUserCode": {
"type": "text",
"analyzer": "standard"
},
"boostValue": {
"type": "keyword"
},
"groupPrice": {
"type": "nested",
"properties": {
"boxLevelPrice": {
"type": "float"
},
"level": {
"type": "text"
}
}
}
}
}
}
操作文檔
主要涉及以下幾個(gè)核心功能
1遂庄、新增
對(duì)文檔的新增操作支持以下類型
PUT /<target>/_doc/<_id>
POST /<target>/_doc/
PUT /<target>/_create/<_id>
POST /<target>/_create/<_id>
以 POST /<target>/_create/<_id>為例,以下將創(chuàng)建文檔ID為1的商品信息:
POST /my_goods_20210423/_create/1
{
"goodsName":"蘋果 51英寸 4K超高清",
"skuCode":"skuCode1",
"brandName":"蘋果",
"closeUserCode":[
"0"
],
"channelType":"cloudPlatform",
"shopCode":"sc00001",
"publicPrice":"8188.88",
"groupPrice":null,
"boxPrice":null,
"boostValue":1.8
}
ES支持批量插入劲赠,_bulk桶插入
POST my_goods_20210423/_bulk
{"index":{"_id":3}}
{"goodsName":"蘋果UA55RU7520JXXZ 53英寸 4K高清","skuCode":"skuCode3","brandName":"美國(guó)蘋果","closeUserCode":["0"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8388.88","groupPrice":null,"boxPrice":[{"boxType":"box1","boxUserCode":["htd003","uc004"],"boxPriceDetail":4388.88},{"boxType":"box2","boxUserCode":["uc005","uc0010"],"boxPriceDetail":5388.88}],"boostValue":1.2}
{"index":{"_id":4}}
{"goodsName":"山東蘋果UA55RU7520JXXZ 蘋果54英寸 5K超高清","skuCode":"skuCode4","brandName":"山東蘋果","closeUserCode":["uc001","uc002","uc003"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8488.88","groupPrice":[{"level":"level1","boxLevelPrice":"2488.88"},{"level":"level2","boxLevelPrice":"3488.88"}],"boxPrice":[{"boxType":"box1","boxUserCode":["uc004","uc005","uc006","uc001"],"boxPriceDetail":4488.88},{"boxType":"box2","boxUserCode":["htd007","htd008","htd009","uc0010"],"boxPriceDetail":5488.88}],"boostValue":1.2}
{"index":{"_id":5}}
{"goodsName":"蘋果UA55R蘋果U7蘋果520JXXZ 55英寸 5K超高清","skuCode":"skuCode5","brandName":"三星蘋果","closeUserCode":["uc001","uc002","uc003"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8488.88","groupPrice":[{"level":"level1","boxLevelPrice":"2500"},{"level":"level2","boxLevelPrice":"3500"}],"boxPrice":[{"boxType":"box1","boxUserCode":["uc004","uc005","uc006","uc001"],"boxPriceDetail":3588.88},{"boxType":"box2","boxUserCode":["htd007","htd008","htd009","uc0010"],"boxPriceDetail":5588.88}],"boostValue":1.2}
{"index":{"_id":6}}
{"goodsName":"三星UA55RU7520JXXZ 51英寸 4K超高清","skuCode":"skuCode1","brandName":"三星","closeUserCode":["0"],"channelType":"cmccPlatform","shopCode":"sc00001","publicPrice":"8188.88","groupPrice":null,"boxPrice":null,"boostValue":1.2}
{"index":{"_id":7}}
{"goodsName":"三星UA55RU7520JXXZ 52英寸 4K超高清","skuCode":"skuCode2","brandName":"三星","closeUserCode":["0"],"channelType":"cmccPlatform","shopCode":"sc00001","publicPrice":"8288.88","groupPrice":null,"boxPrice":[{"boxType":"box1","boxUserCode":["htd002"],"boxPriceDetail":4288.88}],"boostValue":1.2}
{"index":{"_id":8}}
{"goodsName":"三星UA55RU7520JXXZ 52英寸 4K超高清","skuCode":"skuCode2","brandName":"三星","closeUserCode":["uc0022"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8288.88","groupPrice":null,"boxPrice":[{"boxType":"box1","boxUserCode":["uc0022"],"boxPriceDetail":4288.88}],"boostValue":1.2}
{"index":{"_id":9}}
{"goodsName":"三星UA55RU7520JXXZ 52英寸 4K超高清","skuCode":"skuCode2","brandName":"三星","closeUserCode":["uc0022"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8288.88","groupPrice":null,"boxPrice":[{"boxType":"box1","boxUserCode":["uc0022"],"boxPriceDetail":4288.88}],"boostValue":1.2}
{"index":{"_id":10}}
{"goodsName":"三星UA55RU7520JXXZ 52英寸 4K超高清","skuCode":"skuCode2","brandName":"三星","closeUserCode":["uc0022"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8288.88","groupPrice":null,"boxPrice":[{"boxType":"box1","boxUserCode":["uc0022"],"boxPriceDetail":4288.88}],"boostValue":1.8}
2涛目、刪除
對(duì)文檔的刪除操作支持以下類型
DELETE /<index>/_doc/<_id>
刪除文檔ID為2的數(shù)據(jù):
DELETE /my_goods_20210423/_doc/2
另外,刪除操作支持帶多種條件的刪除凛澎,可以使用_delete_by_query,
如下操縱霹肝,將刪除店鋪編碼為“sc00002”的所有商品
POST /my_goods_20210423/_delete_by_query
{
"query": {
"match": {
"shopCode": "sc00002"
}
}
}
3、修改
對(duì)文檔的修改操作支持以下類型
POST /<index>/_update/<_id>
修改文檔ID為1的文檔信息
新增字段
POST /my_goods_20210423/_update/1
{
"doc": {
"shopName": "小王店鋪"
}
}
修改店鋪名稱為:“張三店鋪”
POST /my_goods_20210423/_update/1
{
"doc": {
"shopName": "張三店鋪"
}
}
{
"goodsName" : "蘋果 51英寸 4K超高清",
"skuCode" : "skuCode1",
"brandName" : "蘋果",
"closeUserCode" : [
"0"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8188.88",
"groupPrice" : null,
"boxPrice" : null,
"boostValue" : 1.8,
"shopName" : "張三店鋪"
}
另外塑煎,更新操作還可以使用_update_by_query api沫换,當(dāng)?shù)赇伨幋a為"sc00002"時(shí)修改"publicPrice"為5888.00元
插入文檔ID為2的店鋪商品信息
POST /my_goods_20210423/_create/2
{"goodsName":"蘋果 55英寸 3K超高清","skuCode":"skuCode2","brandName":"蘋果","closeUserCode":["0"],"channelType":"cloudPlatform","shopCode":"sc00002","publicPrice":"6188.88","groupPrice":null,"boxPrice":null,"boostValue":1.0}
此時(shí)查詢返回:
{
"goodsName" : "蘋果 55英寸 3K超高清",
"skuCode" : "skuCode2",
"brandName" : "蘋果",
"closeUserCode" : [
"0"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00002",
"publicPrice" : "6188.88",
"groupPrice" : null,
"boxPrice" : null,
"boostValue" : 1.0
}
更新當(dāng)?shù)赇伨幋a為"sc00002"時(shí)修改"publicPrice"為5888.00元
POST /my_goods_20210423/_update_by_query
{
"script": {
"source": "ctx._source.publicPrice=5888.00",
"lang": "painless"
},
"query": {
"term": {
"shopCode": "sc00002"
}
}
}
再次查詢結(jié)果
GET /my_goods_20210423/_source/2
{
"shopCode" : "sc00002",
"brandName" : "蘋果",
"closeUserCode" : [
"0"
],
"groupPrice" : null,
"boxPrice" : null,
"channelType" : "cloudPlatform",
"boostValue" : 1.0,
"publicPrice" : 5888.0,
"goodsName" : "蘋果 55英寸 3K超高清",
"skuCode" : "skuCode2"
}
當(dāng)有業(yè)務(wù)需要重建索引時(shí)需要用到_reindex api
索引的來(lái)源和目的地必須是已經(jīng)存在的index,、index alias最铁、或者data stream
你可以簡(jiǎn)單的將索引A reindex到索引B讯赏,當(dāng)然也可以帶條件的reindex到索引B
如下所示垮兑,將skuCode=skuCode2的商品信息reindex到索引my_goods_20210423_new中
POST _reindex
{
"source": {
"index": "my_goods_20210423",
"query": {
"match": {
"skuCode": "skuCode2"
}
}
},
"dest": {
"index": "my_goods_20210423_new"
}
}
查詢my_goods_20210423_new索引數(shù)據(jù)
GET my_goods_20210423_new/_search/
{
"took" : 5,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "my_goods_20210423_new",
"_type" : "_doc",
"_id" : "7",
"_score" : 1.0,
"_source" : {
"goodsName" : "三星UA55RU7520JXXZ 52英寸 4K超高清",
"skuCode" : "skuCode2",
"brandName" : "三星",
"closeUserCode" : [
"0"
],
"channelType" : "cmccPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8288.88",
"groupPrice" : null,
"boxPrice" : [
{
"boxType" : "box1",
"boxUserCode" : [
"htd002"
],
"boxPriceDetail" : 4288.88
}
],
"boostValue" : 1.2
}
},
{
"_index" : "my_goods_20210423_new",
"_type" : "_doc",
"_id" : "8",
"_score" : 1.0,
"_source" : {
"goodsName" : "三星UA55RU7520JXXZ 52英寸 4K超高清",
"skuCode" : "skuCode2",
"brandName" : "三星",
"closeUserCode" : [
"uc0022"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8288.88",
"groupPrice" : null,
"boxPrice" : [
{
"boxType" : "box1",
"boxUserCode" : [
"uc0022"
],
"boxPriceDetail" : 4288.88
}
],
"boostValue" : 1.2
}
},
{
"_index" : "my_goods_20210423_new",
"_type" : "_doc",
"_id" : "9",
"_score" : 1.0,
"_source" : {
"goodsName" : "三星UA55RU7520JXXZ 52英寸 4K超高清",
"skuCode" : "skuCode2",
"brandName" : "三星",
"closeUserCode" : [
"uc0022"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8288.88",
"groupPrice" : null,
"boxPrice" : [
{
"boxType" : "box1",
"boxUserCode" : [
"uc0022"
],
"boxPriceDetail" : 4288.88
}
],
"boostValue" : 1.2
}
},
{
"_index" : "my_goods_20210423_new",
"_type" : "_doc",
"_id" : "10",
"_score" : 1.0,
"_source" : {
"goodsName" : "三星UA55RU7520JXXZ 52英寸 4K超高清",
"skuCode" : "skuCode2",
"brandName" : "三星",
"closeUserCode" : [
"uc0022"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8288.88",
"groupPrice" : null,
"boxPrice" : [
{
"boxType" : "box1",
"boxUserCode" : [
"uc0022"
],
"boxPriceDetail" : 4288.88
}
],
"boostValue" : 1.8
}
}
]
}
}
4、查詢
對(duì)文檔的查詢操作支持以下類型
GET <index>/_doc/<_id>
HEAD <index>/_doc/<_id>
GET <index>/_source/<_id>
HEAD <index>/_source/<_id>
查詢文檔ID為1的文檔信息
GET /my_goods_20210423/_doc/1
查詢文檔ID為1的文檔是否存在
只判斷文檔是否存在漱挎,head返回的信息更少系枪、性能更高,滿足特殊業(yè)務(wù)場(chǎng)景使用
HEAD /my_goods_20210423/_doc/1
返回:
200 - OK
只返回文檔信息
查詢時(shí)只返回_source信息
GET /my_goods_20210423/_source/1
返回:
{
"goodsName" : "蘋果 51英寸 4K超高清",
"skuCode" : "skuCode1",
"brandName" : "蘋果",
"closeUserCode" : [
"0"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8188.88",
"groupPrice" : null,
"boxPrice" : null,
"boostValue" : 1.8
}
定制化返回參數(shù)
只獲取_source部分參數(shù)识樱,類似數(shù)據(jù)庫(kù)查詢中的指定字段嗤无,而不是select * 返回所有字段
GET my_goods_20210423/_source/1/?_source_includes=brandName,goodsName
返回:
{
"brandName" : "蘋果",
"goodsName" : "蘋果 51英寸 4K超高清"
}
查詢文檔ID為1的文檔是否存在
只判斷文檔是否存在,head返回的信息更少怜庸、性能更高当犯,滿足特殊業(yè)務(wù)場(chǎng)景使用
HEAD /my_goods_20210423/_doc/1
返回:
200 - OK
批量查詢
ES同時(shí)支持批量查詢,需要使用_mget API割疾,查詢文檔ID等于1和2的文檔信息
GET /my_goods_20210423/_mget
{
"docs": [
{
"_id": "1"
},
{
"_id": "2"
}
]
}
返回:
{
"docs" : [
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "1",
"_version" : 7,
"_seq_no" : 8,
"_primary_term" : 1,
"found" : true,
"_source" : {
"goodsName" : "蘋果 51英寸 4K超高清",
"skuCode" : "skuCode1",
"brandName" : "蘋果",
"closeUserCode" : [
"0"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8188.88",
"groupPrice" : null,
"boxPrice" : null,
"boostValue" : 1.8,
"shopName" : "張三店鋪"
}
},
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "2",
"found" : false
}
]
}
Query DSL
查詢索引包括全文本查詢嚎卫、組合查詢、結(jié)構(gòu)化查詢等宏榕,主要分為query與filter查詢拓诸。
2者查詢是有區(qū)別的:
- query查詢,用于解答文檔是否存在并且告知返回文檔與查詢條件的匹配度麻昼,返回_score評(píng)分供用戶選擇
- filter查詢奠支,只用于返回文檔是否與查詢匹配,但是不會(huì)告訴你匹配度抚芦,在做聚合查詢時(shí)filter經(jīng)常發(fā)揮更大的作用倍谜,因?yàn)闆](méi)有評(píng)分ES的處理速度就會(huì)提高,提升了整體響應(yīng)時(shí)間叉抡。同時(shí)filter可以緩存查詢結(jié)果尔崔,而query則不能緩存
使用場(chǎng)景:如果涉及到全文檢索以及評(píng)分相關(guān)業(yè)務(wù)使用query,其他場(chǎng)景推薦使用filter查詢
組合查詢
boolean查詢
boolean 查詢包含must褥民、filter季春、should、must_not
must為必須匹配并且返回評(píng)分消返,filter忽略評(píng)分载弄,should相當(dāng)于數(shù)據(jù)庫(kù)查詢中的or,must_not 為不匹配,相當(dāng)于不等于
查詢:店鋪編碼=sc00001 且渠道channelType=cloudPlatform 且publicPrice價(jià)格區(qū)間不在8288-8888之間或者品牌包含蘋果
POST /my_goods_20210423/_search
{
"query": {
"bool": {
"must": {
"term":{
"shopCode":"sc00001"
}
},
"filter": {
"term": {
"channelType": "cloudPlatform"
}
},
"must_not": [
{
"range": {
"publicPrice": {
"gte": 8288,
"lte": 8888
}
}
}
],
"should": [
{
"term": {
"brandName": {
"value": "蘋果"
}
}
}
],
"minimum_should_match" : 1
}
}
}
boosting 查詢
boosting用于控制評(píng)分相關(guān)度相關(guān)侦副,可以提升評(píng)分也可以降低評(píng)分
POST /my_goods_20210423/_search
{
"query": {
"boosting": {
"positive": {
"term": {
"skuCode": {
"value": "skuCode1"
}
}
},
"negative": {
"term": {
"goodsName": {
"value": "三星"
}
}
},
"negative_boost": 1
}
}
}
此時(shí)設(shè)置的negative_boost=1侦锯,不提升也不降低,返回:
"hits" : [
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.3862942,
"_source" : {
"goodsName" : "蘋果 51英寸 4K超高清",
"skuCode" : "skuCode1",
"brandName" : "蘋果",
"closeUserCode" : [
"0"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8188.88",
"groupPrice" : null,
"boxPrice" : null,
"boostValue" : 1.8,
"shopName" : "張三店鋪"
}
},
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "6",
"_score" : 1.3862942,
"_source" : {
"goodsName" : "三星UA55RU7520JXXZ 51英寸 4K超高清",
"skuCode" : "skuCode1",
"brandName" : "三星",
"closeUserCode" : [
"0"
],
"channelType" : "cmccPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8188.88",
"groupPrice" : null,
"boxPrice" : null,
"boostValue" : 1.2
}
}
]
可以看到2條文檔記錄評(píng)分一致:"_score" : 1.3862942
當(dāng)我們修改 "negative_boost": 0.2時(shí)秦驯,此時(shí)返回(省略部分無(wú)關(guān)字段):
"hits" : [
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.3862942,
"_source" : {
"goodsName" : "蘋果 51英寸 4K超高清",
"skuCode" : "skuCode1",
"brandName" : "蘋果",
"closeUserCode" : [
"0"
],
"channelType" : "cloudPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8188.88",
"groupPrice" : null,
"boxPrice" : null,
"boostValue" : 1.8,
"shopName" : "張三店鋪"
}
},
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "6",
"_score" : 0.27725884,
"_source" : {
"goodsName" : "三星UA55RU7520JXXZ 51英寸 4K超高清",
"skuCode" : "skuCode1",
"brandName" : "三星",
"closeUserCode" : [
"0"
],
"channelType" : "cmccPlatform",
"shopCode" : "sc00001",
"publicPrice" : "8188.88",
"groupPrice" : null,
"boxPrice" : null,
"boostValue" : 1.2
}
}
]
此時(shí)發(fā)現(xiàn)文檔ID=6的評(píng)分下降到_score" : 0.27725884尺碰,因?yàn)樵趎egative命中了查詢條件,negative_boost在0到1之間時(shí),用于降低評(píng)分亲桥,相反洛心,大于1用于提升評(píng)分
Constant score query 查詢
當(dāng)查詢不關(guān)心TF(詞頻)時(shí),就可以使用constant score query
POST /my_goods_20210423/_search
{
"query": {
"constant_score": {
"filter": {
"term": {
"goodsName": "蘋果"
}
},
"boost": 1.2
}
}
}
返回(省略部分無(wú)關(guān)字段):
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.2,
"_source" : {
"goodsName" : "蘋果UA55RU7520JXXZ 53英寸 4K高清"
}
},
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "4",
"_score" : 1.2,
"_source" : {
"goodsName" : "山東蘋果UA55RU7520JXXZ 蘋果54英寸 5K超高清"
}
}
}
可以看到题篷,文檔ID=3的評(píng)分和文檔ID=4的評(píng)分一樣词身,但是ID=4的匹配相關(guān)度更高,這是由于我們忽略了詞頻對(duì)打分的影響番枚。
Disjunction max query 查詢
Disjunction 查詢也被理解為分離最大化查詢法严,指的是: 將任何與任一查詢匹配的文檔作為結(jié)果返回,但只將最佳匹配的評(píng)分作為查詢的評(píng)分結(jié)果返回葫笼,例如查詢商品名稱和品牌名稱中包含“蘋果”的信息深啤,當(dāng)品牌的評(píng)分高于商品名稱時(shí),則返回品牌的評(píng)分做為總評(píng)分(忽略tie_breaker緩沖)
GET /my_goods_20210423/_search
{
"query": {
"dis_max": {
"tie_breaker": 0.7,
"boost": 1.2,
"queries": [
{
"term": {
"goodsName": {
"value": "蘋果"
}
}
},
{
"term": {
"brandName": {
"value": "蘋果"
}
}
}
]
}
}
}
返回結(jié)果(忽略無(wú)關(guān)字段):
"max_score" : 3.0150018,
"hits" : [
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "1",
"_score" : 3.0150018,
"_source" : {
"goodsName" : "蘋果 51英寸 4K超高清",
"brandName" : "蘋果"
}
},
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "5",
"_score" : 1.3465583,
"_source" : {
"goodsName" : "蘋果UA55R蘋果U7蘋果520JXXZ 55英寸 5K超高清",
"brandName" : "三星蘋果"
}
},
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "4",
"_score" : 1.2337791,
"_source" : {
"goodsName" : "山東蘋果UA55RU7520JXXZ 蘋果54英寸 5K超高清",
"brandName" : "山東蘋果"
}
},
分析:
- id=1的記錄路星,由于品牌只包含“蘋果”2字溯街,ES認(rèn)為這種匹配度更高,所以此條記錄評(píng)分排在第一位
- id=5的記錄洋丐,由于品牌中和ID=4的記錄都包含蘋果且字?jǐn)?shù)一樣呈昔,此時(shí)就要看goodsName包含蘋果的詞頻數(shù)量了,ID=5的品牌中友绝,“蘋果”出現(xiàn)了3次堤尾,而ID=4的值出現(xiàn)了2次,所以評(píng)分沒(méi)有ID=5的高迁客,符合我們的預(yù)期結(jié)果哀峻。
- tie_breaker字段做什么用呢?它是起到了緩沖的作用(取值范圍:0到1之間)哲泊,Disjunction查詢會(huì)將匹配度最高的字段得分做為整個(gè)文檔的得分返回,這種情況其他字段就不起作用了催蝗,難免有點(diǎn)走極端切威,此時(shí)就需要tie_breaker來(lái)做緩存,提升其他字段的影響力丙号,最終的結(jié)果:brandName評(píng)分+goodsName評(píng)分*tie_breaker先朦。作為總評(píng)分返回
Function score query 查詢
Function score 允許你控制查詢?cè)u(píng)分,是用來(lái)控制評(píng)分過(guò)程的終極武器犬缨。最高效的用法是用過(guò)濾器對(duì)結(jié)果的子集應(yīng)用不同的函數(shù)喳魏,同時(shí)運(yùn)用了filter的緩存并且達(dá)到了控制評(píng)分的過(guò)程。
我們想讓山東的蘋果搜索出現(xiàn)美國(guó)蘋果之前怀薛,查詢商品名稱包含“蘋果”刺彩,當(dāng)品牌中包含“美國(guó)”時(shí),權(quán)重設(shè)置為2,當(dāng)出現(xiàn)“山東”時(shí)创倔,權(quán)重設(shè)置為40
GET /my_goods_20210423/_search
{
"query": {
"function_score": {
"query": {
"term": {
"goodsName": {
"value": "蘋果"
}
}
},
"boost": 2,
"functions": [
{
"filter": {
"match":{
"brandName":"美國(guó)"
}
},
"random_score": {
},
"weight": 2
},
{
"filter": {
"match":{
"brandName":"山東"
}
},
"weight": 40
}
],
"max_boost": 60,
"score_mode": "max",
"boost_mode": "multiply",
"min_score": 2
}
}
}
返回主要信息:
"max_score" : 2.2442641,
"hits" : [
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "4",
"_score" : 2.0562985,
"_source" : {
"goodsName" : "山東蘋果UA55RU7520JXXZ 蘋果54英寸 5K超高清",
"brandName" : "山東蘋果"
}
},
{
"_index" : "my_goods_20210423",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.7582327,
"_source" : {
"goodsName" : "蘋果UA55RU7520JXXZ 53英寸 4K高清",
"brandName" : "美國(guó)蘋果",
}
}
]
解釋幾個(gè)參數(shù):
- score_mode
multiply:默認(rèn)嗡害,分?jǐn)?shù)相乘
sum:分?jǐn)?shù)求和
avg:平均分?jǐn)?shù)
first:第一個(gè) function的分?jǐn)?shù)
max:使用評(píng)分最大的分?jǐn)?shù)
min:使用評(píng)分最小的分?jǐn)?shù)
avg舉例,如果2個(gè)函數(shù)返回的分?jǐn)?shù)為1和2畦攘,并且它們的權(quán)重分別為3和4霸妹,則他們的評(píng)分為:(13+24)/(3+4)
其他詳解請(qǐng)參考官方score-functions詳解
全文檢索
match 查詢
match 查詢是一種標(biāo)準(zhǔn)的查詢,示例如下:
GET /my_goods_20210423/_search
{
"query": {
"match": {
"goodsName": "蘋果 高清 英寸"
}
}
}
match查詢是一種boolean類型的查詢知押,可以使用"operator"來(lái)控制boolean 字句叹螟,operator包含 and 和 or(默認(rèn)為 or)
GET /my_goods_20210423/_search
{
"query": {
"match": {
"goodsName": {
"query": "蘋果 高清 英寸",
"operator": "and"
}
}
}
}
返回結(jié)果:
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
}
}
命中為0,因?yàn)闆](méi)有標(biāo)題中包含“蘋果 高清 英寸”詞組的商品信息
match boolean prefix query
添加2條商品名稱是因?yàn)榈臏y(cè)試數(shù)據(jù)台盯,方便測(cè)試
POST my_goods_20210423/_bulk
{"index":{"_id":11}}
{"goodsName":"apple goods test","skuCode":"skuCode3","brandName":"美國(guó)蘋果","closeUserCode":["0"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8388.88","groupPrice":null,"boxPrice":[{"boxType":"box1","boxUserCode":["htd003","uc004"],"boxPriceDetail":4388.88},{"boxType":"box2","boxUserCode":["uc005","uc0010"],"boxPriceDetail":5388.88}],"boostValue":1.2}
{"index":{"_id":12}}
{"goodsName":"apple goods online","skuCode":"skuCode3","brandName":"美國(guó)蘋果","closeUserCode":["0"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8388.88","groupPrice":null,"boxPrice":[{"boxType":"box1","boxUserCode":["htd003","uc004"],"boxPriceDetail":4388.88},{"boxType":"box2","boxUserCode":["uc005","uc0010"],"boxPriceDetail":5388.88}],"boostValue":1.2}
GET /my_goods_20210423/_search
{
"query": {
"match_bool_prefix": {
"goodsName": "apple goods t"
}
}
}
2條剛添加的商品都被查詢到了罢绽,match_bool_prefix原理就相當(dāng)于把詞組分開后的boolean查詢,轉(zhuǎn)換后類似如下查詢:
GET /my_goods_20210423/_search
{
"query": {
"bool" : {
"should": [
{ "term": { "goodsName": "apple" }},
{ "term": { "goodsName": "goods" }},
{ "prefix": { "goodsName": "t"}}
]
}
}
}
match prefix query
用于匹配索引中是否存在所輸入的查詢條件數(shù)據(jù)
GET /my_goods_20210423/_search
{
"query": {
"match_phrase": {
"goodsName": "apple"
}
}
}
比較match_phrase與match區(qū)別爷恳,match_phrase會(huì)將查詢條件的中的信息看做一個(gè)整體有缆,不做分詞去查詢,當(dāng)然你也可以指定分詞類型温亲,而match會(huì)將查詢中的條件做分詞處理后棚壁,再去做查詢
#查詢不到任何數(shù)據(jù),因?yàn)椴淮嬖?goods t'的詞組
GET /my_goods_20210423/_search
{
"query": {
"match_phrase": {
"goodsName": "goods t"
}
}
}
#能查詢到數(shù)據(jù)栈虚,因?yàn)槲臋n中包含goods和t的詞組
GET /my_goods_20210423/_search
{
"query": {
"match": {
"goodsName": "goods t"
}
}
}
match phrase prefix query
返回文檔包含給定查詢條件的文檔袖外,文檔中必須包含給定條件的內(nèi)容且是按照順序的,如"apple goods t" 魂务,商品名稱包含"apple goods test"的數(shù)據(jù)將被查詢到返回曼验。
新增一條測(cè)試數(shù)據(jù)
POST my_goods_20210423/_bulk
{"index":{"_id":13}}
{"goodsName":"apple and goods product ","skuCode":"skuCode3","brandName":"美國(guó)蘋果","closeUserCode":["0"],"channelType":"cloudPlatform","shopCode":"sc00001","publicPrice":"8388.88","groupPrice":null,"boxPrice":[{"boxType":"box1","boxUserCode":["htd003","uc004"],"boxPriceDetail":4388.88},{"boxType":"box2","boxUserCode":["uc005","uc0010"],"boxPriceDetail":5388.88}],"boostValue":1.2}
#只返回goodsName : apple goods test的數(shù)據(jù)
GET /my_goods_20210423/_search
{
"query": {
"match_phrase_prefix": {
"goodsName": "apple goods t"
}
}
}
總結(jié)比較match這四種查詢
Multi-match
多字段匹配,可以在多個(gè)字段中匹配查詢相關(guān)信息粘姜,通過(guò)type參數(shù)可以調(diào)整結(jié)果集
#查詢商品名稱和品牌名稱中包含蘋果的文檔信息
POST /my_goods_20210423/_search
{
"query": {
"multi_match": {
"query": "蘋果",
"type": "best_fields",
"fields": ["goodsName","brandName"],
"tie_breaker": 0.3
}
}
}
type參數(shù)類型詳解:
- best_fields :默認(rèn)鬓照,匹配fields,將評(píng)分最高的分?jǐn)?shù)做為整個(gè)查詢的分?jǐn)?shù)返回
- most_fields:查詢匹配的文檔孤紧,并且返回各個(gè)字段的分?jǐn)?shù)之和的平均值
- cross_fields:跨字段匹配豺裆,匹配多個(gè)字段中是否包含查詢?cè)~組
- phrase:以match_phrase方式運(yùn)行查詢,并返回最佳匹配的評(píng)分做為總評(píng)分
- phrase_prefix:以match_phrase_prefix方式運(yùn)行查詢号显,并返回最佳匹配的評(píng)分做為總評(píng)分
- bool_prefix:在每個(gè)字段上運(yùn)行match_bool_prefix查詢并組合每個(gè)字段的評(píng)分臭猜,詳情參考bool_prefix
以cross_fields為例進(jìn)行實(shí)戰(zhàn)講解
#插入測(cè)試數(shù)據(jù)
PUT my_shop
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1
},
"mappings": {
"properties": {
"firstName":{
"type":"text"
},
"lastName":{
"type":"text"
}
}
}
}
POST my_shop/_bulk
{"index":{"_id":1}}
{"first_name":"Will","last_name":"Smith","age":25}
{"index":{"_id":2}}
{"first_name":"Smith","last_name":"hello","age":21}
{"index":{"_id":3}}
{"first_name":"Will","last_name":"hello","age":20}
#查詢姓名為Will Smith的信息
GET /my_shop/_search
{
"query": {
"multi_match" : {
"query": "Will Smith",
"type": "cross_fields",
"fields": [ "first_name^2", "last_name" ],
"operator": "and"
}
}
}
#返回
"max_score" : 1.9208363,
"hits" : [
{
"_index" : "my_shop",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.9208363,
"_source" : {
"first_name" : "Will",
"last_name" : "Smith",
"age" : 25
}
}
]
另外,first_name提升了權(quán)重押蚤,默認(rèn)為1
Term-level查詢
可以使用term-level 查詢結(jié)構(gòu)化數(shù)據(jù)蔑歌,結(jié)構(gòu)化數(shù)據(jù)如日期范圍、IP地址揽碘、價(jià)格等,下面分別演示在業(yè)務(wù)場(chǎng)景中的實(shí)際使用
- exists查詢
返回包含字段索引值的文檔
#返回包含goodsName字段的索引文檔
GET /my_goods_20210423/_search
{
"query": {
"exists": {
"field": "goodsName"
}
}
}
- fuzzy查詢
返回包含與搜索字詞相似的字詞的文檔,可以用于查詢糾錯(cuò)功能
#以官網(wǎng)例子舉例說(shuō)明
POST /my_index/_bulk
{ "index": { "_id": 1 }}
{ "text": "Surprise me!"}
{ "index": { "_id": 2 }}
{ "text": "That was surprising."}
{ "index": { "_id": 3 }}
{ "text": "I wasn't surprised."}
GET /my_index/_search
{
"query": {
"fuzzy": {
"text": {
"value": "surprize",
"prefix_length": 1
}
}
}
}
#發(fā)揮
"hits" : [
{
"_index" : "my_index",
"_type" : "my_type",
"_id" : "1",
"_score" : 0.9559981,
"_source" : {
"text" : "Surprise me!"
}
},
{
"_index" : "my_index",
"_type" : "my_type",
"_id" : "3",
"_score" : 0.69983494,
"_source" : {
"text" : "I wasn't surprised."
}
}
默認(rèn)如果不設(shè)置次屠,prefix_length就是2
- surprising 錯(cuò)誤3個(gè)位置园匹,不能糾錯(cuò)
- surprize 拼寫錯(cuò)誤,s->z,錯(cuò)誤在一個(gè)位置帅矗,在2個(gè)位置的糾錯(cuò)范圍之內(nèi)
為提高性能偎肃,可以設(shè)置max_expansions,將限制產(chǎn)生模糊文檔的個(gè)數(shù)浑此,
另外累颂,prefix_length不宜設(shè)置過(guò)大,也將影響查詢性能凛俱,同時(shí)錯(cuò)誤過(guò)多也將導(dǎo)致查詢結(jié)果不是用戶期望的紊馏。
- ids查詢
范圍文檔包含ID的文檔信息
GET /my_goods_20210423/_search
{
"query": {
"ids" : {
"values" : ["1", "4", "5"]
}
}
}
- prefix查詢
返回在提供的字段中包含特定前綴的文檔
GET /my_shop_test/_search
{
"query": {
"prefix": {
"shopName": {
"value": "bo"
}
}
}
}
#返回
"hits" : [
{
"_index" : "my_shop_test",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"shopName" : "box",
"shopCode" : "Smith"
}
},
{
"_index" : "my_shop_test",
"_type" : "_doc",
"_id" : "4",
"_score" : 1.0,
"_source" : {
"shopName" : "booex",
"shopCode" : "act"
}
}
]
- range查詢
rand查詢類似數(shù)據(jù)庫(kù)中的 大于、小于范圍查詢
GET my_goods_20210423/_search
{
"query": {
"range": {
"publicPrice": {
"gte": 2000,
"lte": 8488
}
}
}
}
- gt :大于
- gte:大于等于
- lt:小于
- lte:小于等于
- regexp查詢
正則表達(dá)式查詢蒲犬,查詢店鋪編碼以's'開頭朱监,中間包括任何字符以及長(zhǎng)度并且以'1'結(jié)尾的數(shù)據(jù)
GET my_goods_20210423/_search
{
"query": {
"regexp": {
"shopCode": {
"value": "s.*1",
"flags": "ALL",
"case_insensitive": true,
"max_determinized_states": 10000,
"rewrite": "constant_score"
}
}
}
}
- term查詢
#返回確切的文檔內(nèi)容,避免對(duì)text字段類型使用term
GET my_goods_20210423/_search
{
"query": {
"term": {
"brandName": {
"value": "三星",
"boost": 1.0
}
}
}
}
- terms查詢
terms返回一個(gè)或多個(gè)包含精確查詢條件的文檔信息
GET /my_goods_20210423/_search
{
"query": {
"terms": {
"brandName": [ "美國(guó)", "三星" ],
"boost": 1.0
}
}
}
- terms_set查詢
返回最小精確匹配成功的文檔信息原叮,terms_set類似terms 查詢赫编,只不過(guò)terms_se多定義了返回最小匹配的數(shù)量
#新定義商品信息
PUT /my_goods_info
{
"mappings": {
"properties": {
"goodsName": {
"type": "keyword"
},
"sale_property": {
"type": "keyword"
},
"required_matches": {
"type": "long"
}
}
}
}
#添加3條商品測(cè)試數(shù)據(jù)
#銷售屬性 白色、64G奋隶、標(biāo)品
PUT /my_goods_info/_doc/1?refresh
{
"name": "apple",
"sale_property": [ "white", "64","standard" ],
"required_matches": 2
}
#黑色擂送、32G、非標(biāo)品
PUT /my_goods_info/_doc/2?refresh
{
"name": "apple",
"sale_property": [ "black", "32","no standard" ],
"required_matches": 2
}
#黑色唯欣、64 非標(biāo)品
PUT /my_goods_info/_doc/3?refresh
{
"name": "apple",
"sale_property": [ "black", "64","no standard" ],
"required_matches": 2
}
#查詢
GET /my_goods_info/_search
{
"query": {
"terms_set": {
"sale_property": {
"terms": [ "white", "64"],
"minimum_should_match_field": "required_matches"
}
}
}
}
#返回
"hits" : [
{
"_index" : "my_goods_info",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.1149836,
"_source" : {
"name" : "apple",
"sale_property" : [
"white",
"64",
"standard"
],
"required_matches" : 2
}
}
]
- wildcard查詢
返回包含與通配符模式匹配的術(shù)語(yǔ)的文檔
#返回
GET /my_goods_20210423/_search
{
"query": {
"wildcard": {
"shopCode": {
"value": "sc*1",
"boost": 1.0,
"rewrite": "constant_score"
}
}
}
}