Elasticsearch 在競價(jià)廣告中的檢索使用
一疾牲、廣告定向簡述
1.1 在競價(jià)廣告中的定向條件往往如下所示
- ad1 定向?yàn)榈赜虮本?上海,廣州,深圳,18~28歲的旅游,健身行業(yè)男性,并且要求適用的操作系統(tǒng)為ios,android,廣告出價(jià)5塊
- ad2 定向?yàn)榈赜虺啥荚?8~28歲的健身行業(yè)男性,并且要求適用的操作系統(tǒng)為ios和mac,廣告出價(jià)4.8塊
- ad3 定向?yàn)樵?8~38歲的男性,并且要求適用的操作系統(tǒng)為android,廣告出價(jià)5.7塊
- ad4 定向?yàn)樵?8~38歲,并且要求適用的操作系統(tǒng)為ios,廣告出價(jià)5.2塊
1.2 角色對應(yīng)廣告分析
- 角色1: 北京,女性,健身行業(yè),操作系統(tǒng)ios
- 角色2: 廣州,男性,18~28歲,旅游行業(yè),操作系統(tǒng)為ios
- 角色3: 成都,女性,28~38歲,健身行業(yè),操作系統(tǒng)為ios
- 角色4: 成都,男性,28~38歲,健身行業(yè),操作系統(tǒng)為ios
1.3. 認(rèn)真分析后得出下面每個(gè)角色可以推送的廣告如下
- 角色1: ad4
- 角色2: ad1,ad4
- 角色3: ad4
- 角色4: ad2,ad4
往往傳統(tǒng)數(shù)據(jù)庫無法滿足上述的查詢時(shí)延, 大廠往往又開發(fā)自己的倒排索引系統(tǒng), 為了減少成本, 可以使用elasticsearch的布爾查詢.
二冀惭、使用elasticsearch 查詢實(shí)時(shí)查詢廣告
2.1. mysql中, 如果要查詢某個(gè)用戶滿足的廣告條件如下可整理為表達(dá)式
[（不存在性別定向）|| （存在性別定向且滿足條件）]
&& [（不存在年齡定向）|| （存在年齡定向且滿足條件）]
&& [（不存在標(biāo)簽定向）|| （存在標(biāo)簽定向且滿足條件）]
&& [（不存在地域定向）|| （存在地域定向且滿足條件）]
&& [（不存在操作系統(tǒng)定向）|| （存在操作系統(tǒng)定向且滿足條件）]
2.2 準(zhǔn)備工具: postman 或者支持curl命令行, 一臺安裝了docker的機(jī)器
2.2.1 拉取es鏡像,并且運(yùn)行起來
docker pull docker.io/elasticsearch:7.1.1
docker run -d --name es1 -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" b0e9f9f047e6
2.2.2 執(zhí)行命令
postman或者命令行中執(zhí)行 curl --location --request GET 'http://192.168.17.77:9200'
如果返回下面的文檔說明你安裝單機(jī)版本的elasticsearch已經(jīng)安裝完成
{
"name": "cc51cc2a79ce",
"cluster_name": "docker-cluster",
"cluster_uuid": "BveCHkuVTtWwr-rcWDmTpg",
"version": {
"number": "7.1.1",
"build_flavor": "default",
"build_type": "docker",
"build_hash": "7a013de",
"build_date": "2019-05-23T14:04:00.380842Z",
"build_snapshot": false,
"lucene_version": "8.0.0",
"minimum_wire_compatibility_version": "6.8.0",
"minimum_index_compatibility_version": "6.0.0-beta1"
},
"tagline": "You Know, for Search"
}
2.3. 建立廣告索引,查詢廣告位對應(yīng)廣告
通常用戶訪問app拉取廣告是以廣告位為基準(zhǔn), 該廣告位下面有n個(gè)帶有定向條件的廣告.那么 查詢條件就是廣告位id,底價(jià)+以及用戶自身的屬性
創(chuàng)建廣告位id為100的索引
curl --location --request PUT 'http://192.168.17.77:9200/posfor100'
- 增加該索引對應(yīng)的數(shù)據(jù)(類型于mysql的行數(shù)據(jù))
北上廣深,成都分別映射為 1,2,3,4,5
男女映射為1,2
操作系統(tǒng)ios, android,mac 映射為1,2,3
行業(yè)旅游,健身分別映射為 1,2
年齡18~28 映射為2
插入對應(yīng)的4條數(shù)據(jù),假設(shè)上面4個(gè)廣告對應(yīng)的id為 101,102,103,104
curl --location --request POST 'http://192.168.17.77:9200/posfor100/_doc/101' \
--header 'Content-Type: application/json' \
--data-raw '{"city":[1,2,3,4],"ageRange":[2],"gender":[1],"os":[1,2],"industry":[1,2],"price":5}'
curl --location --request POST 'http://192.168.17.77:9200/posfor100/_doc/102' \
--header 'Content-Type: application/json' \
--data-raw '{"city":[5],"ageRange":[2],"gender":[1],"os":[1,3],"industry":[2],"price":4.8}'
curl --location --request POST 'http://192.168.17.77:9200/posfor100/_doc/103' \
--header 'Content-Type: application/json' \
--data-raw '{"ageRange":[2],"gender":[1],"os":[2],"price":5.7}'
curl --location --request POST 'http://192.168.17.77:9200/posfor100/_doc/104' \
--header 'Content-Type: application/json' \
--data-raw '{"ageRange":[2],"os":[1],"price":5.2}'
- 假設(shè)該廣告位100的底價(jià)為3塊錢,使用布爾查詢
角色1 對應(yīng)的查詢
curl --location --request GET 'http://192.168.17.77:9200/posfor100/_search' \
--header 'Content-Type: application/json' \
--data-raw '{"query":{"bool":{"filter":[{"bool":{"should":[{"term":{"gender":{"value":2,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"gender","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"should":[{"term":{"os":{"value":1,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"os","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"should":[{"term":{"city":{"value":1,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"city","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"should":[{"term":{"industry":{"value":2,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"industry","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"should":[{"term":{"ageRange":{"value":2,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"ageRange","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"filter":[{"range":{"price":{"from":3.0,"to":null,"include_lower":true,"include_upper":true,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}}'
得到查詢條件如下, 獲得了id 104的廣告,即是廣告4
{
"took": 403,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.0,
"hits": [
{
"_index": "posfor100",
"_type": "_doc",
"_id": "104",
"_score": 0.0,
"_source": {
"ageRange": [
2
],
"os": [
1
],
"price": 5.2
}
}
]
}
}
角色2對應(yīng)的查詢
curl --location --request GET 'http://192.168.17.77:9200/posfor100/_search' \
--header 'Content-Type: application/json' \
--data-raw '{"query":{"bool":{"filter":[{"bool":{"should":[{"term":{"gender":{"value":1,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"gender","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"should":[{"term":{"os":{"value":1,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"os","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"should":[{"term":{"city":{"value":4,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"city","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"should":[{"term":{"industry":{"value":1,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"industry","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"should":[{"term":{"ageRange":{"value":2,"boost":1.0}}},{"bool":{"must_not":[{"exists":{"field":"ageRange","boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"minimum_should_match":"1","boost":1.0}},{"bool":{"filter":[{"range":{"price":{"from":3.0,"to":null,"include_lower":true,"include_upper":true,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}}'
得到的結(jié)果
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.0,
"hits": [
{
"_index": "posfor100",
"_type": "_doc",
"_id": "101",
"_score": 0.0,
"_source": {
"city": [
1,
2,
3,
4
],
"ageRange": [
2
],
"gender": [
1
],
"os": [
1,
2
],
"industry": [
1,
2
],
"price": 5
}
},
{
"_index": "posfor100",
"_type": "_doc",
"_id": "104",
"_score": 0.0,
"_source": {
"ageRange": [
2
],
"os": [
1
],
"price": 5.2
}
}
]
}
}
由上可得角色1獲取到ad4,角色2獲取到ad1,ad4, 和我們最初得到的結(jié)論是一樣的2, 剩余角色3,角色4對應(yīng)的廣告,請各位親自己動手驗(yàn)證.
三幻梯、小結(jié)
- 學(xué)習(xí)到定向條件可以通過es的布爾表達(dá)式來檢索
參考如下
