es里面的操作,主要分為兩種,一種寫入(增刪改)纪挎,另一種是查詢(搜索)
我們分別要識別出來期贫,哪些寫入操作性能比較慢,哪些查詢操作性能比較慢异袄,先要識別出來有性能問題的這些慢查詢通砍,慢寫入,然后才能去考慮如何優(yōu)化寫入的性能烤蜕,如何優(yōu)化搜索的性能
搜索慢查詢?nèi)罩?/p>
無論是慢查詢?nèi)罩痉馑铮€是慢寫入日志,都是針對shard級別的讽营,因為大家應該知道虎忌,無論你是執(zhí)行增刪改,還是執(zhí)行搜索橱鹏,都是對某個數(shù)據(jù)執(zhí)行寫入或者是搜索膜蠢,其實都是到某個shard上面去執(zhí)行的
shard上面執(zhí)行的慢的寫入或者是搜索,都會記錄在針對這個shard的日志中
閾值的意思蚀瘸,就是說狡蝶,什么叫做慢?搜索贮勃,5s叫做慢贪惹?還是10s叫做慢?或者是1s叫做慢寂嘉?
比如說奏瞬,你設(shè)置一個閾值,5s就是搜索的閾值泉孩,5s就叫做慢硼端,那么一旦一個搜索請求超過了5s之后,就會記錄一條慢搜索日志到日志文件中
shard level的搜索慢查詢?nèi)罩驹幔x將搜索性能較慢的查詢寫入一個專門的日志文件中珍昨。可以針對query phase和fetch phase單獨設(shè)置慢查詢的閾值句喷,而具體的慢查詢閾值設(shè)置如下所示:
在elasticsearch.yml中镣典,設(shè)置
index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 5s
index.search.slowlog.threshold.query.debug: 2s
index.search.slowlog.threshold.query.trace: 500ms
index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 800ms
index.search.slowlog.threshold.fetch.debug: 500ms
index.search.slowlog.threshold.fetch.trace: 200ms
而慢查詢?nèi)罩揪唧w的格式,都是在log4j2.properties中配置的唾琼,比如下面的配置:
appender.index_search_slowlog_rolling.type = RollingFile
appender.index_search_slowlog_rolling.name = index_search_slowlog_rolling
appender.index_search_slowlog_rolling.fileName = ${sys:es.logs}_index_search_slowlog.log
appender.index_search_slowlog_rolling.layout.type = PatternLayout
appender.index_search_slowlog_rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %.10000m%n
appender.index_search_slowlog_rolling.filePattern = ${sys:es.logs}_index_search_slowlog-%d{yyyy-MM-dd}.log
appender.index_search_slowlog_rolling.policies.type = Policies
appender.index_search_slowlog_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.index_search_slowlog_rolling.policies.time.interval = 1
appender.index_search_slowlog_rolling.policies.time.modulate = true
logger.index_search_slowlog_rolling.name = index.search.slowlog
logger.index_search_slowlog_rolling.level = trace
logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling.ref = index_search_slowlog_rolling
logger.index_search_slowlog_rolling.additivity = false
索引慢寫入日志
可以用如下的配置來設(shè)置索引寫入慢日志的閾值:
index.indexing.slowlog.threshold.index.warn: 10s
index.indexing.slowlog.threshold.index.info: 5s
index.indexing.slowlog.threshold.index.debug: 2s
index.indexing.slowlog.threshold.index.trace: 500ms
index.indexing.slowlog.level: info
index.indexing.slowlog.source: 1000
用下面的log4j.properties配置就可以設(shè)置索引慢寫入日志的格式:
appender.index_indexing_slowlog_rolling.type = RollingFile
appender.index_indexing_slowlog_rolling.name = index_indexing_slowlog_rolling
appender.index_indexing_slowlog_rolling.fileName = ${sys:es.logs}_index_indexing_slowlog.log
appender.index_indexing_slowlog_rolling.layout.type = PatternLayout
appender.index_indexing_slowlog_rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %marker%.10000m%n
appender.index_indexing_slowlog_rolling.filePattern = ${sys:es.logs}_index_indexing_slowlog-%d{yyyy-MM-dd}.log
appender.index_indexing_slowlog_rolling.policies.type = Policies
appender.index_indexing_slowlog_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.index_indexing_slowlog_rolling.policies.time.interval = 1
appender.index_indexing_slowlog_rolling.policies.time.modulate = true
logger.index_indexing_slowlog.name = index.indexing.slowlog.index
logger.index_indexing_slowlog.level = trace
logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling.ref = index_indexing_slowlog_rolling
logger.index_indexing_slowlog.additivity = false
把這個東西記錄好了以后兄春,就可以什么呢?es集群運維管理員锡溯,就需要經(jīng)常每天去看看赶舆,正常情況下哑姚,慢查詢應該是比較少數(shù)的。所以比如你每天檢查一次芜茵,如果發(fā)現(xiàn)某個查詢特別慢叙量,就要通知寫這個查詢的RD,開發(fā)人員夕晓,讓他們?nèi)?yōu)化一下性能宛乃。