越來越多的企業(yè)已經(jīng)采用ELK解決方案來對(duì)其公司產(chǎn)生的日志進(jìn)行分析，筆者最近著手在生產(chǎn)環(huán)境部署自己的ELK stack裕坊，本文介紹ELK中elasticsearch5.2集群的實(shí)現(xiàn)舟肉。
一修噪、環(huán)境準(zhǔn)備
1、系統(tǒng)：CentOS 6.8
ip及角色：192.168.1.121（master node） 192.168.122（data node） 192.168.123(client node)
2路媚、JDK
# 筆者使用的jdk版本jdk-8u121-linux-x64.rpm黄琼，下面給出可以直接下載的JDK版本
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u77-b02/jdk-8u77-linux-x64.rpm"
3、elasticsearch安裝
# 分別在三臺(tái)服務(wù)器上安裝elastic整慎，以yum安裝為例
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch# 添加elasticsearch源echo '[elasticsearch-5.x]name=Elasticsearch repository for 5.x packagesbaseurl=https://artifacts.elastic.co/packages/5.x/yumgpgcheck=1gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearchenabled=1autorefresh=1type=rpm-md' | sudo tee /etc/yum.repos.d/elasticsearch.repo# yum makecache && yum install elasticsearch -y
二脏款、elastic配置詳解
1、Elasticsearch cluster 三種角色
master node：master節(jié)點(diǎn)主要用于元數(shù)據(jù)（metadata）處理裤园，如撤师、索引的新增、刪除拧揽、分片
data node: data節(jié)點(diǎn)上保存了數(shù)據(jù)片
client node: client節(jié)點(diǎn)起到路由請(qǐng)求的作用剃盾，可看做負(fù)載均衡器
2、節(jié)點(diǎn)選擇
# 配置文件中給出了三種配置高性能集群拓?fù)浣Y(jié)構(gòu)的模式,如下：
- 如果你想讓節(jié)點(diǎn)從不選舉為主節(jié)點(diǎn),只用來存儲(chǔ)數(shù)據(jù),可作為負(fù)載器
node.master: false
node.data: true
- 如果想讓節(jié)點(diǎn)成為主節(jié)點(diǎn),且不存儲(chǔ)任何數(shù)據(jù),并保有空閑資源,可作為協(xié)調(diào)器
node.master: true
node.data: false
- 如果想讓節(jié)點(diǎn)既不稱為主節(jié)點(diǎn),又不成為數(shù)據(jù)節(jié)點(diǎn),那么可將他作為搜索器,從節(jié)點(diǎn)中獲取數(shù)據(jù),生成搜索結(jié)果等
node.master: false
node.data: false
3淤袜、elasticsearch.yaml配置詳解
# elastic-a1（192.168.1.121）痒谴，master節(jié)點(diǎn)
[root@elastic-a1 ~]# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml
cluster.name: es-cluster
node.name: es-node-a1
node.master: true
node.data: true
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: false
network.host: 192.168.1.121
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.121", "192.168.1.122", "192.168.1.123"]
discovery.zen.minimum_master_nodes: 1
gateway.recover_after_nodes: 2
gateway.recover_after_time: 5m
gateway.expected_nodes: 1
bootstrap.system_call_filter: false
script.engine.groovy.inline.search: on
script.engine.groovy.inline.aggs: on
indices.recovery.max_bytes_per_sec: 20mb
http.cors.enabled: true
http.cors.allow-origin: "*"
# elastic-a2 ，data節(jié)點(diǎn)（192.168.1.122）
[root@elastic-a2 ~]# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml
cluster.name: es-cluster
node.name: es-node-a2
node.master: false
node.data: true
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: false
network.host: 192.168.1.122
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.121", "192.168.1.122", "192.168.1.123"]
discovery.zen.minimum_master_nodes: 1
gateway.recover_after_nodes: 2
gateway.recover_after_time: 5m
gateway.expected_nodes: 1
bootstrap.system_call_filter: false
script.engine.groovy.inline.search: on
script.engine.groovy.inline.aggs: on
indices.recovery.max_bytes_per_sec: 20mb
# elastic-a3饮怯，client節(jié)點(diǎn)（192.168.1.123）
[root@elastic-a3 ~]# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml
cluster.name: es-cluster
node.name: es-node-a3
node.master: false
node.data: false
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: false
network.host: 192.168.1.123
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.121", "192.168.1.122", "192.168.1.123"]
discovery.zen.minimum_master_nodes: 1
gateway.recover_after_nodes: 2
gateway.recover_after_time: 5m
gateway.expected_nodes: 1
bootstrap.system_call_filter: false
script.engine.groovy.inline.search: on
script.engine.groovy.inline.aggs: on
indices.recovery.max_bytes_per_sec: 20mb
# 注闰歪，建議配置參考本文配置，集體到?jīng)]想?yún)?shù)的含義蓖墅，這里不具體給你库倘，自行g(shù)oogle临扮，baidu,如果讀者做實(shí)驗(yàn)時(shí)沒有足夠多的主機(jī)來實(shí)現(xiàn)es-cluster，可以在同一主機(jī)上進(jìn)行配置教翩，這里只需要修改下面一行：
discovery.zen.ping.unicast.hosts: ["192.168.1.121", "192.168.1.122", "192.168.1.123"] 改成 discovery.zen.ping.unicast.hosts: ["0.0.0.0:9300", "0.0.0.0:9301", "0.0.0.0:9302"]
三杆勇、elastic集群?jiǎn)?dòng)
1、分別在三個(gè)主機(jī)上啟動(dòng)elasticsearch服務(wù)饱亿，并查看啟動(dòng)日志是否報(bào)錯(cuò)
/etc/init.d/elasticsearch start tailf
/var/log/elasticsearch/es-cluster.log
2蚜退、啟動(dòng)成功，瀏覽器訪問 http://192.168.1.121:9200/_cluster/health?pretty=true
# 到這里elasticsearch集群已經(jīng)部署完成彪笼，具體使用詳情請(qǐng)自行移步到elsatic官網(wǎng)钻注，查看官方文檔
四、head插件安裝
#（安裝在elastic-a1節(jié)點(diǎn)）
1配猫、參考：https://github.com/mobz/elasticsearch-head
yum install npm git -ycd /usr/share/elasticsearch/gitclonegit://github.com/mobz/elasticsearch-head.gitcdelasticsearch-head && npm installnpm install -g grunt
2幅恋、插件配置
cd /usr/share/elasticsearch/elasticsearch-head/ # 移動(dòng)到插件項(xiàng)目目錄下
vim +4329 _site/app.js #修改app.js中l(wèi)ocalhost為節(jié)點(diǎn)IP，便于連接es-cluster
3泵肄、后臺(tái)運(yùn)行插件
cd /usr/share/elasticsearch/elasticsearch-head/
nohup grunt server &
4捆交、通過插件es-cluster狀態(tài) # http://192.168.1.121:9100/ （）
五、總結(jié)
1腐巢、?unable to install syscall filter
echo "bootstrap.system_call_filter: false" >> /etc/elasticsearch/elasticsearch.yml # 解決報(bào)錯(cuò) 品追！
使用本地 IP（127.0.0.1）時(shí)，Elasticsearch 進(jìn)入 dev mode冯丙，只能從本機(jī)訪問肉瓦，只顯示警告。
使用局域網(wǎng)IP后银还，可以從其他機(jī)器訪問风宁，但啟動(dòng)時(shí)進(jìn)入 production mode，并進(jìn)行 bootstrap check蛹疯，有可能對(duì)不合適的系統(tǒng)參數(shù)報(bào)錯(cuò)戒财。
2、安裝注意事項(xiàng)主要是elasticsearch配置文件捺弦，確保配置文件正確饮寞，然后再去啟動(dòng)elastic節(jié)點(diǎn) 參考鏈接： http://blog.csdn.net/gamer_gyt/article/details/59077189#reply
https://www.gitbook.com/book/chenryn/elk-stack-guide-cn/details
https://www.elastic.co/guide/index.html
