1.ELKB定義
ELKB分別指Elasticsearch蒜危、Logstash 呼伸、 Kibana和beats 四個(gè)開(kāi)源項(xiàng)目身冀，
Elasticsearch 即ES,是一個(gè)搜索和分析引擎。Logstash 是服務(wù)器端數(shù)據(jù)處理管道括享，能夠同時(shí)從多個(gè)來(lái)源采集數(shù)據(jù)搂根，轉(zhuǎn)換數(shù)據(jù)，然后將數(shù)據(jù)發(fā)送到諸如 Elasticsearch 等“存儲(chǔ)庫(kù)”中铃辖。Kibana 則可以讓用戶將數(shù)據(jù)使用圖形和圖表對(duì)數(shù)據(jù)進(jìn)行可視化展示剩愧。
2.安裝elkb
a、Elasticsearch安裝
1） 下載安裝包：wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.0-linux-x86_64.tar.gz
2）配置elasticsearch
#network.host: 192.168.0.1
network.host: 0.0.0.0
http.port: 9200
#cluster.initial_master_nodes: ["node-1", "node-2"]
cluster.initial_master_nodes: ["elkb-node-1"]
#啟用xpack安全驗(yàn)證
xpack.security.enabled: true
#xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true
#設(shè)置kibana需要登錄密碼
./elasticsearch-setup-passwords interactive
3）啟動(dòng)elasticsearch
nohup /app/elkb/elasticsearch-7.10.0/bin/elasticsearch > /dev/null 2>&1 &
vi /etc/sysctl.conf 添加一行：
vm.max_map_count=655360
然后執(zhí)行：sysctl -p 命令讓內(nèi)核生效
curl 127.0.0.1:9200能看到如下圖表示成功：
b.Logstash安裝
下載安裝包：wget https://artifacts.elastic.co/downloads/logstash/logstash-7.10.0-linux-x86_64.tar.gz
進(jìn)入配置目錄：
cd /app/elkb/logstash-7.10.0/config
cp logstash-sample.conf logstash-log.conf
修改配置：
此處用戶配置elastic
#啟動(dòng)：
nohup ../bin/logstash -f ../config/logstash-log.conf --config.reload.automatic > /dev/null 2>&1 &
--config.reload.automatic可以在Logstash不重啟的情況下自動(dòng)加載配置文件
c.Kibana安裝
#下載安裝包
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.10.0-linux-x86_64.tar.gz
修改配置文件kibana.yml
# 端口
server.port: 5601
#訪問(wèn)ip
server.host: "0.0.0.0"
# 訪問(wèn)上下文
server.basePath: "/kibana"
# 重寫(xiě)path
server.rewriteBasePath: true
# The Kibana server's name. This is used for display purposes.
server.name: "kibanaName"
# es地址
elasticsearch.hosts: ["http://localhost:9200"]
# dashboards. Kibana creates a new index if the index doesn't already exist.
kibana.index: ".kibana"
# is proxied through the Kibana server.
elasticsearch.username: "kibana_system"
elasticsearch.password: "wcyq@2022"
logging.dest: /app/elkb/kibana-7.10.0-linux-x86_64/logs/kibana.log
#設(shè)置中文
i18n.locale: "zh-CN"
nignx配置：
啟動(dòng)kibana
#啟動(dòng)
nohup ../bin/kibana > /dev/null 2>&1 &
訪問(wèn)kibana
http://127.0.0.1:5601/
d.filebeat安裝
下載安裝包：wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.10.0-linux-x86_64.tar.gz
修改配置：
filebeat.inputs:
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
#- /var/log/*.log
- /app/logs/*/*.log
output.logstash:
# The Logstash hosts
hosts: ["10.0.0.110:5044"]
啟動(dòng)：
#查看啟用或禁用模塊：
./filebeat modules list
#啟用logstash模塊
./filebeat modules enable logstash
#檢查配置文件是否正確
./filebeat test config
#啟動(dòng)filebeat
nohup ./filebeat -e -c filebeat-log.yml >/dev/null 2>&1 &
注意：該方式啟動(dòng)娇斩，一段時(shí)間后filebeat服務(wù)會(huì)莫名的停掉仁卷，需要自己定義個(gè)servie啟動(dòng)
#創(chuàng)建filebeat.service
touch /usr/lib/systemd/system/filebeat.service
#編輯filebeat.service
vi /usr/lib/systemd/system/filebeat.service
filebeat.service內(nèi)容
[Unit]
Description=filebeat is a lightweight shipper for metrics.
Documentation=https://www.elastic.co/products/beats/filebeat
Wants=network-online.target
After=network-online.target
[Service]
Environment="BEAT_LOG_OPTS=-e"
Environment="BEAT_CONFIG_OPTS=-c /app/filebeat-7.10.0-linux-x86_64/filebeat-log.yml"
Environment="BEAT_PATH_OPTS=-path.home /app/filebeat-7.10.0-linux-x86_64 -path.config /app/filebeat-7.10.0-linux-x86_64 -path.data /app/filebeat-7.10.0-linux-x86_64/data -path.logs /app/filebeat-7.10.0-linux-x86_64/logs"
ExecStart=/app/filebeat-7.10.0-linux-x86_64/filebeat $BEAT_LOG_OPTS $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS
Restart=always
[Install]
WantedBy=multi-user.target
執(zhí)行以下命令：
#重新加載服務(wù)配置文件
systemctl daemon-reload
#開(kāi)機(jī)自啟動(dòng)
systemctl enable filebeat
#啟動(dòng)filebeat
systemctl start filebeat
查看是否啟動(dòng)成功
systemctl status filebeat
啟動(dòng)失敗：
image.png
查看linux系統(tǒng)日志：
tail -333f /var/log/messages
發(fā)現(xiàn)是filebeat配置文件需要為當(dāng)前用戶
image.png
授權(quán)成洗，然后重新啟動(dòng)
chown -R root.root /app/filebeat-7.10.0-linux-x86_64/*
啟動(dòng)成功：
image.png
定時(shí)刪除elk日志五督，防止磁盤(pán)空間爆滿
1.編寫(xiě)腳本，創(chuàng)建定時(shí)任務(wù)刪除
vi elk_log_clear.sh
#!/bin/bash
#刪除ELK15天前的日志
DATE=`date -d "15 days ago" +%Y.%m.%d`
echo 'date:' ${DATE}
curl -s --user elastic:123456 -XGET http://127.0.0.1:9200/_cat/indices?v| grep $DATE | awk -F '[ ]+' '{print $3}' >/tmp/elk.log
for elk in `cat /tmp/elk.log`
do
curl --user elastic:wcyq@2022 -X DELETE "http://127.0.0.1:9200/$elk"
done
-----------------------------
創(chuàng)建定時(shí)任務(wù)
crontab -e
#每天凌晨1點(diǎn)定時(shí)清理elk索引
00 01 * * * bash /app/elkb/scripts/elk_log_clear.sh &>/dev/null
