Elasticsearch
Elasticsearch是一個(gè)基于lucene的全文搜索引擎
1.安裝
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.0.tar.gz
tar -xzvf elasticsearch-5.3.0.tar.gz
mv elasticsearch-5.3.0 /usr/share/elasticsearch-5.3.0
2. 配置
修改配置文件
path.data: /var/lib/elasticsearch //數(shù)據(jù)存放的路徑
path.logs: /var/log/elasticsearch //log的路徑
node.name: wddlc //節(jié)點(diǎn)的名字
network.host: _網(wǎng)卡編號(hào)_
http.port: 9206 //監(jiān)聽的端口號(hào)
配置文件：/usr/share/elasticsearch-5.3.0/config/elasticsearch.yml
如果需要的話修改jvm的內(nèi)存參數(shù)
-Xms1g 初始內(nèi)存大小
-Xmx1g 最大內(nèi)存大小
配置文件：/usr/share/elasticsearch-5.3.0/config/jvm.options
3. 啟動(dòng)
nohup /usr/share/elasticsearch-5.3.0/bin/elasticsearch -Epath.conf=/usr/share/elasticsearch-5.3.0/config/ > /dev/null 2>&1 &
ps -ef | grep elas //關(guān)閉
kill pid
檢測啟動(dòng)是否成功.png
4.一個(gè)小問題
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
這是因?yàn)橄到y(tǒng)進(jìn)程最大虛擬內(nèi)存映射的數(shù)量不足漓踢，用下面的命令設(shè)置一下就可以了
sysctl -w vm.max_map_count=262144
5.監(jiān)控插件
git clone git://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
npm install
npm run start
open
[http://localhost:9100/](http://localhost:9100/)
注意：使用這個(gè)插件的時(shí)候砌烁，要打開es的跨域
http.cors.enabled: true
http.cors.allow-origin: "*"
logstash
logstash是一個(gè)開源的數(shù)據(jù)收集引擎矫限，負(fù)責(zé)從多個(gè)數(shù)據(jù)源匯總數(shù)據(jù)凭戴、過濾、并輸出到指定的輸出中
1.安裝
wget wget https://artifacts.elastic.co/downloads/logstash/logstash-5.3.0.tar.gz
tar -xzvf logstash-5.3.0.tar.gz
mv logstash-5.3.0 /usr/share/logstash-5.3.0
2.配置
input {
beats {
# 監(jiān)聽filebeat的輸入
port => 5043
}
}
filter {
# 刪除無用的key
if ([type] == "log") {
ruby {
code => "
event.to_hash.each { |k,v|
if (!['json', 'type'].include?(k))
event.remove(k)
end
}
"
}
# 根據(jù)ip酪碘，補(bǔ)充地理位置信息
geoip {
source => "[json][ip]"
}
}
}
output {
if ([type] == "log") {
# 輸出到標(biāo)準(zhǔn)輸出
stdout { codec => rubydebug}
# 輸出到剛剛配置好的es里
elasticsearch {
hosts => ["172.16.3.2:9206"]
index => "%{[json][model]}"
}
}
}
2.啟動(dòng)
nohup /usr/share/logstash-5.3.0/bin/logstash -f /usr/share/logstash-5.3.0/config/first-pipeline.conf > /etc/logstash.log 2>&1 &
filebeat
1.安裝
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.3.0-x86_64.rpm
sudo rpm -vi filebeat-5.3.0-x86_64.rpm
2.配置
配置文件在：/etc/filebeat/filebeat.yml
# 設(shè)置輸入
input_type: log
# 表示數(shù)據(jù)的是json
json.key_under_root: true
# 監(jiān)控這個(gè)路徑下的log
paths:
- /var/www/prometheus-library/logs/*.log
# 設(shè)置控制臺(tái)輸出朋譬，方便調(diào)試，正式環(huán)境可以關(guān)掉
output.console:
pretty: true
# 輸出到logstash
output.logstash:
hosts: ["172.16.3.2:5043"]
3.啟動(dòng)
sudo /etc/init.d/filebeat start
kibana
1.安裝
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.3.0-linux-x86_64.tar.gz
tar -xzvf kibana-5.3.0-linux-x86_64.tar.gz
mv kibana-5.3.0-linux-x86_64 /usr/share/kibana-5.3.0-linux-x86_64
2.配置
配置文件：/usr/share/kibana-5.3.0-linux-x86_64/config/kibana.yml
# 設(shè)置es的地址
server.host: "0.0.0.0"
# es請求的路徑
elasticsearch.url: "http://172.16.3.2:9206"
2.啟動(dòng)
nohup /usr/share/kibana-5.3.0-linux-x86_64/bin/kibana > kibana.log 2>&1 &
