1寺鸥、logstash配置
下載
#wget https://artifacts.elastic.co/downloads/logstash/logstash-7.11.1-linux-x86_64.tar.gz
1.1倘核、編輯配置文件
#vim logstash.yml
# ------------ Pipeline Configuration Settings --------------
#
# Where to fetch the pipeline configuration for the main pipeline
#
path.config: "/home/elk/logstash-7.6.0/log_analysis/*.conf"
? //添加解析文件路徑
#
# Pipeline configuration string for the main pipeline
# ------------ Metrics Settings --------------
#
# Bind address for the metrics REST endpoint
#
http.host: 172.16.0.4? ? ? //設(shè)置監(jiān)聽IP
#
# Bind port for the metrics REST endpoint, this option also accept a range
# (9600-9700) and logstash will pick up the first available ports.
#
# http.port: 9600-9700
在/home/elk/logstash-7.6.0/log_analysis/下添加日志解析文件
nginx日志解析
input {
? ? ? ? beats {
? ? ? ? ? ? port => 6114
? ? ? ? }
}
filter {
? ? grok {
? ? ? match => { "message" => [ "%{IPORHOST:Client_IP} (%{WORD:ident}|-) (%{USERNAME:auth}|-) \[%{HTTPDATE:timestamp}\] \"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) \"(?:%{NOTSPACE:referrer}|-)\" %{QS:agentname}",
? ? ? ? ? ? ? ? ? "%{IPORHOST:Client_IP} %{DATA:ident} %{DATA:auth} \[%{HTTPDATE:timestamp}\] \"%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response} %{NUMBER:bytes} \"%{DATA:referrer}\" %{QS:agentname}",
? ? ? ? ? ? ? ? ? "%{IPORHOST:Client_IP} %{DATA:ident} %{DATA:auth} \[%{HTTPDATE:timestamp}\] \"%{DATA:request}\" %{NUMBER:response} %{NUMBER:bytes}"
? ? ]}
? }
? date {
? ? match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" , "MMM dd HH:mm:ss","MMM? d HH:mm:ss"]
? }
? geoip {
? source => "Client_IP"
? add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
? add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
? }
? mutate {
? ? ? convert => [ "[geoip][coordinates]", "float" ]
? }
}
output {
? elasticsearch {
? ? hosts => ["127.0.0.1:9200"]
? ? ? ? index => "logstash-nginx"
? ? ? ? workers => 1
? ? ? ? template_overwrite => true
? }
stdout { codec => rubydebug }
}
1.2、運(yùn)行
運(yùn)行指定解析文件 # ./bin/logstash -f nginx.conf
后臺運(yùn)行 # nohup ./bin/logstash > run.log &? // 該命令需要在logstash設(shè)置解析文件路徑扒俯，因為執(zhí)行命令沒有包含解析文件
