目的
處理多種不同的log格式唐片，并打上標(biāo)簽
input log file
pid:9729 2015-03-25 10:47:44.302777 broker.go:97: resuming listening on [::]:8055
pid:9696 2015-03-25 10:47:44.303046 broker.go:119: quiting...
pid:9696 2015-03-25 10:47:44.303112 broker.go:135: detect uncover broker net error
pid:9696 2015-03-25 10:47:44.303126 broker.go:143: broker wait
pid:9696 2015-03-25 10:47:44.303130 broker.go:145: quit broker
pid:9696 2015-03-25 10:47:44.303136 broker.go:124: I am done.
logstash conf
input {
file {
path => ["/Users/duwei/go/hydra/logs/broker.log", "/Users/duwei/go/hydra/logs/err.log"]
codec => "plain"
start_position => "beginning"
}
}
filter {
grok {
match => ["message", "(?<ts>%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{TIME}).*quit"]
remove_field => [ "message" ]
add_tag => ["ok", "quit"]
}
grok {
match => ["message", "(?<ts>%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{TIME}).*done"]
remove_field => [ "message" ]
add_tag => ["ok", "done"]
}
if "ok" not in [tags] {
drop { }
}
}
output {
stdout {
codec => "rubydebug"
}
}
output
{
"@version" => "1",
"@timestamp" => "2015-03-27T03:15:09.636Z",
"host" => "duwei-rmbp.local",
"path" => "/Users/duwei/go/hydra/logs/broker.log",
"tags" => [
[0] "_grokparsefailure",
[1] "ok",
[2] "done"
],
"ts" => "2015/03/19 16:16:30.020656"
}
{
"@version" => "1",
"@timestamp" => "2015-03-27T03:15:09.645Z",
"host" => "duwei-rmbp.local",
"path" => "/Users/duwei/go/hydra/logs/broker.log",
"ts" => "2015/03/19 16:16:44.435645",
"tags" => [
[0] "ok",
[1] "quit",
[2] "_grokparsefailure"
]
}
說(shuō)明
因?yàn)橹灰幸淮尾幻芯蜁?huì)打上_grokparsefailure的tag，所以增加另外一個(gè)tag: ok來(lái)做成功match的判斷
