原始學(xué)習(xí)資料攻旦，請(qǐng)參考官方文檔：https://www.elastic.co/guide/en/logstash/current/output-plugins.html
Logstash的output模塊,相比于input模塊來(lái)說(shuō)是一個(gè)輸出模塊,output模塊集成了大量的輸出插件,可以輸出到指定文件,也可輸出到指定的網(wǎng)絡(luò)端口,當(dāng)然也可以輸出數(shù)據(jù)到ES.
在這里我只介紹如何輸出到ES,至于如何輸出到端口和指定文件,有很多的文檔資料可查找.
? elasticsearch{?
? ? hosts=>["172.132.12.3:9200"]?
? ? action=>"index"?
? ? index=>"indextemplate-logstash"?
? ? #document_type=>"%{@type}"?
? ? document_id=>"ignore"?
? ? template=>"/opt/logstash-conf/es-template.json"?
? ? template_name=>"es-template.json"?
? ? template_overwrite=>true? ? ?
? ? }
action=>”index” #es要執(zhí)行的動(dòng)作 index, delete, create, update
l index:將logstash.時(shí)間索引到一個(gè)文檔
l delete:根據(jù)id刪除一個(gè)document(這個(gè)動(dòng)作需要一個(gè)id)
l create:建立一個(gè)索引document依溯，如果id存在 動(dòng)作失敗.
l update:根據(jù)id更新一個(gè)document，有一種特殊情況可以u(píng)psert--如果document不是已經(jīng)存在的情況更新document 塑径。參見(jiàn)upsert選項(xiàng)咐容。
l A sprintf style string to change the action based on the content of the event. The value %{[foo]} would use the foo field for the action
document_id=>” ” 為索引提供document id 栅螟，對(duì)重寫elasticsearch中相同id詞目很有用
document_type=>” ”事件要被寫入的document type，一般要將相似事件寫入同一type徐钠，可用%{}引用事件type癌刽，默認(rèn)type=log
index=>”logstash-%{+YYYY,MM.dd}” 事件要被寫進(jìn)的索引，可是動(dòng)態(tài)的用%{foo}語(yǔ)句
hosts=>[“127.0.0.0”] ["127.0.0.1:9200","127.0.0.2:9200"] "https://127.0.0.1:9200"?
manage_template=>true 一個(gè)默認(rèn)的es mapping 模板將啟用（除非設(shè)置為false 用自己的template）
template=>”” 有效的filepath 設(shè)置自己的template文件路徑尝丐，不設(shè)置就用已有的
template_name=>”logstash” 在es內(nèi)部模板的名字
這里需要十分注意的一個(gè)問(wèn)題是,document_id盡量保證值得唯一,這樣會(huì)解決你面即將面臨的ES數(shù)據(jù)重復(fù)問(wèn)題,切記切記!
輸出到文件
input{
? ? beats{
? ? ? ? codec => plain{charset => "UTF-8"}
? ? ? ? port => "5044"
? ? ? ? type => "routing-inspection-log-tofile"
? ? }
}
filter{
? ? mutate{
? ? ? ? remove_field => "@version"
? ? ? ? remove_field => "offset"
? ? ? ? remove_field => "input_type"
? ? ? ? remove_field => "beat"
? ? ? ? remove_field => "tags"
? ? }
? ? ruby{
? ? ? ? code => "event.timestamp.time.localtime"
? ? }
}
output{
? ? if? [type] != "routing-inspection-log-tofile" {? //根據(jù)type類型過(guò)濾显拜，排除不需要保存的輸入
? ? ? file{
? ? ? ? ? ? ? path => "/home/app/logbak/%{+YYYY.MM.dd}-file.txt"
? ? ? ? ? ? ? codec => line {format => "%{[collectValue]}"}//設(shè)置根據(jù)原始數(shù)據(jù)格式保存，不會(huì)帶Json格式
? ? ? }
? ? }
}
開(kāi)始設(shè)置的path => "/tmp/%{+YYYY.MM.dd}-%{host}-file.txt"爹袁，由于沒(méi)有%{host} 這個(gè)字段远荠，導(dǎo)致出現(xiàn)無(wú)法創(chuàng)建文件錯(cuò)誤，刪除這個(gè)之后恢復(fù)正常失息。
[2019-04-13T01:17:44,206][FATAL][logstash.runner ? ? ? ? ?] An unexpected error occurred! {:error=>#<Errno::EACCES: Permission denied - /home/app/logbak/2019.04.12-{"hostname":"WIN-NN7AS1GRN2F","os":{"build":"9200.0","kernel":"6.2.9200.16384 (win8_rtm.120725-1247)","name":"Windows Server 2012 Standard","family":"windows","version":"6.2","platform":"windows"},"name":"WIN-NN7AS1GRN2F","id":"22067c43-882c-43e7-8a37-7c929f621593","architecture":"x86_64"}-file.txt>, :backtrace=>["org/jruby/RubyFile.java:370:in `initialize'", "org/jruby/RubyIO.java:871:in `new'", "/home/app/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-file-4.0.2/lib/logstash/outputs/file.rb:280:in `open'", "/home/app/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-file-4.0.2/lib/logstash/outputs/file.rb:132:in `multi_receive_encoded'", "org/jruby/RubyHash.java:1342:in `each'", "/home/app/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-file-4.0.2/lib/logstash/outputs/file.rb:131:in `multi_receive_encoded'", "org/jruby/ext/thread/Mutex.java:149:in `synchronize'", "/home/app/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-file-4.0.2/lib/logstash/outputs/file.rb:130:in `multi_receive_encoded'", "/home/app/logstash/logstash-core/lib/logstash/outputs/base.rb:90:in `multi_receive'", "/home/app/logstash/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:13:in `multi_receive'", "/home/app/logstash/logstash-core/lib/logstash/output_delegator.rb:47:in `multi_receive'", "/home/app/logstash/logstash-core/lib/logstash/pipeline.rb:420:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/home/app/logstash/logstash-core/lib/logstash/pipeline.rb:419:in `output_batch'", "/home/app/logstash/logstash-core/lib/logstash/pipeline.rb:365:in `worker_loop'", "/home/app/logstash/logstash-core/lib/logstash/pipeline.rb:330:in `start_workers'"]}
[elkuser@localhost logstash]$ cd ..
修改之后恢復(fù)正常
---------------------
作者：xcl119xxcl
來(lái)源：CSDN
原文：https://blog.csdn.net/xcl119xcl/article/details/89244563
版權(quán)聲明：本文為博主原創(chuàng)文章譬淳，轉(zhuǎn)載請(qǐng)附上博文鏈接档址！
