es-日志存儲(chǔ)-Logstash 介紹

本文是elasticsearch官方文檔logstash的翻譯寡壮,你也可<a >查看原文</a>

注:Logstash意思是日志存儲(chǔ),下文中對(duì)本詞使用英文包雀。

Paste_Image.png

Logstash 介紹

Logstash is an open source data collection engine with real-time pipelining capabilities(功能). Logstash can dynamically unify(統(tǒng)一) data from disparate(不同的) sources and normalize the data into destinations(目的地) of your choice. Cleanse(凈化) and democratize(大眾化) all your data for diverse(不同的) advanced downstream(下游) analytics and visualization(形象化) use cases.

Logstash 是開(kāi)源的具有實(shí)時(shí)輸入數(shù)據(jù)能力的數(shù)據(jù)收集引擎鹤盒。Logstash可以把來(lái)自不同的源的數(shù)據(jù)動(dòng)態(tài)的寫(xiě)入你選擇的目的地并對(duì)輸入的數(shù)據(jù)進(jìn)行規(guī)范化。根據(jù)需求進(jìn)行分析和展示具钥。

While Logstash originally(最初) drove innovation(創(chuàng)新) in log collection, its capabilities extend well beyond(超越) that use case. Any type of event can be enriched and transformed(轉(zhuǎn)變) with a broad(寬的) array of input, filter, and output plugins, with many native codecs further(更好地) simplifying(簡(jiǎn)化) the ingestion(吸收) process. Logstash accelerates(加速) your insights(洞察力) by harnessing(利用) a greater volume and variety(多樣) of data.

Logstash升級(jí)版本的時(shí)候,老版本對(duì)新版本的兼容新很好液兽。事件的類(lèi)型是一組input骂删、filter、和output插件四啰,所以可以很方便擴(kuò)展宁玫,可以使用本地編碼更好簡(jiǎn)化吸收日志的過(guò)程。

The Power of Logstash

The ingestion workhorse for Elasticsearch and more
Horizontally(水平) scalable(可擴(kuò)展的) data processing pipeline with strong Elasticsearch and Kibana synergy(協(xié)同)

通過(guò)Elasticearch和kibana進(jìn)行協(xié)同柑晒,實(shí)現(xiàn)數(shù)據(jù)處理的水平擴(kuò)展

Pluggable pipeline architecture
Mix, match, and orchestrate different inputs, filters, and outputs to play in pipeline harmony

插件化管飽結(jié)構(gòu)
混合,匹配和協(xié)調(diào)不同的 inputs, filters, and outputs讓管道和諧而不發(fā)生沖突欧瘪。

Community-extensible and developer-friendly plugin ecosystem
Over 200 plugins available, plus the flexibility of creating and contributing your own

社區(qū)可擴(kuò)展和開(kāi)發(fā)者友好的插件生態(tài)系統(tǒng)
超過(guò)200個(gè)插件可以使用,你也可以很快速創(chuàng)建和貢獻(xiàn)你的插件

Paste_Image.png

Logstash Loves Data

Collect more, so you can know more. Logstash welcomes data of all shapes and sizes.

搜集更多,這樣你就可以知道更多匙赞。Logstash可以存儲(chǔ)各種各樣的數(shù)據(jù)佛掖。

Logs and Metrics
Where it all started.

  • Handle all types of logging data
    • Easily ingest(攝取) a multitude(大批的) of web logs like Apache, and application logs like log4j for Java
    • Capture many other log formats like syslog, Windows event logs, networking and firewall logs, and more
  • Enjoy complementary(補(bǔ)充的) secure log forwarding capabilities with Filebeat
  • Collect metrics from Ganglia, collectd, NetFlow, JMX, and many other infrastructure(基礎(chǔ)設(shè)施) and application platforms over TCP and UDP

日志和度量
這是一切開(kāi)始的地方涌庭。

  • 處理很多類(lèi)型的日志
  • 那些可以容易獲取的<code>Apache</code>之類(lèi)的web日志芥被,應(yīng)用日志如java的log4J
  • 其他的格式化的日志如<i>syslog,Windows event logs,網(wǎng)絡(luò)和防火墻日志等等</i>
  • Filebeat轉(zhuǎn)發(fā)來(lái)的安全日志。
  • Ganglia, collectd, NetFlow, JMX, 和一些其他的(基礎(chǔ)設(shè)施) and 基于 TCP and UDP的應(yīng)用平臺(tái)

The Web
Unlock the World Wide Web.

  • Transform HTTP requests into events (blog)
  • Consume(消耗) from web service firehoses(消防帶) like Twitter for social sentiment(情感) analysis
  • Webhook(網(wǎng)絡(luò)連接) support for GitHub, HipChat, JIRA, and countless(無(wú)數(shù)的) other applications
  • Enables many Watcher alerting use cases
  • Create events by polling(投票) HTTP endpoints on demand (blog)
  • Universally(普遍) capture health, performance, metrics, and other types of data from web application interfaces
  • Perfect for scenarios where the control of polling is preferred over receiving

解鎖萬(wàn)維網(wǎng)坐榆。

  • Http事件

Data Stores and Streams
Discover more value from the data you already own.

  • Better understand your data from any relational database or NoSQL store with a JDBC interface (blog)
  • Unify(統(tǒng)一) diverse(不同的) data streams from messaging queues like Apache Kafka (blog), RabbitMQ, Amazon SQS, and ZeroMQ

數(shù)據(jù)存儲(chǔ)和數(shù)據(jù)流

Sensors and IoT
Explore an expansive breadth of other data.

  • In this age of technological advancement, the massive IoT world unleashes endless use cases through capturing and harnessing data from connected sensors.
  • Logstash is the common event collection backbone for ingestion of data shipped from mobile devices to intelligent homes, connected vehicles, healthcare sensors, and many other industry specific applications.
  • Watch as Logstash, in conjunction with the broader ELK stack, centralizes and enriches sensor data to gain deeper knowledge regarding a residential home.

Easily Enrich Everything

The better the data, the better the knowledge. Clean and transform your data during ingestion to gain near real-time insights immediately at index or output time. Logstash comes out-of-box with many aggregations and mutations along with pattern matching, geo mapping, and dynamic lookup capabilities.

  • Grok is the bread and butter of Logstash filters and is used ubiquitously to derive structure out of unstructured data. Enjoy a wealth of integrated patterns aimed to help quickly resolve web, systems, networking, and other types of event formats.
  • Expand your horizons by deciphering geo coordinates from IP addresses, normalizing datecomplexity, simplifying key-value pairs and CSV data, anonymizing sensitive information, and further enriching your data with local lookups or Elasticsearch queries.
  • Codecs are often used to ease the processing of common event structures like JSON and multilineevents.

Choose Your Stash

Route your data where it matters most. Unlock various downstream analytical and operational use cases by storing, analyzing, and taking action on your data.

Analysis
Elasticsearch
Data stores such as MongoDB and Riak

Archiving
HDFS
S3
Google Cloud Storage

Monitoring
Nagios
Ganglia
Zabbix
Graphite
Datadog
CloudWatch

Alerting
Watcher with Elasticsearch
Email
Pagerduty
HipChat
IRC
SNS

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
  • 序言:七十年代末拴魄,一起剝皮案震驚了整個(gè)濱河市,隨后出現(xiàn)的幾起案子席镀,更是在濱河造成了極大的恐慌羹铅,老刑警劉巖,帶你破解...
    沈念sama閱讀 218,451評(píng)論 6 506
  • 序言:濱河連續(xù)發(fā)生了三起死亡事件愉昆,死亡現(xiàn)場(chǎng)離奇詭異职员,居然都是意外死亡,警方通過(guò)查閱死者的電腦和手機(jī)跛溉,發(fā)現(xiàn)死者居然都...
    沈念sama閱讀 93,172評(píng)論 3 394
  • 文/潘曉璐 我一進(jìn)店門(mén)焊切,熙熙樓的掌柜王于貴愁眉苦臉地迎上來(lái)扮授,“玉大人,你說(shuō)我怎么就攤上這事专肪∩膊” “怎么了?”我有些...
    開(kāi)封第一講書(shū)人閱讀 164,782評(píng)論 0 354
  • 文/不壞的土叔 我叫張陵嚎尤,是天一觀的道長(zhǎng)荔仁。 經(jīng)常有香客問(wèn)我,道長(zhǎng)芽死,這世上最難降的妖魔是什么乏梁? 我笑而不...
    開(kāi)封第一講書(shū)人閱讀 58,709評(píng)論 1 294
  • 正文 為了忘掉前任,我火速辦了婚禮关贵,結(jié)果婚禮上遇骑,老公的妹妹穿的比我還像新娘。我一直安慰自己揖曾,他們只是感情好落萎,可當(dāng)我...
    茶點(diǎn)故事閱讀 67,733評(píng)論 6 392
  • 文/花漫 我一把揭開(kāi)白布。 她就那樣靜靜地躺著炭剪,像睡著了一般练链。 火紅的嫁衣襯著肌膚如雪。 梳的紋絲不亂的頭發(fā)上奴拦,一...
    開(kāi)封第一講書(shū)人閱讀 51,578評(píng)論 1 305
  • 那天媒鼓,我揣著相機(jī)與錄音,去河邊找鬼粱坤。 笑死隶糕,一個(gè)胖子當(dāng)著我的面吹牛瓷产,可吹牛的內(nèi)容都是我干的站玄。 我是一名探鬼主播,決...
    沈念sama閱讀 40,320評(píng)論 3 418
  • 文/蒼蘭香墨 我猛地睜開(kāi)眼濒旦,長(zhǎng)吁一口氣:“原來(lái)是場(chǎng)噩夢(mèng)啊……” “哼株旷!你這毒婦竟也來(lái)了?” 一聲冷哼從身側(cè)響起尔邓,我...
    開(kāi)封第一講書(shū)人閱讀 39,241評(píng)論 0 276
  • 序言:老撾萬(wàn)榮一對(duì)情侶失蹤晾剖,失蹤者是張志新(化名)和其女友劉穎,沒(méi)想到半個(gè)月后梯嗽,有當(dāng)?shù)厝嗽跇?shù)林里發(fā)現(xiàn)了一具尸體齿尽,經(jīng)...
    沈念sama閱讀 45,686評(píng)論 1 314
  • 正文 獨(dú)居荒郊野嶺守林人離奇死亡,尸身上長(zhǎng)有42處帶血的膿包…… 初始之章·張勛 以下內(nèi)容為張勛視角 年9月15日...
    茶點(diǎn)故事閱讀 37,878評(píng)論 3 336
  • 正文 我和宋清朗相戀三年灯节,在試婚紗的時(shí)候發(fā)現(xiàn)自己被綠了循头。 大學(xué)時(shí)的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片绵估。...
    茶點(diǎn)故事閱讀 39,992評(píng)論 1 348
  • 序言:一個(gè)原本活蹦亂跳的男人離奇死亡,死狀恐怖卡骂,靈堂內(nèi)的尸體忽然破棺而出国裳,到底是詐尸還是另有隱情,我是刑警寧澤全跨,帶...
    沈念sama閱讀 35,715評(píng)論 5 346
  • 正文 年R本政府宣布缝左,位于F島的核電站,受9級(jí)特大地震影響浓若,放射性物質(zhì)發(fā)生泄漏渺杉。R本人自食惡果不足惜,卻給世界環(huán)境...
    茶點(diǎn)故事閱讀 41,336評(píng)論 3 330
  • 文/蒙蒙 一七嫌、第九天 我趴在偏房一處隱蔽的房頂上張望少办。 院中可真熱鬧,春花似錦诵原、人聲如沸英妓。這莊子的主人今日做“春日...
    開(kāi)封第一講書(shū)人閱讀 31,912評(píng)論 0 22
  • 文/蒼蘭香墨 我抬頭看了看天上的太陽(yáng)蔓纠。三九已至,卻和暖如春吗蚌,著一層夾襖步出監(jiān)牢的瞬間腿倚,已是汗流浹背。 一陣腳步聲響...
    開(kāi)封第一講書(shū)人閱讀 33,040評(píng)論 1 270
  • 我被黑心中介騙來(lái)泰國(guó)打工蚯妇, 沒(méi)想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留敷燎,地道東北人。 一個(gè)月前我還...
    沈念sama閱讀 48,173評(píng)論 3 370
  • 正文 我出身青樓箩言,卻偏偏與公主長(zhǎng)得像硬贯,于是被迫代替她去往敵國(guó)和親。 傳聞我的和親對(duì)象是個(gè)殘疾皇子陨收,可洞房花燭夜當(dāng)晚...
    茶點(diǎn)故事閱讀 44,947評(píng)論 2 355

推薦閱讀更多精彩內(nèi)容