ElasticSearch
是一個高可用開源全文檢索和分析組件盒刚。提供存儲服務(wù)掀淘，搜索服務(wù)砚亭，大數(shù)據(jù)準實時分析等。一般用于提供一些提供復(fù)雜搜索的應(yīng)扣猫。
我們?yōu)槭裁匆x擇 ElasticSearch 菜循？
因為它是一個準實時的搜索工具，在一般情況下延時少于一秒申尤，它還支持物理上的水平擴展 癌幕，并擁有一套分布式協(xié)調(diào)的管理功能 操作比較簡單 ，包括一些 restful 風格的API 等等昧穿，接下來我們就來進入今天的正題序芦。
前期準備
1.安裝 jdk1.8
yum install java
2.配置源
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
配置elasticsearch
cat >/etc/yum.repos.d/elasticsearch.repo <<EOF
[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
配置kibana
cat >/etc/yum.repos.d/kibana.repo << EOF
[kibana-5.x]
name=Kibana repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
3.安裝
yum install elasticsearch -y
yum install kibana -y
4.修改 es 配置文件
mkdir /data/es-data/logs -p
chown -R elasticsearch.elasticsearch /data/es-data/logs
[root@linux-node2 ~]# grep -v "^#" /etc/elasticsearch/elasticsearch.yml |grep -v "^$"
cluster.name: lx
node.name: linux-node2.lx.com
path.data: /data/es-data
path.logs: /data/es-data/logs
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.168.56.12", "192.168.56.14"]
discovery.zen.minimum_master_nodes: 1
5.修改內(nèi)核參數(shù)
vim /etc/security/limits.conf
* soft nofile 655350
* hard nofile 655350
vim /etc/sysctl.conf
fs.file-max=655350
調(diào)整內(nèi)存大小
cat /data/elasticsearch/config/jvm.options (可設(shè)置為物理內(nèi)存的一半)
-Xms8g
-Xmx8g
sysctl -w vm.max_map_count=262144
6.修改 kibana 配置文件
[root@linux-node5 elasticsearch]# grep -v "^#" /etc/kibana/kibana.yml |grep -v "^$"
server.port: 5601
server.host: "0.0.0.0"
server.name: "lx"
elasticsearch.url: "http://192.168.56.14:9200"
elasticsearch.username: "elastic"
elasticsearch.password: "changeme"
7.x-pack 破解
編輯配置文件
cat LicenseVerifier.java
package org.elasticsearch.license;
import java.nio.*;
import java.util.*;
import java.security.*;
import org.elasticsearch.common.xcontent.*;
import org.apache.lucene.util.*;
import org.elasticsearch.common.io.*;
import java.io.*;
public class LicenseVerifier
{
public static boolean verifyLicense(final License license, final byte[] encryptedPublicKeyData) {
return true;
}
public static boolean verifyLicense(final License license) {
return true;
}
}
安裝 java-devel
yum install java-devel -y
編譯生成LicenseVerifier.class
javac -cp "/usr/share/elasticsearch/lib/elasticsearch-5.6.4.jar:/usr/share/elasticsearch/lib/lucene-core-6.6.1.jar:/usr/share/elasticsearch/plugins/x-pack/x-pack-5.6.4.jar" LicenseVerifier.java
查看生成的LicenseVerifier.class類文件
ll LicenseVerifier.class
替換class文件
cp /usr/share/elasticsearch/plugins/x-pack/x-pack-5.6.4.jar /tmp/test
cd /tmp/test
jar xvf x-pack-5.6.4.jar
cd /tmp/test/org/elasticsearch/license
rm LicenseVerifier.class -f
cp /root/LicenseVerifier.class /tmp/test/org/elasticsearch/license/ #拷貝上文編譯生成的java類文件
jar cvf x-pack-5.6.4.jar /tmp/test/* #壓縮
\cp /tmp/test/x-pack-5.6.4.jar /usr/share/elasticsearch/plugins/x-pack/ #拷貝到原來的x-pack目錄
注意 ：x-pack-5.6.4.jar 應(yīng)該拷貝到 兩個 elasticsearch 集群插件，即目錄 /usr/share/elasticsearch/plugins/x-pack/
重啟 es 集群
systemct restart elasticsearch
獲取 license 證書
https://license.elastic.co/registration
通過填寫的郵件獲取官方發(fā)來的郵件獲取證書
cat li-xiang-d28260d9-6c96-4dd2-92dc-2f14a9787903-v5.json
{"license":{"uid":"d28260d9-6c96-4dd2-92dc-2f14a9787903","type":"platinum","issue_date_in_millis":1511740800000,"expiry_date_in_millis":1827359999000,"max_nodes":100,"issued_to":"li xiang (ceshi)","issuer":"Web Form","signature":"AAAAAwAAAA2lsE14rcZQLw3V/JuUAAABmC9ZN0hjZDBGYnVyRXpCOW5Bb3FjZDAxOWpSbTVoMVZwUzRxVk1PSmkxaktJRVl5MUYvUWh3bHZVUTllbXNPbzBUemtnbWpBbmlWRmRZb25KNFlBR2x0TXc2K2p1Y1VtMG1UQU9TRGZVSGRwaEJGUjE3bXd3LzRqZ05iLzRteWFNekdxRGpIYlFwYkJiNUs0U1hTVlJKNVlXekMrSlVUdFIvV0FNeWdOYnlESDc3MWhlY3hSQmdKSjJ2ZTcvYlBFOHhPQlV3ZHdDQ0tHcG5uOElCaDJ4K1hob29xSG85N0kvTWV3THhlQk9NL01VMFRjNDZpZEVXeUtUMXIyMlIveFpJUkk2WUdveEZaME9XWitGUi9WNTZVQW1FMG1DenhZU0ZmeXlZakVEMjZFT2NvOWxpZGlqVmlHNC8rWVVUYzMwRGVySHpIdURzKzFiRDl4TmM1TUp2VTBOUlJZUlAyV0ZVL2kvVk10L0NsbXNFYVZwT3NSU082dFNNa2prQ0ZsclZ4NTltbU1CVE5lR09Bck93V2J1Y3c9PQAAAQBE870HCp9jTj22SVdEP2uAFLF6ikLdDJFtSlbHuYOki6rPtWxIcw8y+WWpPUT5e8lYZw0GkB8CYT5hFLXZTrBqTsNbYRZ3ABdHS1BnKBlkfE4PPcvnTTt4HtBCawNHaW0BNWQ2BA2fFj6zX3HyYJ8q5OaQk/il0t5f/TkIuf0yo3Y/F3rzDIXOHieBFnVvfG3EpNB4lo+G+e6vPMeOW86PsF9eKvQ24nucGDK3S4WSpwxbP1gZFuTdmE9zDguJhRHrtJ6k//A/Q0Fbo8gFntWgHNp+1OJEklH+VBBZUWo17UMGnjjxDrGlLTZcsz2BPmk7mC7e8gBQY4z7zJ/SgnsU","start_date_in_millis":1511740800000}}
此證書的時間為1年使用時間粤咪，你可以通過下面網(wǎng)站進行換算
http://tool.chinaz.com/Tools/unixtime.aspx
目前我申請了一個10 年的時間
將 "type":"basic" 替換為 "type":"platinum" # 基礎(chǔ)班變更為鉑金版
將 "expiry_date_in_millis":1543363199999 替換為 "expiry_date_in_millis":1827359999000 # 1年變?yōu)?0年
查看當前的 license
curl -XGET -u elastic:changeme 'http://127.0.0.1:9200/_license'
{
"license" : {
"status" : "active",
"uid" : "21389992-4010-4d2c-917b-94b4e3d5a1dc",
"type" : "trial",
"issue_date" : "2017-11-27T05:12:27.999Z",
"issue_date_in_millis" : 1511759547999,
"expiry_date" : "2017-12-27T05:12:27.999Z",
"expiry_date_in_millis" : 1514351547999,
"max_nodes" : 1000,
"issued_to" : "lx",
"issuer" : "elasticsearch",
"start_date_in_millis" : -1
}
}
替換 license
curl -XPUT -u elastic:changeme 'http://127.0.0.1:9200/_xpack/license?acknowledge=true' -d @li-xiang-d28260d9-6c96-4dd2-92dc-2f14a9787903-v5.json
重啟 es 集群
systemctl restart elasticsearch
查看license
[root@linux-node5 license]# curl -XGET -u elastic:changeme 'http://127.0.0.1:9200/_license'
{
"license" : {
"status" : "active",
"uid" : "d28260d9-6c96-4dd2-92dc-2f14a9787903",
"type" : "platinum",
"issue_date" : "2017-11-27T00:00:00.000Z",
"issue_date_in_millis" : 1511740800000,
"expiry_date" : "2027-11-27T23:59:59.000Z",
"expiry_date_in_millis" : 1827359999000,
"max_nodes" : 100,
"issued_to" : "li xiang (ceshi)",
"issuer" : "Web Form",
"start_date_in_millis" : 1511740800000
}
}
配置 X- Pack 告警
本文配置 X- Pack告警是通過 filebeat 收集 Nginx 的日志來做的模擬
配置郵件報警
1.安裝 Nginx
2.給 Nginx 配置 json 格式的數(shù)據(jù)
log_format json '{"@timestamp":"$time_iso8601",'
'"@version":"1",'
'"client":"$remote_addr",'
'"url":"$uri",'
'"status":"$status",'
'"domain":"$host",'
'"host":"$server_addr",'
'"size":$body_bytes_sent,'
'"responsetime":$request_time,'
'"referer": "$http_referer",'
'"ua": "$http_user_agent"'
'}';
access_log logs/access.log json;
3.安裝 filebeta
安裝
yum install -y filebeta
配置filebeta
[root@linux-node4 filebeat]# grep -v "^ #" filebeat.yml|grep -v "^$"|grep -v "^#"
filebeat.prospectors:
- input_type: log
paths:
- /usr/local/nginx/logs/access.log
json.keys_under_root: true
json.overwrite_keys: true
output.elasticsearch:
hosts: ["localhost:9200"]
username: "elastic"
password: "changeme"
參考網(wǎng)址:https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-json
http://www.iyunw.cn/archives/filebeat-shou-ji-json-ge-shi-de-nginx-ri-zhi-fa-song-gei-elasticsearch/
啟動filebeta服務(wù)
systemctl status filebeat.service
這里默認進入 es 的索引是 filebeat-{%}
4.登陸 kibana 添加索引
5.模擬一些 404的頁面，讓其展示
for i in {1..1000};do curl http://192.168.56.14:/lx-0$i;sleep 1;done
6.添加報警觸發(fā)器(收集每分鐘內(nèi)超過訪問狀態(tài)大于 400 且個數(shù)超過 20 報警)
Watch JSON
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"filebeat-2017*"
],
"types": [],
"body": {
"size": 0,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "status:>=400"
}
},
{
"range": {
"@timestamp": {
"gte": "now-1m"
}
}
}
]
}
},
"sort": [
{
"@timestamp": {
"order": "desc"
}
}
]
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 20
}
}
},
"actions": {
"elasticsearch": {
"throttle_period_in_millis": 60000,
"email": {
"profile": "outlook",
"attachments": {
"attached_data": {
"data": {
"format": "json"
}
}
},
"priority": "high",
"to": [
"lixiang@xxxxx.com"
],
"subject": "Nginx {{ctx.payload.hits.total}} errors",
"body": {
"text": "nginx 404 錯誤過多,請注意查看"
}
}
}
}
}
7.這一步特別重要需要在 ES 配置文件設(shè)置否則無法報警(因為我公司使用的是 Exchange 客戶端)
xpack.notification.email.account:
exchange_account:
profile: outlook
email_defaults:
from: warning@xxxxx.com
smtp:
auth: true
starttls.enable: true
host: smtp.partner.outlook.cn
port: 587
user: 123@xxxxxx.com
password: 233g@123
參考網(wǎng)址：https://www.elastic.co/guide/en/x-pack/5.6/actions-email.html #郵件設(shè)置
- 查看郵件報警狀態(tài)以及 kibana 中的狀態(tài)
對于上面的內(nèi)容大家可以參看下面的 url 來了解其中的含義
參考網(wǎng)址:
https://www.elastic.co/guide/en/x-pack/5.6/xpack-alerting.html 對集群和索引事件的警告 Xpack document
配置Webhook報警
對于 webhook 的報警,我們需要前期做一下工作需要了解一個東西web.py,具體 URL 鏈接大家可以訪問這個地方
http://webpy.org/tutorial3.zh-cn#starting
1.用 web.py 啟動一個 webhook 的監(jiān)聽
[root@linux-node5 ~]# cat webhooks.py
#!/usr/bin/evn python
# -*- coding:utf-8 -*-
import web
import os
import demjson
import sys
reload(sys)
sys.setdefaultencoding( "utf-8" )
urls = (
'/log_event_watch','abc',
)
class abc:
def POST(self):
data = web.data()
print data
cmd = '''curl -G -v "http://abc.com/abc/" --data "user=lixiang" --data "media=all" --data-urlencode "subject=test" --data-urlencode "message=%s"''' %(data)
os.system(cmd)
if __name__ == "__main__":
app = web.application(urls, globals())
app.run()
運行 python webhooks.py 9000
注意 ： urls渴杆，因為我的 Watcher ID 為 "log_event_watch"
2.再次回到我們的 kibana 界面,
"Management"->"Edit"
查看 Watches
08.png
查看 Wathch Json 內(nèi)容
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"nginx_access*"
],
"types": [],
"body": {
"size": 0,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "status:>=400"
}
},
{
"range": {
"@timestamp": {
"gte": "now-1m"
}
}
}
]
}
},
"sort": [
{
"@timestamp": {
"order": "desc"
}
}
]
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 5
}
}
},
"actions": {
"email": {
"throttle_period_in_millis": 60000,
"email": {
"profile": "outlook",
"attachments": {
"attached_data": {
"data": {
"format": "json"
}
}
},
"priority": "high",
"to": [
"lixiang@xxxxx.com"
],
"subject": "Nginx {{ctx.payload.hits.total}} errors",
"body": {
"text": "nginx 404 錯誤過多,請注意查看"
}
}
},
"webhook": {
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 5
}
}
},
"webhook": {
"scheme": "http",
"host": "192.168.56.15",
"port": 9000,
"method": "post",
"path": "/{{watch_id}}",
"params": {},
"headers": {},
"body": "Encountered {{ctx.payload.hits.total}} errors"
}
}
}
}
3.模擬觸發(fā)報警
for i in {1..200};do curl http://192.168.56.14:/lx-0$i;sleep 2;done
4.查看結(jié)果寥枝，這里就不貼出來了(微信，短信磁奖，郵件都能收到報警)
5.這就是實現(xiàn)報警消息附帶 URL 地址
官方參考網(wǎng)址:
https://www.elastic.co/guide/en/x-pack/5.6/watcher-getting-started.html
技術(shù)交流QQ群：365534424
今晚九點|基于 Ansible API 的任務(wù)管理平臺
主講師：panda
前 douban 運維工程師囊拜，目前就職于創(chuàng)業(yè)公司。引入 douban 的運維平臺思想比搭，完成公司的自動化運維平臺開發(fā)和建設(shè)冠跷。對運維工程師轉(zhuǎn)運維研發(fā)的困惑和痛點深有感觸，樂于分享自己轉(zhuǎn)型中的五味雜陳身诺。
主要內(nèi)容：
1：中小公司對于 puppet/salt/ansible 選擇之我見
2：Ansible 在生產(chǎn)環(huán)境中的常用場景
3：Playbook API實現(xiàn)任務(wù)管理平臺思路蜜托、難點及實現(xiàn)
分享模式：網(wǎng)絡(luò)直播
分享時間：12月14日（周四）
