一、設(shè)置kafka
參考文檔：
配置參考文檔： https://www.orchome.com/171
https://blog.csdn.net/difffate/article/details/53570344
https://discuss.elastic.co/t/tls-for-filebeat-kafka-output/58756
https://discuss.elastic.co/t/tls-for-filebeat-kafka-output/58756/5
文檔配置：
https://blog.csdn.net/qq_41926119/article/details/104510481
kafka官方文檔：
1.? 生成證書：
keytool -keystore kafka.server.keystore.jks -alias xxx -validity 3000 -genkey -keyalg RSA
openssl req -new -x509 -keyout ca-key -out ca-cert -days 3000 -passout pass:xxx
keytool -keystore kafka.server.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore kafka.server.keystore.jks -alias? xxx -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 3000 -CAcreateserial -passin pass:xxx
keytool -keystore kafka.server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore kafka.server.keystore.jks -alias? xx -import -file cert-signed
2.基于jks生成pem證書：
keytool -importkeystore -srckeystore? kafka.server.keystore.jks -destkeystore client.p12 -deststoretype PKCS12
openssl pkcs12 -in client.p12 -nokeys -out client.cer.pem
openssl pkcs12 -in client.p12 -nodes -nocerts -out client.key.pem
keytool -exportcert -alias xxx -keystore kafka.client.truststore.jks -rfc -file serverpub.pem
3. 查看pem證書：
keytool -printcert -file certificate.pem
多個pem證書合并：
cat cert1.pem cert2.pem > bundle.pem
4. 查看jks證書:
keytool -list -v -keystore? server.truststore.jks
5.生成filebeat證書:（需要把ca.pem 加入到 kafka的truststore.jks中抱慌，完成服務(wù)端授信）
通過ES生成自簽的CA證書：./bin/elasticsearch-certutil ca
生成頒發(fā)的證書：./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --name filebeat? --pem --out filebeat2.zip
keytool -keystore kafka.server.truststore.jks -alias CARoot -import -file ca-cert
6.? kafka的最終配置
listeners=PLAINTEXT://172.28.15.231:9092,SSL://172.28.15.231:9093
ssl.endpoint.identification.algorithm=
ssl.keystore.location=/data/xxx/kafka_ca_cert/kafka.server.keystore.jks
ssl.keystore.password=xxxx
ssl.truststore.location=/data/xxx/kafka_ca_cert/kafka.server.truststore.jks
ssl.keystore.type=JKS
ssl.truststore.type=JKS
ssl.secure.random.implementation=SHA1PRNG
7. 客戶端配置：client-ssl.properties
security.protocol=SSL
ssl.truststore.location=/data/xxx/kafka_ca_cert/kafka.client.truststore.jks
ssl.truststore.password=xxxx
ssl.keystore.type=JKS
ssl.truststore.type=JKS
ssl.endpoint.identification.algorithm=
8. kafka 客戶端生產(chǎn)：
kafka-console-producer.sh --topic test-log? --broker-list xxxx:9093 --producer.config client-ssl.properties
kafka-console-producer.sh --topic test-log? --broker-list xxxx:9093 --producer.config client-ssl.properties
9. kafka 客戶端消費：
kafka-console-consumer.sh --topic test-log --bootstrap-server xxxx:9093 --consumer.config client-ssl.properties
二宙暇、filebeat的一些配置
最終filebeat配置：
filebeat.inputs:
- type: log
? enabled: true
? paths:
? ? - /var/log/nginx/archived/*/*
filebeat.config.modules:
? path: ${path.config}/modules.d/*.yml
? reload.enabled: false
setup.template.settings:
? index.number_of_shards: 1
setup.kibana:
output.kafka:
? #指定版本號
? version: "0.9.0"
? enabled: true
? hosts:
? ? - 172.28.15.231:9093
? topic: "test-log"
? version: "2.0.0"
? ssl:
? ? enabled: true
? ? certificate: '/data/xxx/kafka_ca_cert/filebeat/filebeat.crt'
? ? key: '/data/xxx/kafka_ca_cert/filebeat/filebeat.key'
? ? certificate_authorities: ['/data/xxx/kafka_ca_cert/serverpub.pem']
? ? supported_protocols: [TLSv1.1, TLSv1.2, TLSv1.3]
? ? verification_mode: full
? ? endpoint:
? ? ? identification:
? ? ? ? algorithm:
processors:
? - add_host_metadata: ~
? - add_cloud_metadata: ~
filebeat 的一些配置的官方文檔：
https://www.elastic.co/guide/en/beats/filebeat/master/kafka-output.html
https://www.elastic.co/guide/en/beats/filebeat/master/configuration-ssl.html
https://blog.csdn.net/qq_41926119/article/details/104510481
EFK接入kafka消息隊列:
https://www.ucloud.cn/yun/34441.html
需要注意的是儒洛，filebeat配置的是pem證書鞍恢，kafka和logstash的kafka-input插件用的是jks證書~~~因此房维，證書生成工具最好需要能夠同時生成這兩種證書沼瘫。
常見錯誤集錦：
https://discuss.elastic.co/t/tls-for-filebeat-kafka-output/58756/11
其他配置：
Filebeat與Logstash配置SSL加密通信：
http://www.manongjc.com/detail/14-jogxwlmgmlwauxs.html
https://www.cnblogs.com/sanduzxcvbnm/p/12055038.html
filebeat常用配置項：https://www.elastic.co/guide/en/beats/filebeat/7.x/kafka-output.html
SSL配置項： https://www.elastic.co/guide/en/beats/filebeat/7.16/configuration-ssl.html#client-verification-mode
