本文和之前的文集有較大交集,有不明確的術(shù)語(yǔ)倒槐、配置等旬痹,請(qǐng)參閱:Step by Step 實(shí)現(xiàn)基于 Cloudera 5.8.2 的企業(yè)級(jí)安全大數(shù)據(jù)平臺(tái) - 傳輸層加密配置 - Hadoop 組件傳輸層加密 和 Step by Step 實(shí)現(xiàn)基于 Cloudera 5.8.2 的企業(yè)級(jí)安全大數(shù)據(jù)平臺(tái) - 傳輸層加密配置 - Clouder Manager 組件傳輸層加密。
啰嗦的前言
今天上班發(fā)現(xiàn) HUE 超級(jí)管理員登錄時(shí)在提示頁(yè)面報(bào)錯(cuò):
hadoop.hdfs_clusters.default.webhdfs_url:
Current value: https://hostname:14000/webhdfs/v1 Failed to access filesystem root
Resource Manager:
Failed to contact an active Resource Manager: No Resource Manager are available.
Hive:
Failed to access Hive warehouse: /user/hive/warehouse
Impala:
Failed to authenticate to Impalad, check authentication configurations.
Oozie Editor/Dashboard:
The app won't work without a running Oozie Server.
Pig Editor:
The app won't work without a running Oozie Server.
蛋碎讨越,基本所有服務(wù)都無(wú)法連接了两残,而事實(shí)上所有服務(wù)的服務(wù)端都工作的好好的。這次故障的影響面是:
- FileBrowser 服務(wù)不可用把跨;
- JobBrowser 服務(wù)不可用人弓;
- File ACLs 服務(wù)不可用;
查看 Server Log
着逐,都是一致的報(bào)錯(cuò):
('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')
對(duì)崔赌,筆者的集群之前是開(kāi)啟了 TLS/SSL 加密傳輸,但是什么變更都沒(méi)有耸别,為什么突然報(bào)認(rèn)證失敗呢健芭?參考這篇文章 Step by Step 實(shí)現(xiàn)基于 Cloudera 5.8.2 的企業(yè)級(jí)安全大數(shù)據(jù)平臺(tái) - 傳輸層加密配置 - Hadoop 組件傳輸層加密 回顧下是如何配置 HUE TLS/SSL 加密的。當(dāng)時(shí)在使用 keytool -genkeypair
時(shí)沒(méi)有指定有效期太雨,所以默認(rèn)是90天吟榴。也許是自簽名SSL證書(shū)過(guò)期導(dǎo)致的,但是挖掘了所有可用的日志囊扳,都沒(méi)有告訴筆者類似 Certificate Expired
的信息 ?_?吩翻,開(kāi)源就是要折騰,那么我們就來(lái)嘗試更新證書(shū)吧锥咸,let's have a try狭瞎。
然后事實(shí)證明筆者的判斷是正確的。面對(duì)這樣一個(gè)以后可以導(dǎo)致生產(chǎn)故障的原因(運(yùn)維界因此誕生了一個(gè)新的笑話)搏予,筆者表示捂臉掩面熊锭,用一句話來(lái)開(kāi)脫一下:I'm a newbie in SSL…...下面來(lái)說(shuō)說(shuō)如何去進(jìn)行證書(shū)的更新。
證書(shū)的更新和配置一樣雪侥,主要分為兩塊:Clouder Manager 組件證書(shū)更新碗殷、Hadoop 服務(wù)證書(shū)更新。這一次筆者希望生成一個(gè)1年期的證書(shū)速缨,并且在第9個(gè)月的時(shí)候進(jìn)行提前更新锌妻,本文主要闡述如何進(jìn)行手動(dòng)證書(shū)更新。
證書(shū)更新必備條件
- CM Server 和 所有 CM Agent 都進(jìn)行了 ssh 免密鑰登錄配置旬牲,假設(shè)我們知道所有機(jī)器的用戶名和密碼仿粹,我們可以很方便的通過(guò)
ssh-copy-id
來(lái)實(shí)現(xiàn)搁吓,假設(shè)我們不知道,那么可以使用nc
進(jìn)行公鑰拷貝的方式進(jìn)行打通吭历;
# 在接收端打開(kāi)一個(gè)可用端口以接收數(shù)據(jù)
nc -l ${port} | tar xf -
# 在發(fā)送端發(fā)送數(shù)據(jù)
tar cf - ${file} | nc ${reciever_ip} ${port}
- CM Server 安裝了
pssh
堕仔;
sudo yum install -y pssh
- 所有節(jié)點(diǎn)安裝了
expect
;
pssh -h list_all "sudo yum install -y expect tcl"
- Cloudera Manager 的超級(jí)管理員權(quán)限晌区,用于登錄控制臺(tái)(也就是7183端口)進(jìn)行配置管理和重啟服務(wù)摩骨;
- 假設(shè)你已經(jīng)按照 Step by Step 實(shí)現(xiàn)基于 Cloudera 5.8.2 的企業(yè)級(jí)安全大數(shù)據(jù)平臺(tái) - 傳輸層加密配置 - Clouder Manager 組件傳輸層加密 和 Step by Step 實(shí)現(xiàn)基于 Cloudera 5.8.2 的企業(yè)級(jí)安全大數(shù)據(jù)平臺(tái) - 傳輸層加密配置 - Hadoop 組件傳輸層加密 進(jìn)行了 TLS/SSL 加密配置,因?yàn)橄挛牡暮芏嗦窂蕉际沁@兩篇文章中
mkdir
好的契讲;
和前面所有系列文章一樣仿吞,我們的操作是在 admin 用戶下進(jìn)行的滑频。
自簽名 SSL 證書(shū)更新
假設(shè)集群包含如下機(jī)器列表捡偏,其中 v001001.dc1.domain.com
為 Cloudera Server,我們有一個(gè)文件list_all
包含了所有這些機(jī)器峡迷,一行一臺(tái)機(jī)器的 hostname银伟,用于 pssh
時(shí)進(jìn)行批處理,今天就假設(shè)我們只有3臺(tái)機(jī)器:
v001001.dc1.domain.com
v001002.dc1.domain.com
v001003.dc1.domain.com
Step1. 生成密鑰對(duì)和自簽名證書(shū)(所有節(jié)點(diǎn))
在所有節(jié)點(diǎn)生成密鑰對(duì)和自簽名證書(shū)绘搞,并將它們存儲(chǔ)在 JKS 格式的密鑰庫(kù)(cms.keystore.${DATE}.${HOSTNAME}
)中彤避,這樣我們也方便計(jì)算過(guò)期時(shí)間。 將 -keypass
設(shè)置為與 -storepass
相同的值:
pscp -h list_all generate_jks.sh /tmp
pssh -h list_all "sudo /usr/bin/bash /tmp/generate_jks.sh"
其中腳本 generate_jks.sh
的內(nèi)容如下夯辖,請(qǐng)對(duì) JAVA_HOME
琉预、STORE_PASS
進(jìn)行賦值:
#!/bin/bash
JAVA_HOME=${JAVA_HOME}
STORE_PASS=${STORE_PASS}
BASE_SECURITY_PATH=/opt/cloudera/security/
HOSTNAME=`hostname -f`
DATE=`date "+%Y-%m-%d"`
KEYSTORE_NAME=cms.keystore.${DATE}
KEYSTORE_ALIAS=cms.${DATE}
sudo ${JAVA_HOME}/bin/keytool -genkeypair -keystore ${BASE_SECURITY_PATH}/jks/${KEYSTORE_NAME}.${HOSTNAME} -keyalg RSA -alias ${KEYSTORE_ALIAS}.${HOSTNAME} -dname "CN=${HOSTNAME},OU=Bigdata,O=Domain,L=Hangzhou,ST=Zhejiang,C=CN" -storepass ${STORE_PASS} -keypass ${STORE_PASS} -validity 365
# 修改證書(shū)名稱,更為通用蒿褂,用于 hadoop 服務(wù)
sudo cp ${BASE_SECURITY_PATH}/jks/${KEYSTORE_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/jks/${KEYSTORE_NAME}.hadoopagent
Step2. 生成可信庫(kù)并初始化密碼(所有節(jié)點(diǎn))
將缺省 Java 信任庫(kù)(cacerts
)復(fù)制到備用系統(tǒng)信任庫(kù)(jssecacerts.${DATE}.${HOSTNAME}
)圆米,自簽名證書(shū)導(dǎo)入到 jssecacerts.${DATE}.${HOSTNAME}
,而不修改默認(rèn) cacerts
文件啄栓,并從密鑰庫(kù)(cms.keystore.${DATE}.${HOSTNAME}
)導(dǎo)出證書(shū):
pscp -h list_all generate_ca.sh /tmp
pssh -h list_all "sudo /usr/bin/bash /tmp/generate_ca.sh"
其中腳本 generate_ca.sh
的內(nèi)容如下娄帖,請(qǐng)對(duì) JAVA_HOME
、STORE_PASS
進(jìn)行賦值:
#!/bin/bash
JAVA_HOME=${JAVA_HOME}
BASE_SECURITY_PATH=/opt/cloudera/security/
STORE_PASS=${STORE_PASS}
HOSTNAME=`hostname -f`
DATE=`date "+%Y-%m-%d"`
KEYSTORE_NAME=cms.keystore.${DATE}
KEYSTORE_ALIAS=cms.${DATE}
SELF_CA_NAME=selfsigned.cer.${DATE}
PEM_NAME=cmhost.pem.${DATE}
TRUSTSTORE_NAME=jssecacerts.${DATE}
# 生成可信庫(kù)
sudo cp ${JAVA_HOME}/jre/lib/security/cacerts ${JAVA_HOME}/jre/lib/security/${TRUSTSTORE_NAME}.${HOSTNAME}
# 導(dǎo)出自簽名證書(shū)
sudo ${JAVA_HOME}/bin/keytool -export -alias ${KEYSTORE_ALIAS}.${HOSTNAME} -keystore ${BASE_SECURITY_PATH}/jks/${KEYSTORE_NAME}.${HOSTNAME} -rfc -file ${BASE_SECURITY_PATH}/${SELF_CA_NAME}.${HOSTNAME} -storepass ${STORE_PASS}
# 拷貝自簽名證書(shū)至 x509 目錄昙楚,并賦權(quán)
sudo cp ${BASE_SECURITY_PATH}/${SELF_CA_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${PEM_NAME}.${HOSTNAME}
sudo chown cloudera-scm:cloudera-scm ${BASE_SECURITY_PATH}/x509/${PEM_NAME}.${HOSTNAME}
sudo cp ${BASE_SECURITY_PATH}/${SELF_CA_NAME}.${HOSTNAME} /tmp
修改默認(rèn) jssecacerts.${DATE}.${HOSTNAME}
庫(kù)密碼:
pscp -h list_all modify_jssecacerts_passwd.expect /tmp
pscp -h list_all modify_jssecacerts_passwd.sh /tmp
pssh -h list_all "sudo /bin/bash /tmp/modify_jssecacerts_passwd.sh"
其中腳本 modify_jssecacerts_passwd.expect
的內(nèi)容如下近速,請(qǐng)對(duì) JAVA_HOME
進(jìn)行賦值:
#!/usr/bin/expect -f
set JAVA_HOME ${JAVA_HOME}
set timeout 300
set current_passwd [lindex $argv 0]
set new_passwd [lindex $argv 1]
set hostname [lindex $argv 2]
set truststore_name [lindex $argv 3]
spawn sudo ${JAVA_HOME}/bin/keytool -storepasswd -keystore ${JAVA_HOME}/jre/lib/security/${truststore_name}.${hostname}
sleep 1
expect "Enter keystore password:"
send "${current_passwd}\r"
expect "New keystore password:"
send "${new_passwd}\r"
expect "Re-enter new keystore password:"
send "${new_passwd}\r"
expect eof
其中腳本 modify_jssecacerts_passwd.sh
的內(nèi)容如下,${CURRENT_PASSWD}
為 changeit
, ${NEW_PASSWD}
為新指定的密碼堪旧,需要讀者自行設(shè)置:
#/bin/bash
CURRENT_PASSWD=${CURRENT_PASSWD}
NEW_PASSWD=${NEW_PASSWD}
HOSTNAME=`hostname -f`
DATE=`date "+%Y-%m-%d"`
TRUSTSTORE_NAME=jssecacerts.${DATE}
sudo /usr/bin/expect /tmp/modify_jssecacerts_passwd.expect ${CURRENT_PASSWD} ${NEW_PASSWD} ${HOSTNAME} ${TRUSTSTORE_NAME}
Step3. 拷貝 Cloudera Server 的 PEM 格式證書(shū)至所有節(jié)點(diǎn)
上一步我們生成了 PEM 格式的證書(shū) ${BASE_SECURITY_PATH}/x509/${PEM_NAME}.${HOSTNAME}
削葱,我們需要把它拷貝至所有節(jié)點(diǎn)的相應(yīng)目錄下:
pscp -h list_all ${BASE_SECURITY_PATH}/x509/${PEM_NAME}.v001001.dc1.domain.com /tmp
pssh -h list_all "sudo cp /tmp/${PEM_NAME}.v001001.dc1.domain.com ${BASE_SECURITY_PATH}/x509/"
pssh -h list_all "sudo chown cloudera-scm:cloudera-scm ${BASE_SECURITY_PATH}/x509/${PEM_NAME}.v001001.dc1.domain.com"
Step4. 導(dǎo)入自簽名證書(shū)至本地信任庫(kù)(所有節(jié)點(diǎn))
在每臺(tái)機(jī)器上執(zhí)行,把客戶端的自簽名證書(shū)導(dǎo)入到客戶端本地的信任庫(kù):
pscp -h list_all import_ca.expect /tmp
pscp -h list_all import_ca.sh /tmp
pssh -h list_all "sudo /bin/bash /tmp/import_ca.sh"
其中腳本 import_ca.expect
的內(nèi)容如下淳梦,請(qǐng)對(duì) JAVA_HOME
進(jìn)行賦值:
#!/usr/bin/expect -f
set JAVA_HOME ${JAVA_HOME}
set timeout 300
set storepass [lindex $argv 0]
set hostname [lindex $argv 1]
set date [lindex $argv 2]
set keystore_alias cms.${date}
set self_ca_name selfsigned.cer.${date}
set truststore_name jssecacerts.${date}
spawn sudo ${JAVA_HOME}/bin/keytool -import -alias ${keystore_alias}.${hostname} -file /tmp/${self_ca_name}.${hostname} -keystore ${JAVA_HOME}/jre/lib/security/${truststore_name}.${hostname} -storepass ${storepass}
sleep 1
expect "Trust this certificate?"
send "yes\r"
expect eof
其中腳本 import_ca.sh
的內(nèi)容如下析砸,請(qǐng)對(duì) PASSWD
進(jìn)行賦值,為之前我們新指定的 jssecacerts
信任庫(kù)密碼:
#!/bin/bash
PASSWD=${PASSWD}
HOSTNAME=`hostname -f`
DATE=`date "+%Y-%m-%d"`
sudo /usr/bin/expect /tmp/import_ca.expect ${PASSWD} ${HOSTNAME} ${DATE}
Step5. 生成 JKS 公共可信庫(kù)( Cloudera Server 上執(zhí)行)
把所有客戶端的證書(shū)注入公共庫(kù)谭跨,并且分發(fā):
sudo /bin/bash modify_jssecacerts_public_passwd.sh
/bin/bash get_ca_from_slave.sh
sudo /bin/bash import_ca_public.sh
pscp -h list_agents_hostname ${JAVA_HOME}/jre/lib/security/jssecacerts.${DATE}.public /tmp
pssh -h list_agents_hostname "sudo cp /tmp/jssecacerts.${DATE}.public ${JAVA_HOME}/jre/lib/security/"
其中腳本modify_jssecacerts_public_passwd.sh
的內(nèi)容如下干厚,請(qǐng)對(duì) JAVA_HOME
李滴、CURRENT_PASSWD
、NEW_PASSWD
進(jìn)行賦值蛮瞄,CURRENT_PASSWD
為 changeit
:
#!/bin/bash
JAVA_HOME=${JAVA_HOME}
CURRENT_PASSWD=${CURRENT_PASSWD}
NEW_PASSWD=${NEW_PASSWD}
DATE=`date "+%Y-%m-%d"`
TRUSTSTORE_NAME=jssecacerts.${DATE}
sudo cp ${JAVA_HOME}/jre/lib/security/cacerts ${JAVA_HOME}/jre/lib/security/${TRUSTSTORE_NAME}.public
sudo cp modify_jssecacerts_public_passwd.expect /tmp
sudo /usr/bin/expect /tmp/modify_jssecacerts_public_passwd.expect ${CURRENT_PASSWD} ${NEW_PASSWD} ${TRUSTSTORE_NAME}
其中腳本 get_ca_from_slave.sh
的內(nèi)容如下:
#!/bin/bash
DATE=`date "+%Y-%m-%d"`
SELF_CA_NAME=selfsigned.cer.${DATE}
for slave in `cat list_agents_hostname`
do
scp ${slave}:/tmp/${SELF_CA_NAME}.${slave} /tmp
done
其中腳本 import_ca_public.sh
的內(nèi)容如下:
#!/bin/bash
DATE=`date "+%Y-%m-%d"`
SELF_CA_NAME=selfsigned.cer.${DATE}
NEW_PASSWD=${new_passwd}
sudo cp import_ca_public.expect /tmp
for slave in `cat list_agents_hostname`
do
sudo /usr/bin/expect /tmp/import_ca_public.expect ${NEW_PASSWD} ${slave} ${DATE}
done
其中腳本 modify_jssecacerts_public_passwd.expect
的內(nèi)容如下所坯,請(qǐng)對(duì) JAVA_HOME
進(jìn)行賦值:
#!/usr/bin/expect -f
set JAVA_HOME ${JAVA_HOME}
set timeout 300
set current_passwd [lindex $argv 0]
set new_passwd [lindex $argv 1]
set truststore_name [lindex $argv 2]
spawn sudo ${JAVA_HOME}/bin/keytool -storepasswd -keystore ${JAVA_HOME}/jre/lib/security/${truststore_name}.public
sleep 1
expect "Enter keystore password:"
send "${current_passwd}\r"
expect "New keystore password:"
send "${new_passwd}\r"
expect "Re-enter new keystore password:"
send "${new_passwd}\r"
expect eof
其中腳本 import_ca_public.expect
的內(nèi)容如下,請(qǐng)對(duì) JAVA_HOME
進(jìn)行賦值:
#!/usr/bin/expect -f
set JAVA_HOME ${JAVA_HOME}
set timeout 300
set storepass [lindex $argv 0]
set hostname [lindex $argv 1]
set date [lindex $argv 2]
set keystore_alias cms.${date}
set self_ca_name selfsigned.cer.${date}
set truststore_name jssecacerts.${date}
spawn sudo ${JAVA_HOME}/bin/keytool -import -alias ${keystore_alias}.${hostname} -file /tmp/${self_ca_name}.${hostname} -keystore ${JAVA_HOME}/jre/lib/security/${truststore_name}.public -storepass ${storepass}
sleep 1
expect "Trust this certificate?"
send "yes\r"
expect eof
Step6. 生成 PEM 格式證書(shū)和密鑰(所有節(jié)點(diǎn))
pscp -h list_all init_p12.sh /tmp
pssh -h list_all "sudo /bin/bash /tmp/init_p12.sh"
其中腳本init_p12.sh
內(nèi)容如下挂捅,請(qǐng)對(duì) JAVA_HOME
PASSWD
進(jìn)行賦值芹助,其中 PASSWD
為之前我們初始化的 JKS 密鑰庫(kù)密碼:
#!/bin/bash
JAVA_HOME=${JAVA_HOME}
BASE_SECURITY_PATH=/opt/cloudera/security
HOSTNAME=`hostname -f`
PASSWD=${PASSWD}
DATE=`date "+%Y-%m-%d"`
KEYSTORE_NAME=cms.keystore.${DATE}
KEYSTORE_ALIAS=cms.${DATE}
P12_NAME=cms.pem.${DATE}
P12_KEY_NAME=cms.key.${DATE}
AGENTKEY_PW_NAME=agentkey.pw.${DATE}
sudo -u cloudera-scm ${JAVA_HOME}/bin/keytool -importkeystore -srckeystore ${BASE_SECURITY_PATH}/jks/${KEYSTORE_NAME}.${HOSTNAME} \
-srcstorepass ${PASSWD} -srckeypass ${PASSWD} -destkeystore /tmp/${KEYSTORE_NAME}.p12.${HOSTNAME} \
-deststoretype PKCS12 -srcalias ${KEYSTORE_ALIAS}.${HOSTNAME} -deststorepass ${PASSWD} -destkeypass ${PASSWD}
sudo -u cloudera-scm openssl pkcs12 -in /tmp/${KEYSTORE_NAME}.p12.${HOSTNAME} -passin pass:${PASSWD} -nokeys \
-out ${BASE_SECURITY_PATH}/x509/${P12_NAME}.${HOSTNAME}
sudo -u cloudera-scm openssl pkcs12 -in /tmp/${KEYSTORE_NAME}.p12.${HOSTNAME} -passin pass:${PASSWD} -nocerts \
-out ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.${HOSTNAME} -passout pass:${PASSWD}
sudo echo "${PASSWD}" > /tmp/${AGENTKEY_PW_NAME}.${HOSTNAME}
sudo cp /tmp/${AGENTKEY_PW_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/
sudo chown root.root ${BASE_SECURITY_PATH}/x509/${AGENTKEY_PW_NAME}.${HOSTNAME}
sudo chmod 644 ${BASE_SECURITY_PATH}/x509/${AGENTKEY_PW_NAME}.${HOSTNAME}
sudo -u cloudera-scm cp ${BASE_SECURITY_PATH}/x509/${P12_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${P12_NAME}.cmagent
sudo -u cloudera-scm cp ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.cmagent
sudo cp ${BASE_SECURITY_PATH}/x509/${AGENTKEY_PW_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${AGENTKEY_PW_NAME}.cmagent
sudo -u cloudera-scm cp ${BASE_SECURITY_PATH}/x509/${P12_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${P12_NAME}.hadoopagent
sudo -u cloudera-scm cp ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.hadoopagent
sudo cp ${BASE_SECURITY_PATH}/x509/${AGENTKEY_PW_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${AGENTKEY_PW_NAME}.hadoopagent
Step7. 生成 PEM 公共證書(shū)庫(kù)( Cloudera Server上執(zhí)行,并且分發(fā)至所有節(jié)點(diǎn) )
#!/bin/bash
BASE_SECURITY_PATH=/opt/cloudera/security
DATE=`date "+%Y-%m-%d"`
P12_NAME=cms.pem.${DATE}
for slave in `cat list_agents_hostname`
do
scp ${slave}:${BASE_SECURITY_PATH}/x509/${P12_NAME}.${slave} /tmp
done;
for slave in `cat list_agents_hostname`
do
pem_list="${pem_list} /tmp/${P12_NAME}.${slave}"
done
cat ${pem_list} > /tmp/${P12_NAME}.public
pscp -h list_all /tmp/${P12_NAME}.public /tmp
pssh -h list_all "sudo cp /tmp/${P12_NAME}.public ${BASE_SECURITY_PATH}/x509"
pssh -h list_all "sudo chown cloudera-scm.cloudera-scm ${BASE_SECURITY_PATH}/x509/${P12_NAME}.public"
Step8. 生成免密碼的 Nginx 的 PEM 格式證書(shū)和密鑰( Nginx節(jié)點(diǎn) )
假設(shè) Nginx 服務(wù)器 (v001001.dc1.domain.com) 已經(jīng)生成過(guò) JKS 密鑰闲先,我們需要配置 Nginx 免密鑰訪問(wèn) HUE Server:
#!/bin/bash
BASE_SECURITY_PATH=/opt/cloudera/security
DATE=`date "+%Y-%m-%d"`
P12_NAME=cms.pem.${DATE}
P12_KEY_NAME=cms.key.${DATE}
sudo cp ${BASE_SECURITY_PATH}/x509/${P12_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${P12_NAME}.nginx
sudo cp ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.${HOSTNAME} ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.nginx
sudo cp ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.nginx ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.nginx.bak
sudo openssl rsa -in ${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.nginx.bak -out ${BASE_SECURITY_PATH}x509/${P12_KEY_NAME}.nginx
重啟前確認(rèn)文件
重啟前我們確保下状土,以下文件都已經(jīng)生成:
- 所有節(jié)點(diǎn)上存在,本機(jī)的密鑰對(duì)和自簽名證書(shū)
${BASE_SECURITY_PATH}/jks/${KEYSTORE_NAME}.${HOSTNAME}
伺糠,其中只有 Cloudera Server 的該文件會(huì)用于 :Cloudera Admin Console 的 HTTPS 加密蒙谓; - 所有節(jié)點(diǎn)上存在,本機(jī)的密鑰對(duì)和自簽名證書(shū)
${BASE_SECURITY_PATH}/jks/${KEYSTORE_NAME}.hadoopagent
训桶,用于 Hadoop 服務(wù)傳輸加密 (服務(wù)端)累驮,包括 HDFS、YARN舵揭、HBase谤专、HiveServer2、Oozie午绳、HDFS HTTPFS 等置侍; - 所有節(jié)點(diǎn)上存在,包含本機(jī)自簽名證書(shū)的可信庫(kù)
${JAVA_HOME}/jre/lib/security/${TRUSTSTORE_NAME}.${HOSTNAME}
拦焚; - 所有節(jié)點(diǎn)上存在蜡坊,包含所有主機(jī)自簽名證書(shū)的可信庫(kù)
${JAVA_HOME}/jre/lib/security/${TRUSTSTORE_NAME}.public
,用于 Cloudera Managaer Services 的 SSL 加密耕漱,及 Hadoop 服務(wù)傳輸加密 (客戶端)算色,包括 HDFS、YARN螟够、Oozie灾梦、HDFS HTTPFS; - 所有節(jié)點(diǎn)上存在妓笙,包含本機(jī) PEM 格式的證書(shū)若河、密鑰、密鑰key寞宫,用于 Cloudera Server 對(duì) Agent 的 SSL 證書(shū)認(rèn)證 或 Hadoop 服務(wù)傳輸加密萧福,包括 HUE、 Impala:
-
${BASE_SECURITY_PATH}/x509/${P12_NAME}.cmagent
辈赋; -
${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.cmagent
鲫忍; -
${BASE_SECURITY_PATH}/x509/${AGENTKEY_PW_NAME}.cmagent
膏燕; -
${BASE_SECURITY_PATH}/x509/${P12_NAME}.hadoopagent
; -
${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.hadoopagent
悟民; -
${BASE_SECURITY_PATH}/x509/${AGENTKEY_PW_NAME}.hadoopagent
坝辫;
-
- 所有節(jié)點(diǎn)上存在,包含 Cloudera Server 的 PEM 格式的證書(shū)
${BASE_SECURITY_PATH}/x509/${PEM_NAME}.v001001.dc1.domain.com
射亏,用于 Cloudera Agent 對(duì) Server 的 SSL 證書(shū)認(rèn)證近忙; - 所有節(jié)點(diǎn)上存在,包含所有主機(jī) PEM 格式證書(shū)的公共證書(shū)庫(kù)
${BASE_SECURITY_PATH}/x509/${P12_NAME}.public
智润,用于 HUE及舍、Impala 傳輸加密; - Nginx 節(jié)點(diǎn)上存在窟绷,包含本機(jī) PEM 格式的證書(shū)和密鑰
${BASE_SECURITY_PATH}/x509/${P12_NAME}.nginx
${BASE_SECURITY_PATH}/x509/${P12_KEY_NAME}.nginx
锯玛,密鑰文件需要免密碼,用于 負(fù)載均衡钾麸;
Cloudera Manager 組件重啟
重啟前確認(rèn)配置
以下所有?除密碼相關(guān)的配置項(xiàng)更振,筆者都進(jìn)行了實(shí)體化,也就是帶入了真實(shí)值饭尝,方便讀者進(jìn)行學(xué)習(xí)。${JAVA_HOME}路徑請(qǐng)讀者自行替換献宫。
因?yàn)楣P者的證書(shū)是在3月14日生成的钥平,所以你需要把它替換成你的對(duì)應(yīng)日期。
Admin Console 的 HTTPS 加密訪問(wèn)配置
- 登錄到 Cloudera Manager Administration Console http://192.168.1.1:7183 姊途;
- 選擇
Administration
->Settings
涉瘾; - 點(diǎn)擊
Security
類目; - 確認(rèn)如下配置:
Use TLS Encryption for Admin Console = true
Path to TLS Keystore File = /opt/cloudera/security/jks/cms.keystore.2017-03-14.v001001.dc1.domain.com
Keystore Password = ${KEYSTORE_PASSWORD}
Cloudera Management Services 的 SSL 加密配置
- 打開(kāi) Cloudera Manager Administration Console http://192.168.1.1:7183 選擇
Cloudera Management Service
捷兰;
- 點(diǎn)擊
Configuration
選項(xiàng)卡立叛;
- 選擇
Scope
->Cloudera Management Service (Service-Wide)
;
- 選擇
Category
->Security
贡茅;
- 確認(rèn)以下 TLS/SSL 配置秘蛇,其中
TRUSTSOTRE_FILE_PASSWORD
為之前我們?cè)O(shè)置的jssecacerts.2017-03-14.public
的新密碼:
TLS/SSL Client Truststore File Location = $JAVA_HOME/jre/lib/security/jssecacerts.2017-03-14.public
TLS/SSL Client Truststore File Password = ${TRUSTSOTRE_FILE_PASSWORD}
Cloudera Agent 的 TLS 配置
登錄 Cloudera Manager Admin Console 選擇 Administration
-> Settings
-> Security
確認(rèn)如下配置:
Use TLS Encryption for Agents = TRUE
確認(rèn)每臺(tái) Cloudera Agent 配置是否正確:
pssh -h list_all -P "grep 'use_tls=1' /etc/cloudera-scm-agent/config.ini | wc -l"
如果返回值都為1,則表示OK顶考。
Cloudera Agent 對(duì) Server 的 SSL 證書(shū)認(rèn)證相關(guān)配置
所有節(jié)點(diǎn) /etc/cloudera-scm-agent/config.ini
配置文件赁还,確認(rèn)如下屬性:
pssh -h list_all -P "grep 'verify_cert_file=/opt/cloudera/security/x509/cmhost.pem.2017-03-14.v001001.dc1.domain.com' /etc/cloudera-scm-agent/config.ini | wc -l"
如果返回值都為1,則表示OK驹沿。
Cloudera Server 對(duì) Agent 的 SSL 證書(shū)認(rèn)證相關(guān)配置
所有節(jié)點(diǎn) /etc/cloudera-scm-agent/config.ini
配置文件艘策,確認(rèn)如下屬性:
pssh -h list_all -P "grep 'client_cert_file=/opt/cloudera/security/x509/cms.pem.2017-03-14.cmagent' /etc/cloudera-scm-agent/config.ini | wc -l"
pssh -h list_all -P "grep 'client_key_file=/opt/cloudera/security/x509/cms.key.2017-03-14.cmagent' /etc/cloudera-scm-agent/config.ini | wc -l"
pssh -h list_all -P "grep 'client_keypw_file=/opt/cloudera/security/x509/agentkey.pw.2017-03-14.cmagent' /etc/cloudera-scm-agent/config.ini | wc -l"
如果返回值都為1,則表示OK渊季。
- 登錄 Cloudera Manager Admin Console朋蔫;
- 選擇
Administration
->Settings
罚渐;
- 點(diǎn)擊
Security
類目;
- 確認(rèn)以下 TLS 屬性:
Use TLS Authentication of Agents to Server = TRUE
重啟服務(wù)
重啟 Server 和 Agent:
sudo /opt/cm-5.8.2/etc/init.d/cloudera-scm-server restart
pssh -h list_agents "sudo /bin/systemctl restart cloudera-scm-agent"
在 Cloudera Manager Admin Console驯妄,打開(kāi) Hosts
頁(yè)面搅轿。 如果 Agent 心跳正常,則說(shuō)明TLS 加密正常工作富玷。
重啟 Cloudera Management Services璧坟, 這一步直接可以在 Cloudera Manager Administration Console 操作。
Hadoop 服務(wù)的自簽名 SSL 證書(shū)更新
和 Cloudera 組件類似赎懦,我們需要確認(rèn)配置后才能進(jìn)行服務(wù)重啟雀鹃。
以下所有配置,請(qǐng)注意替換 ${JAVA_HOME} 為讀者自己的路徑励两;
以下所有配置?黎茎,除了 Nginx 配置之外,都是在 Cloudera Manager Admin Console 中進(jìn)行配置的当悔,也就是 http://192.168.1.1:7183傅瞻;
重啟前確認(rèn)配置
HDFS 配置確認(rèn)
PASSWD
為之前設(shè)置的 JKS 密碼:
ssl.server.keystore.location=/opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
ssl.server.keystore.password=${PASSWD}
ssl.server.keystore.keypassword=${PASSWD}
ssl.client.truststore.location=${JAVA_HOME}/jre/lib/security/jssecacerts.2017-03-14.public
ssl.client.truststore.password=${PASSWD}
hadoop.ssl.enabled=true
dfs.datanode.address = 1024
dfs.data.transfer.protection = privacy
Enable TLS/SSL for HttpFS = true
HttpFS TLS/SSL Server JKS Keystore File Location = /opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
HttpFS TLS/SSL Server JKS Keystore File Password = ${PASSWD}
HttpFS TLS/SSL Certificate Trust Store File = ${JAVA_HOME}/jre/lib/security/jssecacerts.2017-03-14.public
HttpFS TLS/SSL Certificate Trust Store Password = ${PASSWD}
YARN 配置確認(rèn)
PASSWD
為上一篇中設(shè)置的 JKS 密碼:
ssl.server.keystore.location=/opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
ssl.server.keystore.password=${PASSWD}
ssl.server.keystore.keypassword=${PASSWD}
ssl.client.truststore.location=${JAVA_HOME}/jre/lib/security/jssecacerts.2017-03-14.public
ssl.client.truststore.password=${PASSWD}
HBase 配置確認(rèn)
PASSWD
為上一篇中設(shè)置的 JKS 密碼:
hdaoop.ssl.enabled, hbase.ssl.enabled = true
ssl.server.keystore.location = /opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
ssl.server.keystore.password=${PASSWD}
ssl.server.keystore.keypassword=${PASSWD}
hbase.rest.ssl.enabled = true
hbase.rest.ssl.keystore.store = /opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
hbase.rest.ssl.keystore.password = ${PASSWD}
hbase.rest.ssl.keystore.keypassword = ${PASSWD}
hbase.thrift.ssl.enabled = true
hbase.thrift.ssl.keystore.store = /opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
hbase.thrift.ssl.keystore.password = ${PASSWD}
hbase.thrift.ssl.keystore.keypassword = ${PASSWD}
Hive 配置確認(rèn)
PASSWD
為上一篇中設(shè)置的 JKS 密碼,請(qǐng)注意替換PASSWD
:
hive.server2.enable.SSL, hive.server2.use.SSL = true
hive.server2.keystore.path = /opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
hive.server2.keystore.password =${PASSWD}
hive.server2.webui.use.ssl = true
hive.server2.webui.keystore.password = ${PASSWD}
hive.server2.webui.keystore.path = /opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
Impala 配置確認(rèn)
請(qǐng)注意替換PASSWD
:
webserver_certificate_file = /opt/cloudera/security/jks/cms.pem.2017-03-14.hadoopagent
webserver_private_key_file = /opt/cloudera/security/jks/cms.key.2017-03-14.hadoopagent
webserver_private_key_password_cmd = ${PASSWD}
ldap_ca_certificate = /opt/cloudera/security/jks/cms.pem.2017-03-14.hadoopagent
client_services_ssl_enabled = true
ssl_server_certificate = webserver_certificate_file = /opt/cloudera/security/x509/cms.pem.2017-03-14.hadoopagent
ssl_private_key = webserver_private_key_file = /opt/cloudera/security/x509/cms.key.2017-03-14.hadoopagent
ssl_private_key_password_cmd = webserver_private_key_password_cmd = ${PASSWD}
ssl_client_ca_certificate = /opt/cloudera/security/x509/cms.key.2017-03-14.public
HUE 配置確認(rèn)
請(qǐng)注意替換 PASSWD
:
ssl_cacerts = /opt/cloudera/security/x509/cms.pem.2017-03-14.public
Enable TLS/SSL for Hue = true
ssl_certificate = /opt/cloudera/security/x509/cms.pem.2017-03-14.hadoopagent
ssl_private_key = /opt/cloudera/security/x509/cms.key.2017-03-14.hadoopagent
ssl_password = ${PASSWD}
因?yàn)槭亲院灻荑€盲憎,需要修改環(huán)境變量嗅骄,在 Hue Service Environment Advanced Configuration Snippet
中添加:
REQUESTS_CA_BUNDLE = /opt/cloudera/security/x509/cms.pem.2017-03-14.public
確認(rèn)已經(jīng)配置 HUE Server 和 HiveServer2 之間的加密,通過(guò) Cloudera Manager Admin Console 對(duì) hue.ini 進(jìn)行追加配置饼疙,配置項(xiàng)為 Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini
:
[beeswax]
[[ssl]]
enabled = true
cacerts = /opt/cloudera/security/x509/cms.pem.2017-03-14.public
validate = true
確認(rèn) HUE 配置溺森,在 hue.ini 中添加和 Impala 的傳輸加密配置,配置項(xiàng)為 Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini
:
[impala]
[[ssl]]
enabled = true
cacerts = /opt/cloudera/security/x509/cms.pem.2017-03-14.public
validate = true
Nginx 配置確認(rèn)
主要是闡述 Nginx 作為 Hue Load Balancer窑眯,其他組件需要做反向代理的配置都可以參考這塊屏积。
/etc/nginx/conf.d/test-cluster.conf
,用于實(shí)現(xiàn) HUE LoadBalancer:
server {
server_name 192.168.1.1;
charset utf-8;
listen 8889 ssl;
ssl_certificate /opt/cloudera/security/x509/cms.pem.2017-03-14.nginx;
ssl_certificate_key /opt/cloudera/security/x509/cms.key.2017-03-14.nginx;
client_max_body_size 0;
location / {
proxy_pass https://hue;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $remote_addr;
}
location /static/ {
alias /opt/cloudera/parcels/CDH/lib/hue/build/static/;
expires 30d;
add_header Cache-Control public;
}
}
upstream hue {
ip_hash;
# List all the Hue instances here for high availability.
server HUE_SERVER_HOSTNAME1:8888 max_fails=3;
server HUE_SERVER_HOSTNAME2:8888 max_fails=3;
...
}
Oozie 配置確認(rèn)
請(qǐng)注意替換 PASSWD
:
Enable TLS/SSL for Oozie = true
Oozie TLS/SSL Server JKS Keystore File Location = /opt/cloudera/security/jks/cms.keystore.2017-03-14.hadoopagent
Oozie TLS/SSL Server JKS Keystore File Password = ${PASSWD}
Oozie TLS/SSL Certificate Trust Store File = ${JAVA_HOME}/jre/lib/security/jssecacerts.2017-03-14.public
Oozie TLS/SSL Certificate Trust Store Password = ${PASSWD}
重啟服務(wù)
使用 Cloudera Manager Admin Console 對(duì)服務(wù)進(jìn)行重啟磅甩。
BTW炊林,別忘記重啟 Nginx。
小結(jié)
本文闡述了卷要,如何對(duì)一個(gè)已經(jīng)服役的 Cloudera Manager 管理的集群進(jìn)行自簽名證書(shū)更新渣聚。有任何不明確的地方,可以微信聯(lián)系我却妨,或者直接簡(jiǎn)書(shū)留言饵逐。