隸屬于文章系列：大數(shù)據(jù)安全實(shí)戰(zhàn) http://www.reibang.com/p/76627fd8399c
- 在參數(shù)中傳入testyarn-nopwd的Kerberos憑證，使用集群上有的用戶提交
bin/spark-submit \
--principal testyarn-nopwd@TT.COM \
--keytab /etc/hadoop/conf/testyarn-nopwd.keytab
--proxy-user hive
--class org.apache.spark.examples.SparkPi
--master yarn --deploy-mode client
./examples/jars/spark-examples_2.11-2.2.0.jar
10
在界面上看到：testyarn-nopwd
image.png
- 使用當(dāng)前終端已有的hdfs的憑證
bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client ./examples/jars/spark-examples_2.11-2.2.0.jar 10
#使用的當(dāng)前已有的憑證：hdfs 在運(yùn)行記錄上看到也是hdfs用戶的運(yùn)行記錄
image.png
- 使用當(dāng)前登錄的hdfs用戶的憑證，代理hive用戶
bin/spark-submit --proxy-user hive --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client ./examples/jars/spark-examples_2.11-2.2.0.jar 10
#在界面上看到：hive
image.png
- 沒憑證時(shí)粘招，代理集群上存在的hive用戶
kdestroy
bin/spark-submit --proxy-user hive --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client ./examples/jars/spark-examples_2.11-2.2.0.jar 10
18/04/24 14:17:52 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://172.20.178.28:4040
18/04/24 14:17:52 INFO spark.SparkContext: Added JAR file:/var/opt/spark-2.2.0-bin-hadoop2.7/./examples/jars/spark-examples_2.11-2.2.0.jar at spark://172.20.178.28:61994/jars/spark-examples_2.11-2.2.0.jar with timestamp 1524550672801
18/04/24 14:17:53 INFO client.RMProxy: Connecting to ResourceManager at v-hadoop-kbds.sz.kingdee.net/172.20.178.28:8032
18/04/24 14:17:53 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
就報(bào)錯(cuò)了,說明使用了代理機(jī)制的時(shí)候拔第，也要有Kerberos憑證。
- 有憑證時(shí)，使用集群上不存在的用戶代理：
bin/spark-submit --proxy-user hellokitty --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client ./examples/jars/spark-examples_2.11-2.2.0.jar 10
## 報(bào)錯(cuò)：
18/04/24 14:51:41 INFO yarn.Client:
client token: N/A
diagnostics: Application application_1524468264642_0007 failed 2 times due to AM Container for appattempt_1524468264642_0007_000002 exited with exitCode: -1000
For more detailed output, check application tracking page:http://v-hadoop-kbds.sz.kingdee.net:8088/cluster/app/application_1524468264642_0007Then, click on links to logs of each attempt.
Diagnostics: Application application_1524468264642_0007 initialization failed (exitCode=255) with output: User hellokitty not found
- 總結(jié)
使用代理機(jī)制提交spark任務(wù)的時(shí)候标沪， 必須有有Kerberos憑證朱沃，proxy-user必須在集群上存在苞轿。
image.png
