本節(jié)主要內容:
impala環(huán)境部署
1.系統(tǒng)環(huán)境:
OS:CentOS Linux release 7.5.1804 (Core)
CPU:2核心
Memory:1GB
運行用戶:root
JDK版本:1.8.0_252
Hadoop版本:cdh5.16.2
2.集群各節(jié)點角色規(guī)劃為:
172.26.37.245 node1.hadoop.com---->namenode啦桌,zookeeper扼菠,journalnode科汗,hadoop-hdfs-zkfc近刘,resourcenode娘荡,historyserver钞护,hbase喉祭,hbase-master养渴,hive雷绢,hive-metastore,hive-server2,hive-hbase理卑,sqoop翘紊,impala,impala-server藐唠,impala-state-store霞溪,impala-catalog
172.26.37.246 node2.hadoop.com---->datanode,zookeeper中捆,journalnode鸯匹,nodemanager,hadoop-client泄伪,mapreduce殴蓬,hbase-regionserver,impala蟋滴,impala-server染厅,hive
172.26.37.247? node3.hadoop.com---->datanode,nodemanager津函,hadoop-client肖粮,mapreduce,hive尔苦,mysql-server涩馆,impala,impala-server允坚,
172.26.37.248? node4.hadoop.com---->namenode魂那,zookeeper,journalnode稠项,hadoop-hdfs-zkfc涯雅,hive,hive-server2展运,impala-shell
3.環(huán)境說明:
本次追加部署
172.26.37.245 node1.hadoop.com---->impala活逆,impala-server,impala-state-store拗胜,impala-catalog
172.26.37.246 node2.hadoop.com---->impala蔗候,impala-server,hive
172.26.37.247? node3.hadoop.com---->impala挤土,impala-server
172.26.37.248? node4.hadoop.com---->impala-shell
一.安裝
node1節(jié)點
? ? ? # yum -y install impala impala-server impala-state-store impala-catalog
node2節(jié)點
? ? ? # yum -y install impala impala-server
node3節(jié)點
? ? ? # yum -y install impala impala-server
node4節(jié)點
? ? ? # yum -y install impala-shell
二.配置
node1節(jié)點
? ? ? # cd /etc/impala/conf
? ? ? # cp /usr/lib/hive/conf/hive-site.xml ./
? ? ? # cp /etc/hadoop/conf/core-site.xml ./
? ? ? # cp /etc/hadoop/conf/hdfs-site.xml ./
? ? ? # cp /usr/lib/hbase/conf/hbase-site.xml ./
node2節(jié)點追加hive配置
? ? ? # cp /usr/lib/hive/conf/hive-site.xml root@node2.hadoop.com:/usr/lib/hive/conf/hive-site.xml
node2節(jié)點
? ? ? # cd /etc/impala/conf
? ? ? # cp /usr/lib/hive/conf/hive-site.xml ./
? ? ? # cp /etc/hadoop/conf/core-site.xml ./
? ? ? # cp /etc/hadoop/conf/hdfs-site.xml ./
? ? ? # cp /usr/lib/hbase/conf/hbase-site.xml ./
node3節(jié)點
? ? ? # cd /etc/impala/conf
? ? ? # cp /usr/lib/hive/conf/hive-site.xml ./
? ? ? # cp /etc/hadoop/conf/core-site.xml ./
? ? ? # cp /etc/hadoop/conf/hdfs-site.xml ./
? ? ? # cp /usr/lib/hbase/conf/hbase-site.xml ./
node1琴庵、node2、node3節(jié)點
? ? ? # cp -p /etc/impala/conf/hdfs-site.xml /etc/impala/conf/hdfs-site.xml.20200705
? ? ? # vi /etc/impala/conf/hdfs-site.xml
? ? ? 增加以下內容
? <property>?
? ? ? <name>dfs.client.read.shortcircuit</name>?
? ? ? <value>true</value>?
? </property>
? <property>?
? ? ? <name>dfs.domain.socket.path</name>?
? ? ? <value>/var/run/hdfs-sockets/dn._PORT</value>?
? </property>
? <property>?
? ? ? <name>dfs.client.file-block-storage-locations.timeout.millis</name>?
? ? ? <value>10000</value>?
? </property>
? ? ? # mkdir /var/run/hdfs-sockets/
? ? ? # chmod -R hdfs:hdfs /var/run/hdfs-sockets/
? ? ? # usermod -a -G hadoop impala
? ? ? # usermod -g -G hdfs impala
? ? ? # vi /etc/impala/conf/hdfs-site.xml
增加以下內容
? <property>?
? ? ? <name>dfs.datanode.hdfs-blocks-metadata.enabled</name>?
? ? ? <value>true</value>?
? </property>
? ? ? # cp -p /etc/hadoop/conf/hdfs-site.xml /etc/hadoop/conf/hdfs-site.xml.20200705
? ? ? # vi /etc/hadoop/conf/hdfs-site.xml
增加以下內容
? <property>?
? ? ? <name>dfs.datanode.hdfs-blocks-metadata.enabled</name>?
? ? ? <value>true</value>?
? </property>
三.啟動服務
Node1節(jié)點
? ? ? # service impala-start-store start
? ? ? # service impala-start-store status
? ? ? # service impala-catalog start
? ? ? # service impala-catalog status
? ? ? # service impala-server start
? ? ? # service impala-server status
Node2節(jié)點
? ? ? # service impala-server start
? ? ? # service impala-server status
Node3節(jié)點
? ? ? # service impala-server start
? ? ? # service impala-server status
四.測試
Node4節(jié)點
? ? ? # vi /etc/default/impala
增加以下內容
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_CATALOG_SERVER=node1.hadoop.com
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala
IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR}"
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_SERVER_ARGS=" \
? ? -log_dir=${IMPALA_LOG_DIR} \
? ? -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
? ? -state_store_port=${IMPALA_STATE_STORE_PORT} \
? ? -use_statestore \
? ? -state_store_host=${IMPALA_STATE_STORE_HOST} \
? ? -be_port=${IMPALA_BACKEND_PORT}"
ENABLE_CORE_DUMPS=false
? ? ? # sudu -u hdfs impala-shell
[not connected] > connect node1.hadoop.com:21000
[node1.hadoop.com:21000] >
[node1.hadoop.com:21000] >