下載所有文件
Impala工具下載地址
下載:bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el7.noarch.rpm
安裝Impala的其他依賴(lài)
yum install -y cyrus-sasl-plain lsb
安裝Hadoop般堆、Hive(安裝方式自行百度)
安裝Impala
主節(jié)點(diǎn)安裝
sudo rpm -ivh bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el7.noarch.rpm
sudo rpm -ivh impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm --nodeps
sudo rpm -ivh impala-kudu-catalog-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-server-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-state-store-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-shell-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-udf-devel-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
從節(jié)點(diǎn)安裝
sudo rpm -ivh bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el7.noarch.rpm
sudo rpm -ivh impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm --nodeps
sudo rpm -ivh impala-kudu-server-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-shell-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-udf-devel-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
配置Impala-bigtop-utils
vim /etc/default/bigtop-utils(主從節(jié)點(diǎn))
修改JAVA_HOME
配置Impala(主從節(jié)點(diǎn))
vim /etc/default/impala
修改主節(jié)點(diǎn)域名
修改Hadoop配置文件
core-site.xml、hdfs-site.xml
# core-site.xml
<!--impala configuration -->
<property>
<name>dfs.client.read.shortcircuit</name>
<value>true</value>
</property>
<property>
<name>dfs.client.read.shortcircuit.skip.checksum</name>
<value>false</value>
</property>
<property>
<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
<value>true</value>
</property>
#hdfs-site.xml
<!--impala configuration -->
<property>
<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.block.local-path-access.user</name>
<value>impala</value>
</property>
<property>
<name>dfs.client.file-block-storage-locations.timeout.millis</name>
<value>60000</value>
</property>
<property>
<name>dfs.domain.socket.path</name>
<value>/var/run/hadoop-hdfs/dn._PORT</value> #hadoop-hdfs手動(dòng)創(chuàng)
</property>
配置整合Hdfs诚啃、Hive
配置impala的conf文件(主從節(jié)點(diǎn))
hadoop配置文件:core-site.xml淮摔、hdfs-site.xml
hive配置文件 :hive-site.xml
拷貝到/etc/impala/conf.dist/目錄下
重啟Hadoop集群
設(shè)置Socket path
在目錄/var/run/目錄下新建目錄hadoop-hdfs(主從節(jié)點(diǎn))
注意:該文件夾可能已經(jīng)存在,應(yīng)當(dāng)確認(rèn)用impala是否有權(quán)限進(jìn)行讀寫(xiě)始赎。如果已經(jīng)存在
將用戶(hù)impala加入該文件所屬的組,并修改該文件組的權(quán)限即: chown -R 775 hadoop-hdfs/
[root@hadoop]# cd /var/run/ #進(jìn)入目錄/var/run/
[root@hadoop]# mkdir hadoop-hdfs #創(chuàng)建目錄hadoop-hdfs文件夾
[root@hadoop]# chown -R 775 hadoop-hdfs/ #設(shè)置權(quán)限
權(quán)限配置
如果想要impala和yarn合作,需要把impala用戶(hù)加入hdfs組(主從節(jié)點(diǎn));
impala在執(zhí)行drop table操作時(shí)俄周,需要把文件移到到hdfs的回收站
所以需要?jiǎng)?chuàng)建一個(gè)hdfs的目錄/user/impala排龄,并將其設(shè)置為impala用戶(hù)可寫(xiě)。
同樣凛忿,impala需要讀取hive數(shù)據(jù)倉(cāng)庫(kù)下的數(shù)據(jù)床牧,故需要把impala用戶(hù)加入hive組。
[root@hadoop]# usermod -G hdfs,hadoop impala
[root@hadoop]# groups impala
impala:impala hdfs hadoop
另:創(chuàng)建impala在hdfs上的目錄并設(shè)置權(quán)限
[root@hadoop]$ hadoop fs -mkdir -p /user/impala
[root@hadoop]$ hadoop fs -chown impala /user/impala
添加Mysql驅(qū)動(dòng)
在/var/lib/impala/目錄下面添加mysql驅(qū)動(dòng)jar
啟動(dòng)impala
主節(jié)點(diǎn)
[root@hadoop]# service impala-state-store restart
[root@hadoop]# service impala-catalog restart
[root@hadoop]# service impala-server restart
從節(jié)點(diǎn)
service impala-server restart
服務(wù)驗(yàn)證
注意:
1、因?yàn)閕mpala安裝需要root權(quán)限,所以可以給hadoop用戶(hù)添加sudo權(quán)限
gpasswd -a hadoop wheel #root執(zhí)行