Ambari部署
Ambari簡介：
Apache Ambari是一種基于Web的工具宵蕉，支持Apache Hadoop集群的供應(yīng)、管理和監(jiān)控楞黄。Ambari目前已支持大多數(shù)Hadoop組件趾诗，包括HDFS、MapReduce呻待、Hive、Pig队腐、 Hbase蚕捉、Zookeper、Sqoop和Hcatalog等柴淘。
Apache Ambari 支持HDFS迫淹、MapReduce、Hive为严、Pig敛熬、Hbase、Zookeper第股、Sqoop和Hcatalog等的集中管理应民。也是5個頂級hadoop管理工具之一。
Ambari界面預(yù)覽
部署環(huán)境：CentOS 6/7
部署步驟：
一夕吻、環(huán)境準(zhǔn)備
1.關(guān)閉防火墻
CentOS 6 :
chkconfig iptables off
service iptables stop
chkconfig --list iptables
CentOS 7：
systemctl disable firewalld.service
systemctl stop firewalld.service
2.關(guān)閉selinux
setenforce 0 (臨時關(guān)閉)
vi /etc/selinux/config
SELINUX=disabled (永久關(guān)閉诲锹，重啟后生效)
3.修改主機(jī)名
CentOS 6:
vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=namenode-1
Centos 7:
hostnamectl set-hostname namenode-1
4.配置本地域名解析（以下為示例）
192.168.5.1 namenode-1
192.168.5.2 datanode-1
192.168.5.3 web1
192.168.5.4 web2
192.168.5.5 datanode-2
192.168.5.6 datanode-3
192.168.5.7 datanode-4
192.168.5.8 ambari
5.配置SSH（Ambari節(jié)點(diǎn)可無密碼訪問其他主機(jī)）
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
ssh-copy-id -i /root/.ssh/id_rsa.pub root@namenode-1
... ...
6.修改YUM源
這里使用阿里云鏡像已保證安裝速度
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bk
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
yum makecache
7.安裝JDK
HADOOP環(huán)境需要的JAVA為1.7以上版本的JDK，這里使用1.8版本的JAVA
上傳1.8版本的JDK包
rpm -ivh jdk-8u131-linux-x64.rpm
配置環(huán)境變量
vim /etc/profile （文件末尾添加）
export JAVA_HOME=/usr/java/jdk1.8.0_131
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
驗(yàn)證是否安裝成功
java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
8.安裝時間同步服務(wù)（NTP）涉馅，確保集群內(nèi)部時間統(tǒng)一
CentOS 6:
yum install -y ntp
chkconfig --list ntpd
chkconfig ntpd on
service ntpd start
CentOS 7:
yum install -y ntp
chkconfig --list ntpd
systemctl is-enabled ntpd
systemctl enabled ntpd
systemctl start ntpd
9.關(guān)閉THP服務(wù)（如果不關(guān)閉transparent_hugepage归园，Ambari在檢查環(huán)境是也會進(jìn)行警告，HDFS性能會因?yàn)門HP服務(wù)未關(guān)閉受到嚴(yán)重的影響）
永久關(guān)閉（重啟后生效）：
CentOS 6:
vi /etc/grub.conf
添加 transparent_hugepage=never
vi /etc/rc.local
添加：
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
exit 0
CentOS 7:
vi /etc/default/grub
在有CMDLINE一行添加 transparent_hugepage=never
grub2-mkconfig -o /boot/grub2/grub.cfg
systemctl disable tuned
臨時關(guān)閉（不需要重啟）：
CentOS 6:
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
cat /sys/kernel/mm/redhat_transparent_hugepage/enabled always madvise [never]
CentOS 7:
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
檢查方法：
cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
有 [never]則表示THP被禁用
10.配置UMASK稚矿，設(shè)定用戶創(chuàng)建目錄的初始權(quán)限
umask 0022
11.禁止離線更新
vi /etc/yum/pluginconf.d/refresh-packagekit
添加enbaled=0
12.備份克掠褂铡（略）
以上的環(huán)境配置是最基本的，每個節(jié)點(diǎn)都需要的配置晤揣∏潘可以做一備份節(jié)點(diǎn)，用于將來創(chuàng)建子節(jié)點(diǎn)克隆使用碉渡，減少重復(fù)工作。當(dāng)然克隆后還有小的調(diào)整母剥，比如要重新配置hostname滞诺，還有SSH配置過程里形导，分發(fā)主節(jié)點(diǎn)里配置好的authorized_keys到各從節(jié)點(diǎn)，還是在克隆所有子節(jié)點(diǎn)后再操作习霹。
二朵耕、安裝Ambari集群
1.制作本地源
使用官方默認(rèn)源的話十分緩慢，因此建議使用本地源進(jìn)行安裝（本地源文件較大淋叶、下載較為緩慢阎曹，酌情考慮）
1).配置HTTP服務(wù)（yum實(shí)質(zhì)上是使用http協(xié)議進(jìn)行下載安裝的）
chkconfig httpd on
service httpd start / systemctl start httpd
2).安裝本地源制作相關(guān)工具
yum -y install yum-utils createrepo yum-plugin-priorities
vi /etc/yum/pluginconf.d/priorities.conf
添加gpgcheck=0
3).下載Ambari和HDP（下載時間1M帶寬預(yù)計需要14小時左右，網(wǎng)絡(luò)不穩(wěn)定時容易斷線）
CentOS 6：
http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.0.3/ambari-2.5.0.3-centos6.tar.gz
http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.0.3/HDP-2.6.0.3-centos6-rpm.tar.gz
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6/HDP-UTILS-1.1.0.21-centos6.tar.gz
CentOS 7:
http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3/ambari-2.5.0.3-centos7.tar.gz
http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.0.3/HDP-2.6.0.3-centos7-rpm.tar.gz
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz
注：以下將按照CentOS 6進(jìn)行部署演示煞檩，CentOS 7請依據(jù)實(shí)際情況進(jìn)行改動
4).創(chuàng)建本地源（我在這里將源文件解壓到了/var/www/html目錄中）
tar zxvf /opt/ambari-2.5.0.3-centos6.tar.gz -C /var/www/html
tar zxvf /opt/HDP-2.6.0.3-centos6-rpm.tar.gz -C /var/www/html
tar zxvf /opt/HDP-UTILS-1.1.0.21-centos6.tar.gz -C /var/www/html
cd /var/www/html/
createrepo ./
wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.0.3/ambari.repo -O /etc/yum.repos.d/ambari.repo
修改ambari.repo,配置為本地源
vi /etc/yum.repos.d/ambari.repo
[ambari-2.5.0.3]
name=ambari Version - ambari-2.5.0.3
baseurl=http://192.168.5.8/ambari/centos6
gpgcheck=1
gpgkey=http://192.168.5.8/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
vi /etc/yum.repos.d/HDP.repo
[HDP-2.6]
name=HDP-2.6
baseurl=http://192.168.5.8/HDP/centos6/
path=/
enabled=1
gpgcheck=0
vi /etc/yum.repos.d/HDP-UTILS.repo
[HDP-UTILS-1.1.0.21]
name=HDP-UTILS-1.1.0.21
baseurl=http://192.168.50.38/HDP-UTILS-1.1.0.20/
path=/
enabled=1
gpgcheck=0
驗(yàn)證是否生效（如果不確定是否正確可以嘗試瀏覽器訪問处嫌，例如 http://192.168.5.8/ambari/centos6
yum clean all
yum makecache
yum repolist
2.安裝數(shù)據(jù)庫
Ambari使用的默認(rèn)數(shù)據(jù)庫是PostgreSQL,用于存儲安裝元數(shù)據(jù)，可以使用自己安裝的MySQL作為Ambari元數(shù)據(jù)庫（我是使用了源碼安裝的方法斟湃，不做過多贅述熏迹，這里只做Yum安裝步驟）
yum -y install mysql mysql-server
chkconfig mysqld on
service mysqld start
3.安裝Ambari
yum install -y ambari-server
執(zhí)行yum install ambari-server成功后,針對mysql數(shù)據(jù)庫再做一些工作：
將mysql-connector-java.jar復(fù)制到/usr/share/java目錄下（在下載的文件中有mysql-connector-java.jar，具體位置為hdp-utils/mysql-connector-java-source/mysql-connector-java-5.1.37/mysql-connector-java-5.1.37-bin.jar）
mkdir /usr/share/java
cp /root/hdp-utils/mysql-connector-java-source/mysql-connector-java-5.1.37/mysql-connector-java-5.1.37-bin.jar /usr/share/java/mysql-connector-java.jar
cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar
編輯Ambari.properties
vi /etc/ambari-server/conf/ambari.properties
添加server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar
在mysql中分別創(chuàng)建數(shù)據(jù)庫ambari,hive,oozie和其相應(yīng)用戶凝赛，創(chuàng)建相應(yīng)的表（如果不愿意將hive和oozie放在和ambari相同的數(shù)據(jù)庫里注暗，可以另外建立數(shù)據(jù)庫）
CREATE DATABASE ambari;
use ambari;
grant all on *.* to ambari@'localhost' identified by 'password';
grant all on *.* to ambari@'hostname' identified by 'password';
grant all on *.* to ambari@'%' identified by 'password';
CREATE DATABASE hive;
use hive;
grant all on *.* to hive@'localhost' identified by 'password';
grant all on *.* to hive@'hostname' identified by 'password';
grant all on *.* to hive@'%' identified by 'password';
CREATE DATABASE oozie;
use oozie;
grant all on *.* to oozie@'localhost' identified by 'password';
grant all on *.* to oozie@'hostname' identified by 'password';
grant all on *.* to oozie@'%' identified by 'password';
4.配置Ambari
執(zhí)行ambari-server setup
ambari-server setup
下面是配置執(zhí)行流程，按照提示操作
（1） 提示是否自定義設(shè)置墓猎。輸入：y
Customize user account for ambari-server daemon [y/n] (n)? y
（2）ambari-server 賬號捆昏。
Enter user account for ambari-server daemon (root):
如果直接回車就是默認(rèn)選擇root用戶
如果輸入已經(jīng)創(chuàng)建的用戶就會顯示：
Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
（3）檢查防火墻是否關(guān)閉
Adjusting ambari-server permissions and ownership...
Checking firewall...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)?
直接回車
（4）設(shè)置JDK。輸入：3
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
如果上面選擇3自定義JDK,則需要設(shè)置JAVA_HOME毙沾。輸入：/usr/java/jdk1.8.0_131
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/jdk1.8.0_131
Validating JDK on Ambari Server...done.
Completing setup...
（5）數(shù)據(jù)庫配置骗卜。選擇：y
Configuring database...
Enter advanced database configuration [y/n] (n)? y
（6）選擇數(shù)據(jù)庫類型。輸入：3
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
==============================================================================
Enter choice (3): 3
（7）設(shè)置數(shù)據(jù)庫的具體配置信息搀军，根據(jù)實(shí)際情況輸入膨俐，如果和括號內(nèi)相同，則可以直接回車罩句。如果想重命名焚刺，就輸入。
Hostname (localhost):hdp131.cancer.com
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):
Re-Enter password:
（8）將Ambari數(shù)據(jù)庫腳本導(dǎo)入到數(shù)據(jù)庫
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)?
如果使用自己定義的數(shù)據(jù)庫门烂，必須在啟動Ambari服務(wù)之前導(dǎo)入Ambari的sql腳本乳愉。這一點(diǎn)在安裝Ambari環(huán)節(jié)提到過。
5.啟動Ambari
ambari-server start
成功后瀏覽器訪問
http://AMBARI_IP:8080
默認(rèn)用戶名admin屯远，密碼admin
