切記:使用root用戶(hù)編譯
1. 源碼下載及解壓
? ? Hadoop官網(wǎng):https://hadoop.apache.org/release/2.7.0.html片排,Download src? ? ? ??
下載完成后將hadoop-2.7.0-src.tar.gz 解壓到/opt/目錄盯蝴,
進(jìn)入解壓出來(lái)的目錄
BUILDING.txt文件是編譯hadoop源碼的說(shuō)明文件;打開(kāi)文件抱既,列出了編譯hadoop源碼時(shí)职烧,要求軟件及對(duì)應(yīng)版本要求。
本機(jī)環(huán)境:jdk 和maven已經(jīng)安裝好防泵,跳過(guò)
2. 環(huán)境準(zhǔn)備
? ?安裝findbugs:brew install findbugs,安裝完成查看版本:spotbugs -version
? ? 安裝autoconf:brew install autoconf
? ? 安裝automake:brew install automake
? ? 安裝libtool: brew install libtool , brew link libtool
? ? 安裝Protocol buffers (Google's data interchange format),必須是2.5.0:brew search protobuf蚀之,發(fā)現(xiàn)沒(méi)有這個(gè)版本,那么我們編譯安裝择克,安裝完成查看版本:protoc --version
? ? 下載安裝包:https://github.com/protocolbuffers/protobuf/releases?after=v3.0.0-alpha-1恬总,找到對(duì)應(yīng)的版本? ??
? ? 下載完成后解壓:sudo tar -xzvf protobuf-2.5.0.tar.gz -C /opt/
????進(jìn)入目錄:cd protobuf-2.5.0
? ? 執(zhí)行安裝配置命令:? ??
????????./autogen.sh & ./configure \?
? ? ? ? make \
????????make install
? ? ?最后驗(yàn)證是否安裝成功:protoc --version
安裝cmake: brew install cmake
安裝zlib: brew install zlib
安裝openssl:brew install openssl
安裝snappy(若想讓hadoop支持snappy壓縮,得先安裝snappy):
注意安裝snappy之前肚邢,需要先安裝cmake壹堰,pkg-config,因?yàn)閟nappy是依賴(lài)cmake和pkg-config的,如果已經(jīng)安裝了snappy骡湖,可以重新install贱纠,brew reinstall snappy
????brew install cmake
????brew install pkg-config
安裝snappy成功后,查看:brew info snappy
安裝jansson:brew install jansson
3. 編譯hadoop
? ? 進(jìn)入源碼目錄:cd /opt/hadoop-2.7.0-src/
? ? 執(zhí)行編譯命令:mvn package -Pdist,native -DskipTests -Dtar
????如果中途編譯失敗响蕴,并且不要文檔谆焊、hadoop支持snappy、支持openssl的話(huà)浦夷,請(qǐng)使用這個(gè)命令:
? ??mvn clean package -Pdist,native -DskipTests -Dtar -Drequire.snappy -Dsnappy.lib=/usr/local/Cellar/snappy/1.1.7_1/lib? -Dbundle.snappy -Drequire.openssl
說(shuō)明:支持snappy(若不需要支持snappy辖试,把這些選項(xiàng)去除即可):Drequire.snappy -Dsnappy.lib=/usr/local/Cellar/snappy/1.1.7_1/lib -Dbundle.snappy
????編譯過(guò)程中遇到的問(wèn)題:
? ? ? ? 1)No such file or directory:一般就是權(quán)限的問(wèn)題,切換到root再執(zhí)行看看劈狐」扌ⅲ或者你可以將更改目錄權(quán)限sudo chown -R chriszhao:staff /opt/hadoop-2.7.0-src/* (chriszhao:改成你自己的用戶(hù)名)
? ? ? ? 2)hadoop-pipes: An Ant BuildException has occured: exec returned: 1:???
? ? ? ? ? ? 查看build-main.xml
? ? ? ? 執(zhí)行: cmake?/opt/hadoop-2.7.0-src/hadoop-tools/hadoop-pipes/src/ -DJVM_ARCH_DATA_MODEL=64,發(fā)現(xiàn)如下錯(cuò)誤:
? ? ? ? Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the System variable OPENSSL_ROOT_DIR (missing: OPENSSL_INCLUDE_DIR), suggesting that not found openssl suggested that the need to add environment variables
? ? ? ? 然后修改~/.zshrc, 之后再次執(zhí)行以上cmake 命令,如果沒(méi)錯(cuò)就再?lài)L試編譯
? ??????export LDFLAGS="-L/usr/local/opt/openssl@1.1/lib"
????????export CPPFLAGS="-I/usr/local/opt/openssl@1.1/include"
????????export PKG_CONFIG_PATH="/usr/local/opt/openssl@1.1/lib/pkgconfig"
????????export OPENSSL_INCLUDE_DIR="/usr/local/opt/openssl@1.1/include"
? ? ? ? export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"
????????3)因?yàn)榫W(wǎng)絡(luò)問(wèn)題導(dǎo)致tomcat包下不來(lái)肥缔,多嘗試幾次編譯
4. 編譯完成后
? ? 想要的結(jié)果包在hadoop-dist/target/下