由于https://github.com/kontext-tech/winutils還未發(fā)布3.3.6版本芦瘾，因此嘗試源碼編譯。
本文 hadoop-3.3.6 編譯后 winutils 等文件獲取 GitHub 地址
環(huán)境和安裝包準(zhǔn)備款咖，見2
- Git-2.43.0-64-bit.exe
- jdk-8u381-windows-x64.exe
- node-v21.2.0-x64.msi
- apache-ant-1.10.14-bin.zip
- apache-maven-3.9.5-bin.zip
- protoc-25.1-win64.zip
- cmake-3.28.0-rc5-windows-x86_64.zip
- GetGnuWin32-0.6.3.exet
- zlib.tar.gz
- Visual Studio 2010 Professional
- hadoop-3.3.6-src.tar.gz
zlib編譯
方法一：
- VS2010打開
zlib路徑\contrib\vstudio\vc10\zlibvc.sln
zlib的vs sln解決方案文件包含6個(gè)工程吩抓。其中：
(1). zlibvc生成動(dòng)態(tài)鏈接庫(kù)zlibwapi.dll和對(duì)應(yīng)的導(dǎo)入庫(kù)zlibwapi.lib
(2). zlibstat生成靜態(tài)鏈接庫(kù)zlibstat.lib
(3). testzlibdll測(cè)試生成的動(dòng)態(tài)鏈接庫(kù)zlibwapi.dll
(4). testzlib測(cè)試生成的靜態(tài)鏈接庫(kù)zlibstat.lib
-
debug模式下編譯
zlibvc和zlibstat发绢，在zlib路徑\contrib\vstudio\vc10\x64\的ZlibStatDebug和ZlibDllDebug文件夾中獲取編譯生成的文件
zlib1 將這三個(gè)文件存入單獨(dú)的文件夾中氏义，將路徑加入系統(tǒng)環(huán)境變量path
方法二：
VS命令行工具進(jìn)入到zlib根目錄杜顺，執(zhí)行 nmake -f win32/Makefile.msc ,在根目錄下生成：zlib.lib(靜態(tài)庫(kù)) zdll.lib(動(dòng)態(tài)庫(kù)的導(dǎo)入庫(kù)) zlib1.dll(動(dòng)態(tài)庫(kù))
配置文件更改
1. maven阿里云鏡像
修改apache-maven-3.9.5路徑\conf\settings.xml文件鏡像mirrors節(jié)點(diǎn)垢粮，使用阿里云倉(cāng)庫(kù)，添加如下代碼劲够。
<mirror>
<id>aliyunmaven</id>
<mirrorOf>*</mirrorOf>
<name>阿里云公共倉(cāng)庫(kù)</name>
<url>https://maven.aliyun.com/repository/public</url>
</mirror>
不更改會(huì)很慢震桶，親測(cè)！Ｕ饕铩蹲姐！
2. Node版本
- 編輯
hadoop-3.3.6-src\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-catalog\hadoop-yarn-applications-catalog-webapp的pom.xml文件 - find
nodeVersion，將<nodeVersion>${nodejs.version}</nodeVersion>改為自己的node版本<nodeVersion>v21.2.0</nodeVersion> - 如果不更改會(huì)出現(xiàn)錯(cuò)誤如下炒瘸。
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.11.2:yarn (yarn install) on project
hadoop-yarn-applications-catalog-webapp: Failed to run task: 'yarn ' failed.
org.apache.commons.exec.
ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
3. 越過Javadoc檢查
- 在
pom.xml中禁用嚴(yán)格檢查<additionalparam>-Xdoclint:none</additionalparam>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<configuration>
<additionalparam>-Xdoclint:none</additionalparam>
</configuration>
</plugin>
</plugins>
特別是hadoop-3.3.6-src\hadoop-common-project\hadoop-annotations\pom.xml需要更改淤堵，否則會(huì)出現(xiàn)如下錯(cuò)誤。
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:jar (module-javadocs) on project hadoop-annotations: MavenReportException: Error while generating Javadoc:
- mvn命令構(gòu)建時(shí)跳過 Javadoc
mvn 其他參數(shù) -Dmaven.javadoc.skip=true verify
編譯Hadoop
VS命令行工具
cd Hadoop文件夾下執(zhí)行命令
mvn clean package -Pdist,native-win -DskipTests -Dtar -Dmaven.javadoc.skip=true
- 編譯后文件目錄為
hadoop-3.3.6-src\hadoop-dist\target
dist
Error,其他報(bào)錯(cuò)見3
- RelWithDebInfo does not exist
hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\bin\RelWithDebInfo does not exist
錯(cuò)誤是這個(gè)目錄不存在顷扩，則在hadoop-hdfs-project\hadoop-hdfs-native-client\target\native下手動(dòng)創(chuàng)建這個(gè)目錄，并根據(jù)提示在重新執(zhí)行命令時(shí)添加參數(shù)-rf :hadoop-hdfs-native-client慰毅，如下隘截。
mvn package -Pdist,native-win -DskipTests -Dtar -Dmaven.javadoc.skip=true -rf :hadoop-hdfs-native-client
- hadoop-hdfs-native-client
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.x:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec failonerror="true" dir="hadoop路徑\hadoop-hdfs-project\hadoop-hdfs-native-client\target/native" executable="cmake">
修改hadoop-hdfs-project\hadoop-hdfs-native-client\pom.xml文件failonerror的值true為false
- convert-ms-winutils
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (convert-ms-winutils) on project hadoop-common: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (convert-ms-winutils) on project hadoop-common: Command execution failed.`
- 添加環(huán)境變量 C:\Windows\Microsoft.NET\Framework64\v4.0.30319
- 或修改沖突文件名，確定引用
C:\Windows\Microsoft.NET\Framework64\v4.0.30319文件
cvtres
相關(guān)文章：
- windows上編譯hadoop
- Windows10 Hadoop3.3.0源碼編譯
- Hadoop 3.2.1 win10 64位系統(tǒng) vs2015 編譯
- Hadoop源碼編譯打包
- hadoop源碼編譯
- Windows 7 64bit和Visual Studio 2010下安裝及使用Zlib 1.2.5
- VS2010 編譯 Zlib庫(kù)-1.2.11（zip解壓庫(kù)）
- Windows中zlib的安裝與配置
- Unable to build Maven project due to Javadoc error?
- maven項(xiàng)目install時(shí)候一直報(bào)錯(cuò)汹胃，檢查Maven-javadoc-plugin聲明錯(cuò)誤（Java 8 配置Maven-javadoc-plugin）或者是：警告: @throws 沒有說明
- Windows7-64編譯hadoop-3.2.0
- Hadoop on Windows Building/ Installation Error
