Windows配置Hadoop
Hadoop版本:2.9.2
首先,你得安裝idea.

安裝Maven

下載地址:https://maven.apache.org/download.cgi
然后配置

file->settings->build->build tools->maven
需要配置, maven home, user setting file和 local repository

也可以使用idea默認(rèn)的maven創(chuàng)建項(xiàng)目. 我是默認(rèn)創(chuàng)建的項(xiàng)目.

配置依賴

項(xiàng)目創(chuàng)建完畢之后, 在pom.xml進(jìn)行配置
可以查看HDFS對(duì)應(yīng)的版本進(jìn)行下載:https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-hdfs

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.xyxj</groupId>
    <artifactId>hadoop</artifactId>
    <version>1.0-SNAPSHOT</version>


    <!--依賴-->
    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-hdfs</artifactId>
            <version>2.9.2</version>
        </dependency>
    </dependencies>

</project>

代碼

萬事俱備, 開始擼碼.
之前我寫過命令行的方式介紹,不清楚可以看看我前面的介紹

import org.apache.commons.io.FileSystemUtils;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

/**
 * Created by silen on 2018/12/1 1:02
 * Copyright (c) 2018 in FORETREE
 */
public class Demo {


    public static void main(String[] args) {
        try {
            //1.配置
            Configuration conf = new Configuration();
            conf.set("fs.defaultFS", "hdfs://127.0.0.1:9000/");
            conf.set("dfs.client.use.datanode.hostname", "true");
            //2.文件系統(tǒng)
            FileSystem fs = FileSystem.get(conf);
            //3. 創(chuàng)建文件夾
            fs.mkdirs(new Path("/bbb/ccc"));
            //4.上傳文件
            fs.copyFromLocalFile(new Path("D:/word.json"),
                    new Path("/bbb/ccc"));
            fs.rename(new Path("/bbb/ccc/word.json"),
                    new Path("/aaa/1.json"));
            //5.下載文件
            fs.copyToLocalFile(new Path("/aaa/1.json"), new Path("D:/"));


        } catch (Exception e) {
            e.printStackTrace();
        }
    }

}

操作是將D盤的word.json上傳到/bbb/ccc中.在改名移動(dòng)到/aaa中在從/aaa中將改名的1.json的文件進(jìn)行下載到D盤上.

查看

打開:http://localhost:50070/ 下的Utilities下的brow file system

也可以通過命令行查看

C:\Users\silen>hadoop fs -ls /aaa
Found 2 items
-rw-r--r--   3 silen supergroup   27354320 2018-12-01 01:43 /aaa/1.json
-rw-r--r--   1 silen supergroup   27354320 2018-11-29 23:16 /aaa/word.json