avro是一個序列化框架

特點是

• 豐富的數據結構
• 使用快速的壓縮二進制數據格式
• 自包含文件傻盟，模式和數據
• RPC
• 簡單的動態(tài)語言結合功能吩坝，Avro 和動態(tài)語言結合后咨油，讀寫數據文件和使用 RPC 協(xié)議都不需要生成代碼俭令，而代碼生成作為一種可選的優(yōu)化只值得在靜態(tài)類型語言中實現

pom依賴

    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro</artifactId>
        <version>1.9.2</version>
    </dependency>

1. 不生成code示例

1.1 編寫avsc文件User.avsc

{
    "namespace":"com.example.person",
    "type":"record",
    "name":"User",
    "fields":[
        {"name":"name", "type":"string"},
        {"name":"id", "type":"int"},
        {"name":"salary","type":"int"},
        {"name":"age","type":"int"},
        {"name":"address","type":"string"}
    ]
}

1.2 按User.avsc進行序列化漆魔，寫文件

    public static void serializing() throws IOException {
        Schema schema = new Schema.Parser().parse(new File("D:\\2020\\avro-test\\src\\main\\avro\\User.avsc"));
        File file = new File("D:\\avro-test\\src\\main\\avro\\User.avro");
        
        DatumWriter<GenericRecord> datumWriter = new GenericDatumWriter<GenericRecord>(schema);
        DataFileWriter<GenericRecord> dataFileWriter = new DataFileWriter<GenericRecord>(datumWriter);
        dataFileWriter.create(schema, file);

        GenericRecord a = new GenericData.Record(schema);
        a.put("name", "zhangsan");
        a.put("id", 1);
        a.put("salary", 1000);
        a.put("age", 27);
        a.put("address", "shanghai tangzheng");
        dataFileWriter.append(a);

        a = new GenericData.Record(schema);
        a.put("name", "lisi");
        a.put("id", 2);
        a.put("salary", 2000);
        a.put("age", 29);
        a.put("address", "shanghai tangzheng");
        dataFileWriter.append(a);

        dataFileWriter.close();
    }

1.3 按User.avsc讀取序列化的文件

    public static void deserializing() throws IOException {
        Schema schema = new Schema.Parser().parse(new File("D:\\avro-test\\src\\main\\avro\\User.avsc"));
        File file = new File("D:\\avro-test\\src\\main\\avro\\User.avro");

        DatumReader<GenericRecord> datumReader = new GenericDatumReader<GenericRecord>(schema);
        DataFileReader<GenericRecord> dataFileReader = new DataFileReader<GenericRecord>(file, datumReader);
        GenericRecord user = null;
        while(dataFileReader.hasNext()) {
            user = dataFileReader.next();
            System.out.println(user);
        }
        dataFileReader.close();
    }

輸出如下

{"name": "zhangsan", "id": 1, "salary": 1000, "age": 27, "address": "shanghai tangzheng"}
{"name": "lisi", "id": 2, "salary": 2000, "age": 29, "address": "shanghai tangzheng"}

2. 生成code示例

2.1 編寫avsc文件User.avsc

{
    "namespace":"com.example.person",
    "type":"record",
    "name":"User",
    "fields":[
        {"name":"name", "type":"string"},
        {"name":"id", "type":"int"},
        {"name":"salary","type":"int"},
        {"name":"age","type":"int"},
        {"name":"address","type":"string"}
    ]
}

2.2 調用avro-tools生成java代碼

java -jar avro-tools-1.9.2.jar compile schema User.avsc ouput

生成的java類 User位于output/com/example/person目錄下那先。

2.3 序列化文件

    public static void testSerial() throws IOException {
        DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
        DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
        dataFileWriter.create(User.SCHEMA$, new File("D:\\2020\\src\\main\\avro\\User.avro));

        User a = new User();
        a.setName("zhangsan");
        a.setId(1);
        a.setSalary(1000);
        a.setAge(28);
        a.setAddress("shanghai pudong");
        dataFileWriter.append(a);

        a = new User();
        a.setName("lisi");
        a.setId(2);
        a.setSalary(2000);
        a.setAge(32);
        a.setAddress("shanghai pudong");
        dataFileWriter.append(a);

        a = new User();
        a.setName("mawu");
        a.setId(3);
        a.setSalary(3000);
        a.setAge(38);
        a.setAddress("shanghai pudong");
        dataFileWriter.append(a);

        dataFileWriter.close();
        System.out.println("ok...");
    }

2.4 反序列化文件

public static void testDeserial() throws IOException {
        DatumReader<User> reader = new SpecificDatumReader<User>(User.class);
        DataFileReader<User> dataFileReader = new DataFileReader<User>(new File(path), reader);
        User user = null;
        while(dataFileReader.hasNext()) {
            user = dataFileReader.next();
            System.out.println(user);
        }
    }

輸出結果如下

{"name": "zhangsan", "id": 1, "salary": 1000, "age": 28, "address": "shanghai pudong"}
{"name": "lisi", "id": 2, "salary": 2000, "age": 32, "address": "shanghai pudong"}
{"name": "mawu", "id": 3, "salary": 3000, "age": 38, "address": "shanghai pudong"}