最近在做基于標簽的圈人,使用了開源的Roaringbitmap淹办。Greenplum安裝完Roaringbitmap插件后勇婴,可以構(gòu)建roaringbitmap焚志,不過這些操作都是在數(shù)據(jù)庫內(nèi)完成。如果在程序中構(gòu)建RoaringBitmap胆筒,如何將數(shù)據(jù)寫入GP邮破?
其實很簡單,使用官方自己到JDBC驅(qū)動仆救,將roaringbitmap序列化后抒和,就可以寫入了。
1.首先在gp數(shù)據(jù)庫中建表
create table btable(id int,userids bytea);注意 這里是bytea 二進制類型彤蔽。
2.在程序中序列化roaringbitmap摧莽,將二進制數(shù)據(jù)寫入數(shù)據(jù)庫
public static String url = "jdbc:pivotal:greenplum://127.0.0.1:5432;DatabaseName=gpadmin";
// 數(shù)據(jù)庫用戶名
public static String username = "gpadmin";
// 數(shù)據(jù)庫密碼
public static String password = "gpadmin";
public static Connection conn = null;
static {
try {
try {
conn = DriverManager.getConnection(url, username, password);
} catch (SQLException e) {
e.printStackTrace();
}
Class.forName("com.pivotal.jdbc.GreenplumDriver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
String insql = "INSERT INTO btest(id,bs) VALUES (?,?)";
PreparedStatement addstmt = conn.prepareStatement(insql);
RoaringBitmap mrb = RoaringBitmap.bitmapOf(1, 2, 3, 1000);
System.out.println("starting with bitmap " + mrb);
mrb.runOptimize(); //to improve compression
byte[] array = new byte[mrb.serializedSizeInBytes()];
try {
mrb.serialize(new java.io.DataOutputStream(new java.io.OutputStream() {
int c = 0;
@Override
public void close() {
}
@Override
public void flush() {
}
@Override
public void write(int b) {
array[c++] = (byte) b;
}
@Override
public void write(byte[] b) {
write(b, 0, b.length);
}
@Override
public void write(byte[] b, int off, int l) {
System.arraycopy(b, off, array, c, l);
c += l;
}
}));
} catch (IOException ioe) {
// should never happen because we write to a byte array
throw new RuntimeException("unexpected error while serializing to a byte array");
}
addstmt.setInt(1,1);
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(array);
addstmt.setBinaryStream(2,byteArrayInputStream);
addstmt.executeUpdate();
3.將二進制數(shù)據(jù)轉(zhuǎn)化成roaringbitmap,這也是最為關(guān)鍵的一步。
select cast( cast(userids as varchar) as roaringbitmap) from btale
至此顿痪,已經(jīng)可以將roaringbitmap寫入數(shù)據(jù)庫镊辕。
至于,如何從數(shù)據(jù)中讀取roaringbitmap员魏,暫時還沒有找到方法丑蛤。