數(shù)據(jù)量10000行以下
-
DataStudio頁面下載(對(duì)應(yīng)工作空間需開啟下載設(shè)置)
import java.util.List;
import com.aliyun.odps.Instance;
import com.aliyun.odps.Odps;
import com.aliyun.odps.OdpsException;
import com.aliyun.odps.account.Account;
import com.aliyun.odps.account.AliyunAccount;
import com.aliyun.odps.data.Record;
import com.aliyun.odps.task.SQLTask;
public class testSql {
private static final String accessId = "";
private static final String accessKey = "";
private static final String endPoint = "http://service.odps.aliyun.com/api";
private static final String project = "";
private static final String sql = "select category from iris;";
public static void
main(String[] args) {
Account account = new AliyunAccount(accessId, accessKey);
Odps odps = new Odps(account);
odps.setEndpoint(endPoint);
odps.setDefaultProject(project);
Instance i;
try {
i = SQLTask.run(odps, sql);
i.waitForSuccess();
List<Record> records = SQLTask.getResult(i);
for(Record r:records){
System.out.println(r.get(0).toString());
}
} catch (OdpsException e) {
e.printStackTrace();
}
}
}
數(shù)據(jù)量10000行以上
- 本地spark讀取odps表，寫本地文件
- 離線數(shù)據(jù)集成到OSS, 然后通過OSS下載到本地
- 使用JDBC方式連接MaxCompute
- 使用批量數(shù)據(jù)通道SDK
<dependency>
<groupId>com.aliyun.odps</groupId>
<artifactId>odps-sdk-core</artifactId>
<version>0.31.3-public</version>
</dependency>
注意點(diǎn)：是用0.24.0版本只支持一下數(shù)據(jù)類型 BIGINT, DOUBLE, BOOLEAN, DATETIME, STRING, DECIMAL, MAP, ARRAY;
報(bào)錯(cuò)如下：
ErrorCode=Local Error, ErrorMessage=Invalid json content.
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.loadFromJson(TableTunnel.java:1483)
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.initiate(TableTunnel.java:1334)
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.<init>(TableTunnel.java:1215)
at com.aliyun.odps.tunnel.TableTunnel.createDownloadSession(TableTunnel.java:371)
at TunnelCheck.main(TunnelCheck.java:36)
Caused by: java.lang.IllegalArgumentException: No enum constant com.aliyun.odps.OdpsType.INT
at java.lang.Enum.valueOf(Enum.java:238)
at com.aliyun.odps.OdpsType.valueOf(OdpsType.java:27)
at com.aliyun.odps.tunnel.TunnelTableSchema.parseColumn(TunnelTableSchema.java:60)
at com.aliyun.odps.tunnel.TunnelTableSchema.<init>(TunnelTableSchema.java:36)
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.loadFromJson(TableTunnel.java:1480)
... 4 more
