在一個(gè)項(xiàng)目中需要Excel批量導(dǎo)入數(shù)據(jù)槽畔,然后就寫(xiě)了一個(gè)測(cè)試程序哲银。
這次測(cè)試的數(shù)據(jù)是九千條數(shù)據(jù)最快7秒左右可以全部入庫(kù)莱没,用的是Mysql數(shù)據(jù)庫(kù)v5.6初肉,Mybatis框架。我試驗(yàn)了下饰躲,不用多線(xiàn)程需要47秒牙咏。
用 Executor 創(chuàng)建線(xiàn)程池
@Configuration
@EnableAsync
public class ExecutorConfig {
@Bean
public Executor asyncServiceExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
//配置核心線(xiàn)程數(shù)
executor.setCorePoolSize(10);
//配置最大線(xiàn)程數(shù)
executor.setMaxPoolSize(10);
//配置隊(duì)列大小
executor.setQueueCapacity(99999);
//配置線(xiàn)程池中的線(xiàn)程的名稱(chēng)前綴
executor.setThreadNamePrefix("async-service-remit-");
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
//執(zhí)行初始化
executor.initialize();
return executor;
}
}
使用多線(xiàn)程插入數(shù)據(jù)庫(kù)
@Async("asyncServiceExecutor")
@Override
public void staffIntoStore(String filePath,Long companyUuid) {
ExcelHandle handle = new ExcelHandle();
try {
File file = new File(filePath);
List<String> list = new ArrayList<>();
list.add("name");
list.add("idcard");
// 從excel表中獲取數(shù)據(jù)
List<Map<String,Object>> data = handle.getListValue("D:\\test.xlsx",list,0,file);
int dataSize = data.size();
// 這個(gè)是記錄插入數(shù)據(jù)的進(jìn)度
staffEnteringProcessCache.clearProgress(companyUuid);
staffEnteringProcessCache.setTotal(companyUuid,dataSize);
// 每次插入50條數(shù)據(jù)
int step = 50;
int totalTasks = (dataSize % step == 0 ? dataSize/step : (dataSize/step + 1));
final CountDownLatch countDownLatch = new CountDownLatch(totalTasks);
long startTime1 = System.currentTimeMillis();
for(int j = 0; j < dataSize; j=j+step){
final int start = j;
final int perCount = (dataSize - start) < step ? (dataSize - start) : step;
asyncServiceExecutor.execute(new Runnable() {
public void run() {
try {
System.out.println("多線(xiàn)程開(kāi)始: start == " + start + " , 多線(xiàn)程個(gè)數(shù)count" + perCount);
int count = staffMapper.insertBatch(data.subList(start,perCount+start));
System.out.println(staffEnteringProcessCache.incre(companyUuid,count));
countDownLatch.countDown();
} catch (Exception e) {
e.printStackTrace();
}
}
});
}
try {
countDownLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("線(xiàn)程池循環(huán)耗時(shí)=======" + (System.currentTimeMillis() - startTime1));
} catch (IOException e) {
e.printStackTrace();
}
}
打印日志.jpg
第一次寫(xiě)文章,寫(xiě)的不好的地方嘹裂,多多見(jiàn)諒妄壶,覺(jué)得有點(diǎn)幫助,請(qǐng)多多點(diǎn)贊寄狼,謝謝 _