Mybatis--逐條插入和批量插入---效率對比

首先，設(shè)置一下mysql的max_allowed_packet大小,當(dāng)數(shù)據(jù)量大的時候款咖，批量插入會報的錯何暮。修改mysql下的my.ini文件
在在[mysqld]段或者mysql的server配置段進行修改。默認(rèn)大小是1M(1048576bit)铐殃。

max_allowed_packet = 100M

保存好之后海洼，重啟mysql.

### Cause: com.mysql.jdbc.PacketTooBigException: Packet for query is too large (4642540 > 1048576). You can change this value on the server by setting the max_allowed_packet' variable.
; ]; Packet for query is too large (4642540 > 1048576). You can change this value on the server by setting the max_allowed_packet' variable.; nested exception is com.mysql.jdbc.PacketTooBigException: Packet for query is too large (4642540 > 1048576). You can change this value on the server by setting the max_allowed_packet' variable.
    at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:110)
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
    at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:73)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:446)
    at com.sun.proxy.$Proxy92.insert(Unknown Source)
    at org.mybatis.spring.SqlSessionTemplate.insert(SqlSessionTemplate.java:278)
    at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:58)
    at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:59)
    at com.sun.proxy.$Proxy105.batchInsertStone(Unknown Source)
    at com.nenu.serviceImpl.StoneAnalysisServiceImpl.readExcelValue(StoneAnalysisServiceImpl.java:585)
    at com.nenu.serviceImpl.StoneAnalysisServiceImpl.excel2sql(StoneAnalysisServiceImpl.java:213)
    at com.nenu.controller.AdminAnalysisController.excel2sql(AdminAnalysisController.java:314)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:877)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.servlet.resource.ResourceUrlEncodingFilter.doFilter(ResourceUrlEncodingFilter.java:60)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1468)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Unknown Source)
Caused by: com.mysql.jdbc.PacketTooBigException: Packet for query is too large (4642540 > 1048576). You can change this value on the server by setting the max_allowed_packet' variable.
    at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3681)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2512)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2486)
    at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1858)
    at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1197)
    at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
    at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:46)
    at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
    at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50)
    at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
    at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:185)
    at sun.reflect.GeneratedMethodAccessor82.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:433)
    ... 65 more

application.properties
批量操作，要加：allowMultiQueries=true

spring.jpa.show-sql=true
server.port=8080
spring.datasource.url=jdbc:mysql://localhost:3308/insertdatabase?useUnicode=true&characterEncoding=utf8&allowMultiQueries=true
spring.datasource.username=root
spring.datasource.password=mysql
spring.datasource.driver-class-name=com.mysql.jdbc.Driver

mybatis.type-aliases-package=com.nenu.domain
mybatis.mapper-locations=classpath:Mapper/*.xml
spring.mvc.static-path-pattern=/static/**

spring.thymeleaf.mode=HTML5
spring.thymeleaf.encoding=UTF-8
spring.thymeleaf.servlet.content-type=text/html
spring.thymeleaf.cache=false

domain實體層

package com.nenu.domain;
import java.io.Serializable;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;

public class User implements Serializable{
    
    private static final long serialVersionUID = 6777870445002691891L;
    @Id
    @GeneratedValue
    private int id;
    private String name;
    public int getId() {
        return id;
    }
    public void setId(int id) {
        this.id = id;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
}

controller控制層

package com.nenu.controller;

import java.util.ArrayList;
import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;

import com.nenu.domain.User;
import com.nenu.service.UserService;

@Controller
public class UserController {
    long startTime =0;
    @Autowired
    private UserService userService;
    @RequestMapping(value="insert",method=RequestMethod.GET)
    public String show() {
        List<User> list = new ArrayList<>();
        for (int i = 0; i < 1000; i++) {
            User user = new User();
            user.setName("name"+i);
            list.add(user);
        }
        
        startTime = System.currentTimeMillis();//記錄當(dāng)前時間(毫秒)
        for (int i = 0; i <list.size(); i++) {
            userService.insertUser(list.get(i));
        }
        System.out.println("逐條插入 "+list.size()+" 條,耗時："+(System.currentTimeMillis()-startTime)/1000);
        userService.clearAllUser();//刪除數(shù)據(jù)
        startTime = System.currentTimeMillis();//記錄當(dāng)前時間(毫秒)
        userService.batchInsertUser(list);
        System.out.println("批量插入 "+list.size()+" 條,耗時："+(System.currentTimeMillis()-startTime)/1000);
        userService.clearAllUser();//刪除數(shù)據(jù)
        list.clear();
        for (int i = 0; i < 10000; i++) {
            User user = new User();
            user.setName("name"+i);
            list.add(user);
        }
        startTime = System.currentTimeMillis();//記錄當(dāng)前時間(毫秒)
        for (int i = 0; i <list.size(); i++) {
            userService.insertUser(list.get(i));
        }
        System.out.println("逐條插入 "+list.size()+" 條,耗時："+(System.currentTimeMillis()-startTime)/1000);
        userService.clearAllUser();//刪除數(shù)據(jù)
        startTime = System.currentTimeMillis();//記錄當(dāng)前時間(毫秒)
        userService.batchInsertUser(list);
        System.out.println("批量插入 "+list.size()+" 條,耗時："+(System.currentTimeMillis()-startTime)/1000);
        userService.clearAllUser();//刪除數(shù)據(jù)
        list.clear();
        for (int i = 0; i < 100000; i++) {
            User user = new User();
            user.setName("name"+i);
            list.add(user);
        }
        startTime = System.currentTimeMillis();//記錄當(dāng)前時間(毫秒)
        for (int i = 0; i <list.size(); i++) {
            userService.insertUser(list.get(i));
        }
        System.out.println("逐條插入 "+list.size()+" 條,耗時："+(System.currentTimeMillis()-startTime)/1000);
        userService.clearAllUser();//刪除數(shù)據(jù)
        startTime = System.currentTimeMillis();//記錄當(dāng)前時間(毫秒)
        userService.batchInsertUser(list);
        System.out.println("批量插入 "+list.size()+" 條,耗時："+(System.currentTimeMillis()-startTime)/1000);
        userService.clearAllUser();//刪除數(shù)據(jù)   
        return "index";
    }
}

dao層

package com.nenu.dao;

import java.util.List;

import com.nenu.domain.User;

public interface UserDao {
    int insertUser(User user);
    int batchInsertUser(List<User> list);
    void clearAllUser();
}

service層

package com.nenu.service;

import java.util.List;

import com.nenu.domain.User;

public interface UserService {
    int insertUser(User user);
    int batchInsertUser(List<User> list);
    void clearAllUser();
}

serviceImpl層

package com.nenu.serviceImpl;

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import com.nenu.dao.UserDao;
import com.nenu.domain.User;
import com.nenu.service.UserService;
@Service
public class UserServiceImpl implements UserService{
    @Autowired
    private UserDao userDao;

    @Override
    public int insertUser(User user) {
        // TODO Auto-generated method stub
        return userDao.insertUser(user);
    }

    @Override
    public int batchInsertUser(List<User> list) {
        // TODO Auto-generated method stub
        return userDao.batchInsertUser(list);
    }

    @Override
    public void clearAllUser() {
        // TODO Auto-generated method stub
        userDao.clearAllUser();
    }
}

userMapper.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace ="com.nenu.dao.UserDao">
    <resultMap type="com.nenu.domain.User" id="UserMap">
        <result column="id" property="id"/>
        <result column="name" property="name"/>
    </resultMap>
    
    <parameterMap type="com.nenu.domain.User" id="User"/>
    
    <!-- 插入一條記錄 -->
    <insert id="insertUser" parameterMap="User" useGeneratedKeys="true" keyProperty="id">
        insert into
        tb_user(id,name)
        values 
        (#{id},#{name})
    </insert>
    <!-- 批量插入記錄 -->
    <insert id="batchInsertUser" parameterType="java.util.List" useGeneratedKeys="true" keyProperty="id">
        insert into
        tb_user(id,name)
        values 
        <foreach collection="list" item="user" separator=",">
        (#{user.id},#{user.name})
        </foreach>
        
    </insert>
        <!-- 刪除表中全部記錄 -->
    <update id="clearAllUser">
        TRUNCATE table tb_user
    </update>
</mapper>

對比.png

可以看出富腊，批量插入比逐條插入速度快的不止千萬倍坏逢。

最后編輯于：2018.12.31 13:02:32

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者

人面猴
序言：七十年代末，一起剝皮案震驚了整個濱河市赘被，隨后出現(xiàn)的幾起案子是整，更是在濱河造成了極大的恐慌，老刑警劉巖民假，帶你破解...
沈念sama閱讀 217,657評論 6贊 505
死咒
序言：濱河連續(xù)發(fā)生了三起死亡事件浮入，死亡現(xiàn)場離奇詭異，居然都是意外死亡羊异，警方通過查閱死者的電腦和手機事秀，發(fā)現(xiàn)死者居然都...
沈念sama閱讀 92,889評論 3贊 394
救了他兩次的神仙讓他今天三更去死
文/潘曉璐我一進店門，熙熙樓的掌柜王于貴愁眉苦臉地迎上來野舶，“玉大人易迹，你說我怎么就攤上這事∑降溃” “怎么了睹欲？”我有些...
開封第一講書人閱讀 164,057評論 0贊 354
道士緝兇錄：失蹤的賣姜人
文/不壞的土叔我叫張陵，是天一觀的道長一屋。經(jīng)常有香客問我窘疮，道長，這世上最難降的妖魔是什么陆淀？我笑而不...
開封第一講書人閱讀 58,509評論 1贊 293
?港島之戀（遺憾婚禮）
正文為了忘掉前任考余，我火速辦了婚禮，結(jié)果婚禮上轧苫，老公的妹妹穿的比我還像新娘楚堤。我一直安慰自己疫蔓，他們只是感情好，可當(dāng)我...
茶點故事閱讀 67,562評論 6贊 392
惡毒庶女頂嫁案：這布局不是一般人想出來的
文/花漫我一把揭開白布身冬。她就那樣靜靜地躺著衅胀，像睡著了一般。火紅的嫁衣襯著肌膚如雪酥筝。梳的紋絲不亂的頭發(fā)上滚躯，一...
開封第一講書人閱讀 51,443評論 1贊 302
城市分裂傳說
那天，我揣著相機與錄音嘿歌，去河邊找鬼掸掏。笑死，一個胖子當(dāng)著我的面吹牛宙帝，可吹牛的內(nèi)容都是我干的丧凤。我是一名探鬼主播，決...
沈念sama閱讀 40,251評論 3贊 418
雙鴛鴦連環(huán)套：你想象不到人心有多黑
文/蒼蘭香墨我猛地睜開眼步脓，長吁一口氣：“原來是場噩夢啊……” “哼愿待！你這毒婦竟也來了？” 一聲冷哼從身側(cè)響起靴患，我...
開封第一講書人閱讀 39,129評論 0贊 276
萬榮殺人案實錄
序言：老撾萬榮一對情侶失蹤仍侥，失蹤者是張志新（化名）和其女友劉穎，沒想到半個月后鸳君，有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體农渊，經(jīng)...
沈念sama閱讀 45,561評論 1贊 314
?護林員之死
正文獨居荒郊野嶺守林人離奇死亡，尸身上長有42處帶血的膿包…… 初始之章·張勛以下內(nèi)容為張勛視角年9月15日...
茶點故事閱讀 37,779評論 3贊 335
?白月光啟示錄
正文我和宋清朗相戀三年相嵌，在試婚紗的時候發(fā)現(xiàn)自己被綠了腿时。大學(xué)時的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片况脆。...
茶點故事閱讀 39,902評論 1贊 348
活死人
序言：一個原本活蹦亂跳的男人離奇死亡饭宾，死狀恐怖，靈堂內(nèi)的尸體忽然破棺而出格了，到底是詐尸還是另有隱情看铆，我是刑警寧澤，帶...
沈念sama閱讀 35,621評論 5贊 345
?日本核電站爆炸內(nèi)幕
正文年R本政府宣布盛末，位于F島的核電站弹惦，受9級特大地震影響，放射性物質(zhì)發(fā)生泄漏悄但。R本人自食惡果不足惜棠隐，卻給世界環(huán)境...
茶點故事閱讀 41,220評論 3贊 328
男人毒藥：我在死后第九天來索命
文/蒙蒙一、第九天我趴在偏房一處隱蔽的房頂上張望檐嚣。院中可真熱鬧助泽，春花似錦、人聲如沸。這莊子的主人今日做“春日...
開封第一講書人閱讀 31,838評論 0贊 22
一樁弒父案隐解，背后竟有這般陰謀
文/蒼蘭香墨我抬頭看了看天上的太陽。三九已至诫睬，卻和暖如春煞茫，著一層夾襖步出監(jiān)牢的瞬間，已是汗流浹背摄凡。一陣腳步聲響...
開封第一講書人閱讀 32,971評論 1贊 269
情欲美人皮
我被黑心中介騙來泰國打工续徽，沒想到剛下飛機就差點兒被人妖公主榨干…… 1. 我叫王不留，地道東北人亲澡。一個月前我還...
沈念sama閱讀 48,025評論 2贊 370
代替公主和親
正文我出身青樓炸宵，卻偏偏與公主長得像，于是被迫代替她去往敵國和親谷扣。傳聞我的和親對象是個殘疾皇子土全，可洞房花燭夜當(dāng)晚...
茶點故事閱讀 44,843評論 2贊 354

Mybatis--逐條插入和批量插入---效率對比

可以看出富腊，批量插入比逐條插入速度快的不止千萬倍坏逢。

推薦閱讀更多精彩內(nèi)容