数据库批量插入这么讲究的么?

Docker

首先,多条数据的插入,可选的方案:

foreach循环插入

拼接sql,一次执行

使用批处理功能插入

搭建测试环境`

sql文件:

drop database IF EXISTS test; CREATE DATABASE test; use test; DROP TABLE IF EXISTS `user`; CREATE TABLE `user` ( `id` int(11) NOT NULL, `name` varchar(255) DEFAULT "", `age` int(11) DEFAULT 0, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

应用的配置文件:

server: port: 8081 spring: #数据库连接配置 datasource: driver-class-name: com.mysql.cj.jdbc.Driver url: jdbc:mysql://127.0.0.1:3306/test?characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true&&serverTimezone=UTC&setUnicode=true&characterEncoding=utf8&&nullCatalogMeansCurrent=true&&autoReconnect=true&&allowMultiQueries=true username: root password: 123456 #mybatis的相关配置 mybatis: #mapper配置文件 mapper-locations: classpath:mapper/*.xml type-aliases-package: com.aphysia.spingbootdemo.model #开启驼峰命名 configuration: map-underscore-to-camel-case: true logging: level: root: error

启动文件,配置了Mapper文件扫描的路径:

import org.mybatis.spring.annotation.MapperScan; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.EnableAutoConfiguration; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication @MapperScan("com.aphysia.springdemo.mapper") public class SpringdemoApplication { public static void main(String[] args) { SpringApplication.run(SpringdemoApplication.class, args); } }

Mapper文件一共准备了几个方法,插入单个对象,删除所有对象,拼接插入多个对象:

import com.aphysia.springdemo.model.User; import org.apache.ibatis.annotations.Param; import java.util.List; public interface UserMapper { int insertUser(User user); int deleteAllUsers(); int insertBatch(@Param("users") List<User>users); }

Mapper.xml文件如下:

<?xml version="1.0" encoding="UTF-8" ?> <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" > <mapper namespace="com.aphysia.springdemo.mapper.UserMapper"> <insert parameterType="com.aphysia.springdemo.model.User"> insert into user(id,age) values(#{id},#{age}) </insert> <delete> delete from user where id>0; </delete> <insert parameterType="java.util.List"> insert into user(id,age) VALUES <foreach collection="users" item="model" index="index" separator=","> (#{model.id}, #{model.age}) </foreach> </insert> </mapper>

测试的时候,每次操作我们都删除掉所有的数据,保证测试的客观,不受之前的数据影响。

不同的测试 1. foreach 插入

先获取列表,然后每一条数据都执行一次数据库操作,插入数据:

@SpringBootTest @MapperScan("com.aphysia.springdemo.mapper") class SpringdemoApplicationTests { @Autowired SqlSessionFactory sqlSessionFactory; @Resource UserMapper userMapper; static int num = 100000; static int id = 1; @Test void insertForEachTest() { List<User> users = getRandomUsers(); long start = System.currentTimeMillis(); for (int i = 0; i < users.size(); i++) { userMapper.insertUser(users.get(i)); } long end = System.currentTimeMillis(); System.out.println("time:" + (end - start)); } } 2. 拼接sql插入

其实就是用以下的方式插入数据:

INSERT INTO `user` (`id`, `age`) VALUES (1, 11), (2, 12), (3, 13), (4, 14), (5, 15); @Test void insertSplicingTest() { List<User> users = getRandomUsers(); long start = System.currentTimeMillis(); userMapper.insertBatch(users); long end = System.currentTimeMillis(); System.out.println("time:" + (end - start)); } 3. 使用Batch批量插入

将MyBatis session 的 executor type 设为 Batch ,使用sqlSessionFactory将执行方式置为批量,自动提交置为false,全部插入之后,再一次性提交:

@Test public void insertBatch(){ SqlSession sqlSession = sqlSessionFactory.openSession(ExecutorType.BATCH, false); UserMapper mapper = sqlSession.getMapper(UserMapper.class); List<User> users = getRandomUsers(); long start = System.currentTimeMillis(); for(int i=0;i<users.size();i++){ mapper.insertUser(users.get(i)); } sqlSession.commit(); sqlSession.close(); long end = System.currentTimeMillis(); System.out.println("time:" + (end - start)); } 4. 批量处理+分批提交

在批处理的基础上,每1000条数据,先提交一下,也就是分批提交。

@Test public void insertBatchForEachTest(){ SqlSession sqlSession = sqlSessionFactory.openSession(ExecutorType.BATCH, false); UserMapper mapper = sqlSession.getMapper(UserMapper.class); List<User> users = getRandomUsers(); long start = System.currentTimeMillis(); for(int i=0;i<users.size();i++){ mapper.insertUser(users.get(i)); if (i % 1000 == 0 || i == num - 1) { sqlSession.commit(); sqlSession.clearCache(); } } sqlSession.close(); long end = System.currentTimeMillis(); System.out.println("time:" + (end - start)); } 初次结果,明显不对?

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/zwxzyj.html