并发自定义主键生成策略

并发自定义主键生成策略在项目开发中,我遇到一个需求.就是要生成自定义的主键.主键的格式为:前缀+ 当天日期+ 自增长序列号自增长序列号: 这个序列号是每天从0或1开始自增长的在了解完需要之后,我做了简单的分析之后就马上进入编码模式,这里我使用的是mysql数据,全部代码已经上传到github,有需要的同学自行下载目录结构(项目管理工具用的maven)srcmainjavacom.kco.bean.SequenceNumberBean com.kco.dao.SequenceNumberDao com.kco.Enum.SequenceNumberEnum com.kco.service.SequenceNumberService com.kco.service.SequenceNumberServiceImpl resourcesMETA-INF/mybatis/mapper/SequenceNumberDao.xml META-INF/mybatis/sql-map-config.xml META-INF/spring/spring-base-jdbc.xml log4j.properties testjavacom.kco.TestSequenceNumberService pom.xml编码1, 首先创建一个maven工程,配置一下jar依赖pom.xml<!--?xml version="1.0" encoding="UTF-8"?--><project xmlns="/POM/4.0.0" xmlns:xsi="/2001/XMLSchema-instance"xsi:schemalocation="/POM/4.0.0/xsd/maven-4.0.0.xsd"><modelversion>4.0.0</modelversion><groupid>com.kco</groupid>mytestcode</artifactid><version>1.0-SNAPSHOT</version><dependencies><dependency><groupid>mons</groupid>commons-lang3</artifactid><version>3.3.2</version></dependency><dependency><groupid>mons</groupid>commons-io</artifactid><version>1.3.2</version></dependency><dependency><groupid>mons</groupid>commons-collections4</artifactid><version>4.1</version></dependency><dependency><groupid>mons</groupid>commons-dbcp2</artifactid><version>2.0.1</version></dependency><dependency><groupid>mons</groupid>commons-pool2</artifactid><version>2.3</version></dependency><!-- test --><dependency><groupid>junit</groupid>junit</artifactid><version>4.10</version><scope>test</scope></dependency><dependency><groupid>org.springframework</groupid>spring-test</artifactid><scope>test</scope><version>4.2.3.RELEASE</version></dependency><dependency><groupid>mysql</groupid>mysql-connector-java</artifactid><version>5.1.12</version><scope>runtime</scope></dependency><dependency><groupid>org.mybatis</groupid>mybatis</artifactid><version>3.2.7</version></dependency><!-- spring --><dependency><groupid>org.springframework</groupid>spring-context</artifactid><version>4.2.3.RELEASE</version></dependency><dependency><groupid>org.springframework</groupid>spring-context-support</artifactid><version>4.2.3.RELEASE</version></dependency><dependency><groupid>org.springframework</groupid>spring-tx</artifactid><version>4.2.3.RELEASE</version></dependency><dependency><groupid>org.springframework</groupid>spring-jdbc</artifactid><version>4.2.3.RELEASE</version></dependency><dependency><groupid>org.springframework</groupid>spring-beans</artifactid><version>4.2.3.RELEASE</version></dependency><dependency><groupid>org.springframework</groupid>spring-aop</artifactid><version>4.2.3.RELEASE</version></dependency><!-- aop --><dependency><groupid>org.aspectj</groupid>aspectjrt</artifactid><version>1.8.8</version></dependency><dependency><groupid>org.aspectj</groupid>aspectjtools</artifactid><version>1.8.8</version></dependency><dependency><groupid>org.aspectj</groupid>aspectjweaver</artifactid><version>1.8.8</version><!--spring mybatis 整合--><dependency><groupid>org.mybatis</groupid>mybatis-spring</artifactid><version>1.2.2</version></dependency><dependency><groupid>log4j</groupid><artifactid>log4j</artifactid><version>1.2.17</version></dependency><dependency><groupid>org.slf4j</groupid><artifactid>slf4j-log4j12</artifactid><version>1.7.13</version></dependency><dependency><groupid>org.slf4j</groupid><artifactid>slf4j-api</artifactid><version>1.7.13</version></dependency><dependency><groupid>org.slf4j</groupid><artifactid>jcl-over-slf4j</artifactid><scope>runtime</scope><version>1.7.13</version></dependency></dependencies><build><plugins><plugin><groupid>org.apache.maven.plugins</groupid><artifactid>maven-compiler-plugin</artifactid><version>3.5.1</version><configuration><source>1.8<target>1.8</target><encoding>UTF-8</encoding></configuration><source><source><source><source></plugin><source><source><source><source><source><source><source><source><source><sour ce><groupid>org.apache.maven.plugins</groupid><artifactid>maven-resources-plugin</artifactid><version>2.6</version><configuration><nonfilteredfileextensions><nonfilteredfileextension>tbl</nonfilteredfileextension></nonfilteredfileextensions></configuration></plugin></plugins><source><source><source><source><source><source><source><source><source><sou rce><source><source><source><source><source><source><source><source><source><source></build><source><source><source><source><source><source><source><source><source><sourc e><source><source><source><source><source><source><source><source><source><source><so urce><source><source><source><source><source><source><source><source><source><source> <source><source><source><source></project>2., 初始化数据库<!--?xml version="1.0" encoding="UTF-8"?--><beans xmlns="/schema/beans" xmlns:aop="/schema/aop"xmlns:context="/schema/context"xmlns:tx="/schema/tx"xmlns:xsi="/2001/XMLSchema-instance"xsi:schemalocation="/schema/beans/schema/beans/spring-beans.xsd/schema/context/schema/context/spring-context.xsd/schema/aop/schema/aop/spring-aop.xsd/schema/tx/schema/tx/spring-tx.xsd"><context:annotation-config><context:component-scan base-package="com.kco"><!-- 配置config的数据库--><bean class="mons.dbcp2.BasicDataSource" id="dataSource"> <property name="driverClassName" value="com.mysql.jdbc.Driver"><property name="url" value="jdbc:mysql:///test?useUnicode=yes&characterEncoding=utf8&allowMultiQueries=true"><property name="username" value="root"><property name="password" value="123456"><property name="minIdle" value="10"><property name="maxIdle" value="10"><property name="maxWaitMillis" value="-1"><property name="maxTotal" value="10"></property></property></property></property></property></property></property></property> </bean><!-- spring和MyBatis完美整合--><bean class="org.mybatis.spring.SqlSessionFactoryBean" id="sqlSessionFactory"> <property name="dataSource" ref="dataSource"><property name="configLocation" value="classpath:META-INF/mybatis/sql-map-config.xml"><property name="mapperLocations" value="classpath*:META-INF/mybatis/mapper/*Dao.xml"></property></property></property></bean><bean class="org.mybatis.spring.mapper.MapperScannerConfigurer" id="mapperScannerConfigurer"><property name="annotationClass" value="org.springframework.stereotype.Repository"><property name="basePackage" value="com.kco.dao"></property></property></bean><bean class="org.springframework.jdbc.datasource.DataSourceTransactionManager" id="transactionManager"><property name="dataSource" ref="dataSource"></property></bean><tx:advice id="txAdvice"><tx:attributes><tx:method name="*" propagation="REQUIRED" rollback-for="ng.Throwable"></tx:method></tx:attributes></tx:advice></aop:advisor></aop:config></context:component-scan></context:annotation-config></beans>4, 配置mybatis sql-map-config.xml?<!--?xml version="1.0" encoding="UTF-8" ?--><configuration><properties><property name="dialect" value="mysql"><property name="pageSqlId" value=".*Page$"></property></property></properties><settings><setting name="cacheEnabled" value="false"><setting name="lazyLoadingEnabled" value="false"></setting></setting></settings></configuration>5, 因为要生成自定义主键一般在开发阶段就已经能确定,所以我这里使用了一个枚举类来管理所有的自定义主键com.kco.Enum.SequenceNumberEnumpackage com.kco.Enum;import com.kco.bean.SequenceNumberBean;/*** com.kco.Enum* Created by swlv on 2016/10/25.*/public enum SequenceNumberEnum {GD(new SequenceNumberBean("GD","工单主键生成策略", 1, 1, 8));private SequenceNumberBean sequenceNumberBean;SequenceNumberEnum(SequenceNumberBean sequenceNumberBean){this.sequenceNumberBean = sequenceNumberBean;}public SequenceNumberBean getSequenceNumberBean() {return sequenceNumberBean;}}6, 数据库PUB_SEQUENCE_NUMBER对应的bean类com.kco.bean.SequenceNumberBean?package com.kco.bean;/*** 主键生成策略的bean* com.kco.bean* Created by swlv on 2016/10/25.*/public class SequenceNumberBean {private String prefix;private String name;private String today;private int minNum;private int currentNum;private int numLength;public String getPrefix() {return prefix;}public void setPrefix(String prefix) {this.prefix = prefix;}public String getName() {return name;}public void setName(String name) { = name;}public String getToday() {return today;}public void setToday(String today) {this.today = today;}public int getMinNum() {return minNum;}public void setMinNum(int minNum) { this.minNum = minNum;}public int getCurrentNum() {return currentNum;}public void setCurrentNum(int currentNum) {this.currentNum = currentNum;}public int getNumLength() {return numLength;}public void setNumLength(int numLength) {this.numLength = numLength;}public SequenceNumberBean() {}public SequenceNumberBean(String prefix, String name, int minNum, int currentNum, int numLength) {this.prefix = prefix; = name;this.minNum = minNum;this.currentNum = currentNum;this.numLength = numLength;}@Overridepublic String toString() {return "SequenceNumberBean{" +"prefix='" + prefix + '\'' +", name='" + name + '\'' +", today='" + today + '\'' +", minNum=" + minNum +", currentNum=" + currentNum +", numLength=" + numLength +'}';}}7, 服务层的接口类以及实现com.kco.service.SequenceNumberServicepackage com.kco.service;import com.kco.Enum.SequenceNumberEnum;/*** com.kco.service* Created by swlv on 2016/10/25.*/public interface SequenceNumberService {/*** 生成一个主键* @param sequenceNumberEnum 主键生成类型* @return 返回一个生成的主键*/String newSequenceNumber(SequenceNumberEnum sequenceNumberEnum);}com.kco.service.SequenceNumberServiceImplpackage com.kco.service;import com.kco.Enum.SequenceNumberEnum;import com.kco.bean.SequenceNumberBean;import com.kco.dao.SequenceNumberDao;import org.springframework.stereotype.Service;import javax.annotation.Resource;/*** com.kco.service* Created by swlv on 2016/10/25.*/@Servicepublic class SequenceNumberServiceImpl implements SequenceNumberService{ @Resourceprivate SequenceNumberDao sequenceNumberDao;@Overridepublic synchronized String newSequenceNumber(SequenceNumberEnum sequenceNumberEnum) {if (sequenceNumberEnum == null){return null;}try {SequenceNumberBean sequenceNumberBean = sequenceNumberDao.newSequenceNumber(sequenceNumberEnum.getSequenceNumberBean() .getPrefix());Thread.sleep(100);if (sequenceNumberBean == null){sequenceNumberBean = sequenceNumberEnum.getSequenceNumberBean();sequenceNumberBean.setToday(sequenceNumberDao.getToday());}Thread.sleep(100);sequenceNumberDao.updateSequenceNumber(sequenceNumberBean);Thread.sleep(100);return String.format("%s%6s%08d",sequenceNumberBean.getPrefix(),sequenceNumberBean.getToday(), CurrentNum());} catch (InterruptedException e) {e.printStackTrace();return "";}}}8, 数据库dao层接口com.kco.dao.SequenceNumberDao?package com.kco.dao;import com.kco.bean.SequenceNumberBean;import org.apache.ibatis.annotations.Param;import org.springframework.stereotype.Repository;/*** com.kco.dao* Created by swlv on 2016/10/25.*/@Repositorypublic interface SequenceNumberDao {/*** 根据前缀生成一个序列号信息* @param prefix 前缀* @return 新的序列号信息*/SequenceNumberBean newSequenceNumber(@Param("prefix") String prefix);/*** 将生成的序列号信息更新到数据库中* @param sequenceNumberBean 需要更新信息*/void updateSequenceNumber(@Param("bean") SequenceNumberBean sequenceNumberBean);/*** 获取数据库当天日期* @return*/String getToday();}9., dao层mybatis的实现SequenceNumberDao.xml<!--?xml version="1.0" encoding="UTF-8"?--><mapper namespace="com.kco.dao.SequenceNumberDao"><resultmap id="sequenceNumberResultMap" type="com.kco.bean.SequenceNumberBean"> <id column="prefix" property="prefix"><result column="name" property="name"><result column="today" property="today"><result column="minNum" property="minNum"><result column="currentNum" property="currentNum"><result column="numLength" property="numLength"></result></result></result></result></result></id></resultmap><select id="newSequenceNumber" resultmap="sequenceNumberResultMap"> SELECT prefix,name,today,minNum,currentNum + 1 as currentNum,numLengthFROM PUB_SEQUENCE_NUMBERWHERE prefix = #{prefix}AND today = DATE_FORMAT(CURRENT_DATE(),'%Y%m%d')</select><update id="updateSequenceNumber" parametertype="com.kco.bean.SequenceNumberBean">REPLACE INTO PUB_SEQUENCE_NUMBER(prefix,name,today,minNum,currentNum,numLength)VALUES(#{bean.prefix},#{},DATE_FORMAT(CURRENT_DATE(),'%Y%m%d'),#{bean.minNum},#{bean.currentNum},#{bean.numLength})</update><select id="getToday" resulttype="string">SELECT DATE_FORMAT(CURRENT_DATE(),'%Y%m%d')</select></mapper>10, 编写测试代码(模拟并发测试) com.kco.TestSequenceNumberServicepackage com.kco;import com.kco.Enum.SequenceNumberEnum;import com.kco.service.SequenceNumberService;import mons.collections4.CollectionUtils;import org.springframework.context.ApplicationContext;import org.springframework.context.support.ClassPathXmlApplicationContext;import java.util.ArrayList;import java.util.Collections;import java.util.List;import java.util.Queue;import java.util.concurrent.ArrayBlockingQueue;/*** com.cmbchina.base.service* Created by swlv on 2016/10/25.*/public class TestSequenceNumberService {public static void main(String[] args) throws InterruptedException {ApplicationContext applicationContext = new ClassPathXmlApplicationContext("META-INF/spring/spring-base-jdbc.xml");SequenceNumberService sequenceNumberService = applicationContext.getBean(SequenceNumberService.class);Queue<string> queue = new ArrayBlockingQueue<string>(200);List<thread> list = new ArrayList<>();for (int i = 0;i < 200; i ++){list.add(new Thread(()->{String key = sequenceNumberService.newSequenceNumber(SequenceNumberEnum.GD);queue.add(key);}));}for (Thread thread : list){thread.start();}while (queue.size() != 200);System.out.println(queue);}}</thread></string></string>11, 至此整个项目就已经搭建完成细节解析在com.kco.service.SequenceNumberServiceImpl#newSequenceNumber的实现中在更新当天日期字符串时,使用的是sequenceNumberBean.setToday(sequenceNumberDao.getToday());这是因为服务器的时间跟数据库的时间有可能不一致,这里以数据库的时间为准测试1, 查询数据库的数据SELECT count(1) FROM PUB_SEQUENCE_NUMBER结果:count(1)2, 清空数据后,再运行com.kco.TestSequenceNumberService#main3, 然后查一下数据库SELECT * FROM PUB_SEQUENCE_NUMBERprefix NAME today minNum currentNum numLengthGD 工单主键生成策略20161026 1 176 8问题出现了,明明我是启动了200个线程在跑,怎么才生成176个主键,检查代码也没有问题啊,newSequenceNumber也加了关键字synchronized.先把这个问题放一下,来回忆一下synchronized的用法,synchronized是在线程池在访问共同变量时使其加锁,达到互斥的效果共同变量?共同变量?咦,好像有什么不对劲的地方.newSequenceNumber使用的都是局部变量,没有共同变量啊!不对?是有共同变量,那个共同变量就是数据库的记录除此之外使用synchronized还有一个问题,这个项目是web项目的一部分,一般在部署web项目都是有几个单边,即在不同的PC部署几个一模一样的代码,再通过集群访问的方式随机访问其中一台.那么这几个应用其实就是多进程程序,都不再一个JVM中执行,加synchronized来保证互斥,是不够的.4, 既然知道是数据库同步出现了问题,那么怎么保证数据库同步呢?上网搜索一下,发现数据库有悲观锁和乐观锁解决方法就是在Dao层的实现<select id="newSequenceNumber" resultmap="sequenceNumberResultMap">SELECT prefix,name,today,minNum,currentNum + 1 as currentNum,numLengthFROM PUB_SEQUENCE_NUMBERWHERE prefix = #{prefix}AND today = DATE_FORMAT(CURRENT_DATE(),'%Y%m%d')</select>改为?<select id="newSequenceNumber" resultmap="sequenceNumberResultMap">SELECT prefix,name,today,minNum,currentNum + 1 as currentNum,numLengthFROM PUB_SEQUENCE_NUMBERWHERE prefix = #{prefix}AND today = DATE_FORMAT(CURRENT_DATE(),'%Y%m%d') for UPDATE </select>5, 然后再重新查一下数据库SELECT * FROM PUB_SEQUENCE_NUMBERprefix NAME today minNum currentNum numLengthGD 工单主键生成策略20161026 1 200 8发现问题解决了,这里是用到了数据库锁表的机制来实现同步.以后有机会再学一下数据库同步,悲观锁和乐观锁.因为这个我还不太熟悉,这里就不卖弄了.6., 修改一下com.kco.Enum.SequenceNumberEnum 再增加一个自定义主键GDD(new SequenceNumberBean("GDD","工单主键生成策略", 1, 1, 8))7., 清空数据,然后运行com.kco.TestSequenceNumberService#main8, 修改com.kco.TestSequenceNumberService2#main将String key = sequenceNumberService.newSequenceNumber(SequenceNumberEnum.GD);改为String key = sequenceNumberService.newSequenceNumber(SequenceNumberEnum.GDD);再次运行com.kco.TestSequenceNumberService#main, 这是用来模拟两个程序同时访问数据的情况9, 查询数据SELECT * FROM PUB_SEQUENCE_NUMBERprefix NAME today minNum currentNum numLengthGD 工单主键生成策略20161026 1 200 8GDD 工单主键生成策略20161026 1 200 8数据正常10., 清空数据,连续运行两次com.kco.TestSequenceNumberService2#main模拟多进程对生成同一个自定义主键11, 查询结果SELECT * FROM PUB_SEQUENCE_NUMBERprefix NAME today minNum currentNum numLengthGDD 工单主键生成策略20161026 1 400 8至此, 全部搞定遗留问题。