背景
- 隨著同類項(xiàng)目越來(lái)越多廊营,從產(chǎn)品歪泳、架構(gòu)角度考慮,需要把重復(fù)的蛔外、基礎(chǔ)的東西抽離出來(lái)醇王,用以復(fù)用
- 用戶并沒(méi)有達(dá)到指數(shù)級(jí)增長(zhǎng)淋昭,從部署的角度考慮,暫時(shí)不需要做微服務(wù)
- 因此伶氢,目前只做數(shù)據(jù)庫(kù)、工程的拆分瘪吏,在部署層面仍然使用單一服務(wù)形式癣防。
- 在以上形式下,就需要配置多數(shù)據(jù)源掌眠,下面是入坑指南
入坑過(guò)程
項(xiàng)目單數(shù)據(jù)源初始狀態(tài)
項(xiàng)目的基礎(chǔ)包
- spring boot
- spring data jpa
- querydsl
- mysql8.0
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- AuditingEntityListener -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<!-- mysql -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<!--queryDSL-->
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-jpa</artifactId>
<version>${querydsl.version}</version>
</dependency>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-apt</artifactId>
<version>${querydsl.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
初步的想法和說(shuō)明
- 項(xiàng)目module怎么搭根據(jù)你們自己的要求搭建蕾盯,但是請(qǐng)注意Spring習(xí)慣使用packageScan,為了后續(xù)搭建方便蓝丙,最好對(duì)多模塊進(jìn)行分包
- 對(duì)多數(shù)據(jù)源的配置和使用级遭,一條基本原則就是多數(shù)據(jù)源配置盡可能的與業(yè)務(wù)開發(fā)無(wú)關(guān),最好開發(fā)無(wú)感渺尘,需要事務(wù)的時(shí)候和原來(lái)一樣使用@Transactional
- 原先想要拆分的Entity之間通過(guò)@ManyToOne挫鸽、@OneToMany等JPA的標(biāo)準(zhǔn)接口做的關(guān)聯(lián)關(guān)系,我們希望:
- 只做不同數(shù)據(jù)源的JPA的packageScan沧烈,讓對(duì)應(yīng)的JPA自動(dòng)建立到對(duì)應(yīng)的庫(kù)中
- 底層可以支持自動(dòng)識(shí)別并進(jìn)行跨庫(kù)查詢掠兄,當(dāng)我使用由關(guān)聯(lián)關(guān)系的Entity時(shí)
- 如果底層支持第二條,我們?cè)倏紤]事務(wù)和Session
配置多數(shù)據(jù)源
簡(jiǎn)單的數(shù)據(jù)源配置
如何配置多數(shù)據(jù)源
Configure Two DataSources
If you need to configure multiple data sources, you can apply the same tricks that are described in the previous section. You must, however, mark one of the DataSource instances as @Primary, because various auto-configurations down the road expect to be able to get one by type.
If you create your own DataSource, the auto-configuration backs off. In the following example, we provide the exact same feature set as the auto-configuration provides on the primary data source:
@Bean
@Primary
@ConfigurationProperties("app.datasource.first")
public DataSourceProperties firstDataSourceProperties() {
return new DataSourceProperties();
}
@Bean
@Primary
@ConfigurationProperties("app.datasource.first.configuration")
public HikariDataSource firstDataSource() {
return firstDataSourceProperties().initializeDataSourceBuilder().type(HikariDataSource.class).build();
}
// u can build the second datasource as same as shown above
@Bean
@ConfigurationProperties("app.datasource.second")
public BasicDataSource secondDataSource() {
return DataSourceBuilder.create().type(BasicDataSource.class).build();
}
- 當(dāng)你使用多數(shù)據(jù)源配置datasources時(shí),注意設(shè)置@Primary
- 推薦使用DataSourceProperties方式蚂夕,type推薦HikariDataSource迅诬,這個(gè)是Spring單數(shù)據(jù)源的默認(rèn)類型
Use Two EntityManagers
Even if the default EntityManagerFactory works fine, you need to define a new one, otherwise the presence of the second bean of that type switches off the default. You can use the EntityManagerBuilder provided by Spring Boot to help you to create one. Alternatively, you can use the LocalContainerEntityManagerFactoryBean directly from Spring ORM, as shown in the following example:
@Bean
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(EntityManagerFactoryBuilder builder) {
return builder
.dataSource(customerDataSource())
.packages(Customer.class)
.persistenceUnit("customers")
.build();
}
@Bean
public LocalContainerEntityManagerFactoryBean orderEntityManagerFactory(EntityManagerFactoryBuilder builder) {
return builder
.dataSource(orderDataSource())
.packages(Order.class)
.persistenceUnit("orders")
.build();
}
- 一旦你開啟多數(shù)據(jù)源配置,自定義EntityManagerFactory無(wú)法避免
- 你有兩種方式創(chuàng)建它婿牍,EntityManagerBuilder或者直接new LocalContainerEntityManagerFactoryBean
- 注意3薮!5戎G温!I弦!2肌!7鄢@绷怠!DH怼N肮恰!下面的說(shuō)明如果你沒(méi)有看將會(huì)有很多坑燃异,后面會(huì)一點(diǎn)一點(diǎn)補(bǔ)完携狭。大致的意思就是說(shuō)最好用Spring自己的EntityManagerFactoryBuilder去生成LocalContainerEntityManagerFactoryBean,否則后果自負(fù)
When you create a bean for LocalContainerEntityManagerFactoryBean yourself, any customization that was applied during the creation of the auto-configured LocalContainerEntityManagerFactoryBean is lost. For example, in case of Hibernate, any properties under the spring.jpa.hibernate prefix will not be automatically applied to your LocalContainerEntityManagerFactoryBean. If you were relying on these properties for configuring things like the naming strategy or the DDL mode, you will need to explicitly configure that when creating the LocalContainerEntityManagerFactoryBean bean. On the other hand, properties that get applied to the auto-configured EntityManagerFactoryBuilder, which are specified via spring.jpa.properties, will automatically be applied, provided you use the auto-configured EntityManagerFactoryBuilder to build the LocalContainerEntityManagerFactoryBean bean.
- 注:這里的代碼用的bean name和上面db config bean沒(méi)有關(guān)系回俐,只是個(gè)例子逛腿,讀者自行修改以適應(yīng)自己的項(xiàng)目,下同仅颇,本質(zhì)上就是配置兩個(gè)數(shù)據(jù)源鳄逾,最后會(huì)給出完整配置和源代碼github地址,先了解大概即可
源碼分析
EntityManagerFactoryBuilder
- 位置:spring-boot-autoconfigure-2.1.4.RELEASE.jar
- JpaBaseConfiguration
- 注意
@EnableConfigurationProperties(JpaProperties.class)
灵莲,實(shí)例化屬性取決于JpaProperties.class配置的spring.jpa
雕凹,稍后看ymal文件即可 - @ConditionalOnMissingBean表明如果我們自己不實(shí)例化EntityManagerFactoryBuilder,則Spring會(huì)幫我們構(gòu)造一個(gè)
- 注意
package org.springframework.boot.autoconfigure.orm.jpa;
@Configuration
@EnableConfigurationProperties(JpaProperties.class)
@Import(DataSourceInitializedPublisher.Registrar.class)
public abstract class JpaBaseConfiguration implements BeanFactoryAware {
private final DataSource dataSource;
private final JpaProperties properties;
private final JtaTransactionManager jtaTransactionManager;
private final TransactionManagerCustomizers transactionManagerCustomizers;
private ConfigurableListableBeanFactory beanFactory;
protected JpaBaseConfiguration(DataSource dataSource, JpaProperties properties,
ObjectProvider<JtaTransactionManager> jtaTransactionManager,
ObjectProvider<TransactionManagerCustomizers> transactionManagerCustomizers) {
this.dataSource = dataSource;
this.properties = properties;
this.jtaTransactionManager = jtaTransactionManager.getIfAvailable();
this.transactionManagerCustomizers = transactionManagerCustomizers
.getIfAvailable();
}
@Bean
@ConditionalOnMissingBean
public PlatformTransactionManager transactionManager() {
JpaTransactionManager transactionManager = new JpaTransactionManager();
if (this.transactionManagerCustomizers != null) {
this.transactionManagerCustomizers.customize(transactionManager);
}
return transactionManager;
}
@Bean
@ConditionalOnMissingBean
public JpaVendorAdapter jpaVendorAdapter() {
AbstractJpaVendorAdapter adapter = createJpaVendorAdapter();
adapter.setShowSql(this.properties.isShowSql());
adapter.setDatabase(this.properties.determineDatabase(this.dataSource));
adapter.setDatabasePlatform(this.properties.getDatabasePlatform());
adapter.setGenerateDdl(this.properties.isGenerateDdl());
return adapter;
}
@Bean
@ConditionalOnMissingBean
public EntityManagerFactoryBuilder entityManagerFactoryBuilder(
JpaVendorAdapter jpaVendorAdapter,
ObjectProvider<PersistenceUnitManager> persistenceUnitManager,
ObjectProvider<EntityManagerFactoryBuilderCustomizer> customizers) {
EntityManagerFactoryBuilder builder = new EntityManagerFactoryBuilder(
jpaVendorAdapter, this.properties.getProperties(),
persistenceUnitManager.getIfAvailable());
customizers.orderedStream()
.forEach((customizer) -> customizer.customize(builder));
return builder;
}
@Bean
@Primary
@ConditionalOnMissingBean({ LocalContainerEntityManagerFactoryBean.class,
EntityManagerFactory.class })
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
EntityManagerFactoryBuilder factoryBuilder) {
Map<String, Object> vendorProperties = getVendorProperties();
customizeVendorProperties(vendorProperties);
return factoryBuilder.dataSource(this.dataSource).packages(getPackagesToScan())
.properties(vendorProperties).mappingResources(getMappingResources())
.jta(isJta()).build();
}
}
HibernateJpaConfiguration
- JpaBaseConfiguration的默認(rèn)實(shí)現(xiàn)
- 注意
@ConditionalOnSingleCandidate(DataSource.class)
政冻,因?yàn)槲覀冇卸鄠€(gè)DataSource
實(shí)例枚抵,所以它會(huì)失效,也就是說(shuō)hibernate的默認(rèn)配置無(wú)法生效 -
@EnableConfigurationProperties(HibernateProperties.class)
明场,表明spring.jpa.hibernate
也無(wú)效
@Configuration
@EnableConfigurationProperties(HibernateProperties.class)
@ConditionalOnSingleCandidate(DataSource.class)
class HibernateJpaConfiguration extends JpaBaseConfiguration {...}
自動(dòng)裝配
- org.springframework.boot.autoconfigure.orm.jpa.JpaRepositoriesAutoConfiguration
- org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration
- org.springframework.boot.autoconfigure.data.jpa.JpaRepositoriesAutoConfigureRegistrar
PlatformTransactionManager
The configuration above almost works on its own. To complete the picture, you need to configure TransactionManagers for the two EntityManagers as well. If you mark one of them as @Primary, it could be picked up by the default JpaTransactionManager in Spring Boot. The other would have to be explicitly injected into a new instance. Alternatively, you might be able to use a JTA transaction manager that spans both.
@Bean(name = Constant.TRANSACTION_MANAGER_PROJECT)
@Primary
public JpaTransactionManager transactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactoryBean().getObject());
return jpaTransactionManager;
}
@Bean(name = Constant.TRANSACTION_MANAGER_EVENT)
public JpaTransactionManager transactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactoryBean().getObject());
return jpaTransactionManager;
}
- spring沒(méi)有給demo汽摹,這是我自己加的,差不多就是這個(gè)樣子
EntityManager & JPAQueryFactory
@Bean(name = Constant.ENTITY_MANAGER_PROJECT)
@Primary
public EntityManager entityManager() {
return entityManagerFactoryBean().getObject().createEntityManager();
}
@Bean(name = Constant.JPA_QUERY_FACTORY_PROJECT)
@Primary
public JPAQueryFactory jpaQueryFactory() {
return new JPAQueryFactory(entityManager());
}
@Bean(name = Constant.ENTITY_MANAGER_EVENT)
public EntityManager entityManager() {
return entityManagerFactoryBean().getObject().createEntityManager();
}
@Bean(name = Constant.JPA_QUERY_FACTORY_EVENT)
public JPAQueryFactory jpaQueryFactory() {
return new JPAQueryFactory(entityManager());
}
- 因?yàn)槲覀冺?xiàng)目用到了querydsl苦锨,所以需要JPAQueryFactory
@EnableJpaRepositories
If you use Spring Data, you need to configure @EnableJpaRepositories accordingly, as shown in the following example
@Configuration
@EnableJpaRepositories(
basePackageClasses = {ProjectRepository.class},
entityManagerFactoryRef = Constant.ENTITY_MANAGER_FACTORY_PROJECT,
transactionManagerRef = Constant.TRANSACTION_MANAGER_PROJECT)
public class JpaConfigurationProject {
...
}
@Configuration
@EnableJpaRepositories(
basePackageClasses = {EventRepository.class},
entityManagerFactoryRef = Constant.ENTITY_MANAGER_FACTORY_EVENT,
transactionManagerRef = Constant.TRANSACTION_MANAGER_EVENT)
public class JpaConfigurationEvent {
...
}
- 你也可以使用basePackages配置掃描
Initialize a Database Using JPA
JPA has features for DDL generation, and these can be set up to run on startup against the database. This is controlled through two external properties:
- spring.jpa.generate-ddl (boolean) switches the feature on and off and is vendor independent.
- spring.jpa.hibernate.ddl-auto (enum) is a Hibernate feature that controls the behavior in a more fine-grained way. This feature is described in more detail later in this guide.
兩種方式都可以逼泣,第一種更通用趴泌,第二種具體到hibernate,粒度更細(xì)
跨服務(wù)拉庶、數(shù)據(jù)庫(kù)應(yīng)該通過(guò)服務(wù)接口實(shí)現(xiàn)嗜憔,而不是DB
- 配置完成后,啟動(dòng)服務(wù)報(bào)錯(cuò):@OneToOne or @ManyToOne on xxx.xxx.currency references an unknown entity
- 這個(gè)是stackoverflow的討論
- https://stackoverflow.com/questions/21133131/error-using-onetoone-manytoone-with-jpa-and-multiple-data-sources
- 核心的意思就是:No. You cannot have entity x in database x and entity y in database y and then expect entity x and y to work together. They work separately but you cannot have relations between entity x and entity y
- 很顯然氏仗,跨數(shù)據(jù)源的操作吉捶,需要通過(guò)服務(wù)接口實(shí)現(xiàn)是正統(tǒng)
移除@OneToOne、@ManyToOne
- 實(shí)體關(guān)系依賴改為ID依賴
默認(rèn)hibernate配置失效
See HibernateJpaAutoConfiguration and JpaBaseConfiguration for more details.
方式1(Spring推薦)
- Hibernate uses two different naming strategies to map names from the object model to the corresponding database names. The fully qualified class name of the physical and the implicit strategy implementations can be configured by setting the spring.jpa.hibernate.naming.physical-strategy and spring.jpa.hibernate.naming.implicit-strategy properties, respectively. Alternatively, if ImplicitNamingStrategy or PhysicalNamingStrategy beans are available in the application context, Hibernate will be automatically configured to use them.
- By default, Spring Boot configures the physical naming strategy with SpringPhysicalNamingStrategy. This implementation provides the same table structure as Hibernate 4: all dots are replaced by underscores and camel casing is replaced by underscores as well. Additionally, by default, all table names are generated in lower case. For example, a TelephoneNumber entity is mapped to the telephone_number table. If your schema requires mixed-case identifiers, define a custom SpringPhysicalNamingStrategy bean, as shown in the following example:
@Bean
SpringPhysicalNamingStrategy caseSensitivePhysicalNamingStrategy() {
return new SpringPhysicalNamingStrategy() {
@Override
protected boolean isCaseInsensitive(JdbcEnvironment jdbcEnvironment) {
return false;
}
};
}
@Bean
public PhysicalNamingStrategy physicalNamingStrategy() {
return new PhysicalNamingStrategyStandardImpl();
}
方式2(Spring推薦)
spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
方式3(自定義HibernateConfiguration)
- 上面源碼時(shí)也說(shuō)過(guò)皆尔,HibernateJpaConfiguration只對(duì)單數(shù)據(jù)源生效呐舔,我們直接模仿它寫一個(gè)實(shí)現(xiàn)類給LocalContainerEntityManagerFactoryBean配置即可
package cc.gegee.pangolin.config.atomikos;
import lombok.val;
import org.hibernate.boot.model.naming.ImplicitNamingStrategy;
import org.hibernate.boot.model.naming.PhysicalNamingStrategy;
import org.hibernate.cfg.AvailableSettings;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.boot.autoconfigure.orm.jpa.HibernateProperties;
import org.springframework.boot.autoconfigure.orm.jpa.HibernatePropertiesCustomizer;
import org.springframework.boot.autoconfigure.orm.jpa.HibernateSettings;
import org.springframework.boot.autoconfigure.orm.jpa.JpaProperties;
import org.springframework.boot.jdbc.EmbeddedDatabaseConnection;
import org.springframework.boot.jdbc.SchemaManagement;
import org.springframework.boot.jdbc.SchemaManagementProvider;
import org.springframework.context.annotation.Configuration;
import org.springframework.orm.hibernate5.SpringBeanContainer;
import org.springframework.util.ClassUtils;
import javax.sql.DataSource;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.StreamSupport;
/**
* copy from org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration
* used for default hibernate setting
*/
@Configuration
public class HibernateConfiguration {
private final ObjectProvider<SchemaManagementProvider> providers;
private final HibernateProperties hibernateProperties;
private final JpaProperties properties;
private final ObjectProvider<PhysicalNamingStrategy> physicalNamingStrategy;
private final ObjectProvider<ImplicitNamingStrategy> implicitNamingStrategy;
private final ConfigurableListableBeanFactory beanFactory;
private final ObjectProvider<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers;
public HibernateConfiguration(ObjectProvider<SchemaManagementProvider> providers, HibernateProperties hibernateProperties, JpaProperties properties, ObjectProvider<PhysicalNamingStrategy> physicalNamingStrategy, ObjectProvider<ImplicitNamingStrategy> implicitNamingStrategy, ConfigurableListableBeanFactory beanFactory, ObjectProvider<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers) {
this.providers = providers;
this.hibernateProperties = hibernateProperties;
this.properties = properties;
this.physicalNamingStrategy = physicalNamingStrategy;
this.implicitNamingStrategy = implicitNamingStrategy;
this.beanFactory = beanFactory;
this.hibernatePropertiesCustomizers = hibernatePropertiesCustomizers;
}
/**
* 獲取配置文件信息
*/
public Map<String, Object> getVendorProperties(DataSource dataSource) {
List<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers = determineHibernatePropertiesCustomizers(
physicalNamingStrategy.getIfAvailable(),
implicitNamingStrategy.getIfAvailable(), beanFactory,
this.hibernatePropertiesCustomizers.orderedStream()
.collect(Collectors.toList()));
Supplier<String> defaultDdlMode = () -> new HibernateDefaultDdlAutoProvider(providers)
.getDefaultDdlAuto(dataSource);
val vendorProperties = new LinkedHashMap<>(this.hibernateProperties.determineHibernateProperties(
properties.getProperties(),
new HibernateSettings().ddlAuto(defaultDdlMode)
.hibernatePropertiesCustomizers(
hibernatePropertiesCustomizers)));
// DdlTransactionIsolatorJtaImpl could not locate TransactionManager to suspend any current transaction; base JtaPlatform impl
vendorProperties.put("hibernate.transaction.jta.platform", AtomikosJtaPlatform.class.getName());
// i can not find any reason to set this value, if any question happen, try to set it
vendorProperties.put("javax.persistence.transactionType", "JTA");
// vendorProperties.put("javax.persistence.cache.storeMode", CacheStoreMode.BYPASS);
return vendorProperties;
}
/**
* 命名策略自動(dòng)判斷
*/
private List<HibernatePropertiesCustomizer> determineHibernatePropertiesCustomizers(
PhysicalNamingStrategy physicalNamingStrategy,
ImplicitNamingStrategy implicitNamingStrategy,
ConfigurableListableBeanFactory beanFactory,
List<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers) {
List<HibernatePropertiesCustomizer> customizers = new ArrayList<>();
if (ClassUtils.isPresent(
"org.hibernate.resource.beans.container.spi.BeanContainer",
getClass().getClassLoader())) {
customizers
.add((properties) -> properties.put(AvailableSettings.BEAN_CONTAINER,
new SpringBeanContainer(beanFactory)));
}
if (physicalNamingStrategy != null || implicitNamingStrategy != null) {
customizers.add(new NamingStrategiesHibernatePropertiesCustomizer(
physicalNamingStrategy, implicitNamingStrategy));
}
customizers.addAll(hibernatePropertiesCustomizers);
return customizers;
}
/**
* 自動(dòng)進(jìn)行建表操作
*/
static class HibernateDefaultDdlAutoProvider implements SchemaManagementProvider {
private final Iterable<SchemaManagementProvider> providers;
HibernateDefaultDdlAutoProvider(Iterable<SchemaManagementProvider> providers) {
this.providers = providers;
}
public String getDefaultDdlAuto(DataSource dataSource) {
if (!EmbeddedDatabaseConnection.isEmbedded(dataSource)) {
return "none";
}
SchemaManagement schemaManagement = getSchemaManagement(dataSource);
if (SchemaManagement.MANAGED.equals(schemaManagement)) {
return "none";
}
return "create-drop";
}
@Override
public SchemaManagement getSchemaManagement(DataSource dataSource) {
return StreamSupport.stream(this.providers.spliterator(), false)
.map((provider) -> provider.getSchemaManagement(dataSource))
.filter(SchemaManagement.MANAGED::equals).findFirst()
.orElse(SchemaManagement.UNMANAGED);
}
}
private static class NamingStrategiesHibernatePropertiesCustomizer
implements HibernatePropertiesCustomizer {
private final PhysicalNamingStrategy physicalNamingStrategy;
private final ImplicitNamingStrategy implicitNamingStrategy;
NamingStrategiesHibernatePropertiesCustomizer(PhysicalNamingStrategy physicalNamingStrategy, ImplicitNamingStrategy implicitNamingStrategy) {
this.physicalNamingStrategy = physicalNamingStrategy;
this.implicitNamingStrategy = implicitNamingStrategy;
}
/**
* 數(shù)據(jù)庫(kù)命名映射策略
*
* @param hibernateProperties the JPA vendor properties to customize
*/
@Override
public void customize(Map<String, Object> hibernateProperties) {
if (this.physicalNamingStrategy != null) {
hibernateProperties.put("hibernate.physical_naming_strategy", this.physicalNamingStrategy);
}
if (this.implicitNamingStrategy != null) {
hibernateProperties.put("hibernate.implicit_naming_strategy", this.implicitNamingStrategy);
}
}
}
}
- 最后在配置
LocalContainerEntityManagerFactoryBean
的地方setProperties
即可
多數(shù)據(jù)源事務(wù)
多數(shù)據(jù)源事務(wù)場(chǎng)景及其問(wèn)題
- 只涉及單庫(kù)單服務(wù)事務(wù),沒(méi)問(wèn)題
- 跨庫(kù)跨服務(wù)事務(wù)慷蠕,有問(wèn)題珊拼,當(dāng)內(nèi)部服務(wù)調(diào)用成功之后,而當(dāng)前服務(wù)隨后拋出異常流炕,則內(nèi)部服務(wù)(不同PlatformTransactionManager)不會(huì)回滾杆麸,因?yàn)槭莾蓚€(gè)事務(wù)(借助數(shù)據(jù)庫(kù)實(shí)現(xiàn)的)
解決方案
- 弱一致性
- 消息隊(duì)列
- 強(qiáng)一致性
- XA協(xié)議,兩階段提交協(xié)議2PC浪感、三階段提交協(xié)議3PC是對(duì)2的優(yōu)化
- 最終一致性
- 事務(wù)補(bǔ)償(TCC)
選擇哪個(gè)方案
- 選擇的幾個(gè)維度:一致性強(qiáng)度、吞吐量饼问、實(shí)現(xiàn)復(fù)雜度影兽、架構(gòu)搭建后開發(fā)的復(fù)雜度
- 我們項(xiàng)目的現(xiàn)狀:用戶不多、基本都是數(shù)據(jù)庫(kù)的操作莱革、不希望開發(fā)為此額外做很多開發(fā)
- 這個(gè)帖子我覺(jué)得說(shuō)的蠻有道理的:https://www.atomikos.com/Blog/ToXAOrNotToXA
- 所以選擇XA協(xié)議
分布式事務(wù)XA協(xié)議
- spring官方推薦XA實(shí)現(xiàn)框架:https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-jta
- 選擇XA實(shí)現(xiàn)Atomikos框架
- 關(guān)于XA協(xié)議峻堰,網(wǎng)上有很多資料了,關(guān)于它的二階段提交簡(jiǎn)單來(lái)說(shuō):
- 從模型上講:一個(gè)事務(wù)管理器盅视,多個(gè)資源管理器
- 所謂二階段:第一階段捐名,多個(gè)資源管理器只執(zhí)行不提交。第二階段闹击,如果多個(gè)資源管理器都執(zhí)行成功镶蹋,分別commit,否則rollback
使用atomikos對(duì)原項(xiàng)目改造
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jta-atomikos</artifactId>
</dependency>
use atomikos to impl AbstractJtaPlatform
import com.atomikos.icatch.jta.UserTransactionManager;
import org.hibernate.engine.transaction.jta.platform.internal.AbstractJtaPlatform;
import javax.transaction.TransactionManager;
import javax.transaction.UserTransaction;
public class AtomikosJtaPlatform extends AbstractJtaPlatform {
private static final long serialVersionUID = 1L;
private UserTransactionManager utm;
public AtomikosJtaPlatform() {
utm = new UserTransactionManager();
}
@Override
protected TransactionManager locateTransactionManager() {
return utm;
}
@Override
protected UserTransaction locateUserTransaction() {
return utm;
}
}
datasource
private AtomikosDataSourceBean buildAtomikosDataSourceBean(DataSourceProperties dataSourceProperties, String resourceName) throws SQLException {
val xaDataSource = dataSourceProperties.initializeDataSourceBuilder()
.type(MysqlXADataSource.class).build();
xaDataSource.setPinGlobalTxToPhysicalConnection(true);
val atomikosDataSourceBean = new AtomikosDataSourceBean();
atomikosDataSourceBean.setXaDataSource(xaDataSource);
atomikosDataSourceBean.setUniqueResourceName(resourceName);
atomikosDataSourceBean.setPoolSize(10);
return atomikosDataSourceBean;
}
-
MysqlXADataSource
:主流數(shù)據(jù)庫(kù)都支持XA協(xié)議赏半,我用的是mysql贺归,只要實(shí)現(xiàn)javax.sql.XADataSource
即可 -
PinGlobalTxToPhysicalConnection
:這個(gè)要打開否則無(wú)法使用jta事務(wù)
一個(gè)jta事務(wù)管理器
import cc.gegee.common.jpa.Constant;
import com.atomikos.icatch.jta.UserTransactionImp;
import com.atomikos.icatch.jta.UserTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.transaction.jta.JtaTransactionManager;
import javax.transaction.UserTransaction;
@Configuration
public class JtaTransactionManagerConfig {
@Primary
@Bean(name = Constant.TRANSACTION_MANAGER_JTA)
public JtaTransactionManager regTransactionManager () {
UserTransactionManager userTransactionManager = new UserTransactionManager();
UserTransaction userTransaction = new UserTransactionImp();
return new JtaTransactionManager(userTransaction, userTransactionManager);
}
}
修改原先的配置(另一個(gè)類同)
-
@EnableJpaRepositories
的transactionManagerRef
指向JtaTransactionManager -
LocalContainerEntityManagerFactoryBean
打開JTA
import cc.gegee.common.jpa.Constant;
import com.querydsl.jpa.impl.JPAQueryFactory;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.orm.jpa.SharedEntityManagerCreator;
import javax.persistence.EntityManager;
import javax.sql.DataSource;
import java.util.Objects;
@Configuration
@EnableJpaRepositories(
basePackages = {
Constant.PACKAGES_1
, Constant.PACKAGES_2
, Constant.PACKAGES_3
, Constant.PACKAGES_4
},
entityManagerFactoryRef = Constant.ENTITY_MANAGER_FACTORY_1,
transactionManagerRef = Constant.TRANSACTION_MANAGER_JTA
)
public class JpaConfigurationPangolin {
private final DataSource dataSource;
private final EntityManagerFactoryBuilder builder;
private final HibernateConfiguration hibernateConfiguration;
public JpaConfigurationPangolin(@Qualifier(Constant.DATA_SOURCE_1) DataSource dataSource, EntityManagerFactoryBuilder builder, HibernateConfiguration hibernateConfiguration) {
this.dataSource = dataSource;
this.builder = builder;
this.hibernateConfiguration = hibernateConfiguration;
}
@Bean(name = Constant.ENTITY_MANAGER_FACTORY_1)
@Primary
public LocalContainerEntityManagerFactoryBean entityManagerFactoryBean() {
return builder
.dataSource(dataSource)
.properties(hibernateConfiguration.getVendorProperties(dataSource))
.packages(
Constant.PACKAGES_1
, Constant.PACKAGES_2
, Constant.PACKAGES_3
, Constant.PACKAGES_4
)
.persistenceUnit(Constant.PERSISTENCE_UNIT_1)
.jta(true)
.build();
}
@Bean(name = Constant.ENTITY_MANAGER_1)
@Primary
public EntityManager entityManager() {
// return SharedEntityManagerCreator.createSharedEntityManager(Objects.requireNonNull(entityManagerFactoryBean().getObject()));
return Objects.requireNonNull(entityManagerFactoryBean().getObject()).createEntityManager();
}
@Bean(name = Constant.JPA_QUERY_FACTORY_1)
@Primary
public JPAQueryFactory jpaQueryFactory() {
return new JPAQueryFactory(entityManager());
}
}
使用
- 和原來(lái)沒(méi)什么區(qū)別:在service上配置
@Transactional(rollbackFor = Exception.class)
即可
Hibernate Session的管理問(wèn)題
- OK,到此多數(shù)據(jù)源的配置完結(jié)了断箫,以為可以放松一段一時(shí)間了拂酣,誰(shuí)知測(cè)試發(fā)現(xiàn)一個(gè)奇怪的現(xiàn)象,就是恰巧某個(gè)功能用JPA做了修改之后仲义,使用了querydsl做了查詢婶熬,將結(jié)果返回剑勾。但是查詢來(lái)的值是修改之前的。此外赵颅,其他使用querydsl查詢的地方永遠(yuǎn)不再更新修改后的查詢結(jié)果虽另。
一條彎路
- 經(jīng)過(guò)初步的測(cè)試排查就是querydsl和JPA有'沖突'
- 再經(jīng)過(guò)debug,每次請(qǐng)求JPAQueryFactory使用的session都是同一個(gè)性含,并且其他請(qǐng)求也是同一個(gè)
- 我們知道hibernate默認(rèn)開啟的是一級(jí)緩存(session)洲赵,由于和JPA使用的是不同的兩個(gè)session,JPAQueryFactory的session一直是同一個(gè)商蕴,而JPA執(zhí)行完后又沒(méi)有通知JPAQueryFactory的session叠萍,導(dǎo)致一直查詢的是歷史值
- 開始以為是JPAQueryFactory的session機(jī)制是有緩存了不會(huì)查詢導(dǎo)致,于是打開mysql的日志系統(tǒng)
## 局域網(wǎng)服務(wù)器
ssh xxx@192.168.2.xxx
## 用的docker
docker ps
## 進(jìn)入docker 容器
docker exec -it mysql8.0 /bin/bash
## 搜索my.cnf文件位置
mysql --help --verbose | grep my.cnf
vim my.cnf
## 開啟log
在 my.cnf 設(shè)置 general_log = 1
## 查看
mysql -uroot -p"pwd"
show variables where variable_name like "%general_log%";
## 監(jiān)視日志
tail -f xxxx.log
- 發(fā)現(xiàn)有查詢绪商,但是不知道為什么苛谷,結(jié)果不更新,且仍然從緩存取值
另一角度
- 以前單數(shù)據(jù)源是怎樣的格郁?
- 測(cè)試了下單數(shù)據(jù)源時(shí)腹殿,同時(shí)使用JPA和querydsl,沒(méi)有問(wèn)題例书,并且使用的是同一個(gè)session
- 單數(shù)據(jù)源和多數(shù)據(jù)源querydsl的差別僅在于
EntityManager
是自己創(chuàng)建的锣尉,這里是關(guān)鍵
尋找Spring是如何實(shí)例化EntityManager
- debug JPAQueryFactory實(shí)例化時(shí)的EntityManager,發(fā)現(xiàn)描述為:Shared EntityManager proxy for target factory [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean@3937fd51]
- 我們找到
org.springframework.orm.jpa.SharedEntityManagerCreator
决采,定位到它有一個(gè)關(guān)鍵屬性synchronizedWithTransaction
自沧,最終通過(guò)SharedEntityManagerInvocationHandler
動(dòng)態(tài)代理實(shí)例化EntityManager,增強(qiáng)EntityManager - 我們模擬Spring單數(shù)據(jù)源創(chuàng)建EntityManager树瞭,如下:
@Bean(name = Constant.ENTITY_MANAGER_1)
@Primary
public EntityManager entityManager() {
return SharedEntityManagerCreator.createSharedEntityManager(Objects.requireNonNull(entityManagerFactoryBean().getObject()));
}
后記
- 至此我們完成了多數(shù)據(jù)源的所有配置
- 實(shí)際上拇厢,Spring官網(wǎng)提供了多數(shù)據(jù)源的使用的例子,但是是沒(méi)有考慮事務(wù)晒喷、session管理的問(wèn)題的孝偎,或者說(shuō)是分開考慮
- EntityManager線程安全性:JPA/Hibernate Persistence Context
resources
- GitHub - todo
reference
- SpringBoot2.x Data JPA 多數(shù)據(jù)源爬坑
- Spring Boot 配置 : HibernateJpaConfiguration
- Spring事務(wù)傳播機(jī)制與分布式事務(wù)解決方案
- 嵌套事務(wù)、掛起事務(wù)凉敲,Spring 事務(wù)機(jī)制有什么奧秘衣盾?
- 分布式事務(wù)管理之JTA與鏈?zhǔn)绞聞?wù)
- 還不理解“分布式事務(wù)”?這篇給你講清楚
- Docker下MySQL數(shù)據(jù)庫(kù)日志的打開方法
- Distributed transactions in Spring, with and without XA
- Spring的分布式事務(wù)實(shí)現(xiàn)-使用和不使用XA(翻譯)
- REST微服務(wù)的分布式事務(wù)實(shí)現(xiàn)-分布式系統(tǒng)爷抓、事務(wù)以及JTA介紹
- 微服務(wù)分布式事務(wù)4種解決方案實(shí)戰(zhàn)