一逊笆、基本介紹
ShardingSphere定位為關(guān)系型數(shù)據(jù)庫(kù)中間件
image.png
功能列表
| 功能列表 | 數(shù)據(jù)分片 | 分布式事務(wù) | 數(shù)據(jù)庫(kù)治理 |
|---|---|---|---|
| - | 分庫(kù) & 分表 | 標(biāo)準(zhǔn)化事務(wù)接口 | 配置動(dòng)態(tài)化 |
| - | 讀寫(xiě)分離 | XA強(qiáng)一致事務(wù) | 編排 & 治理 |
| - | 分片策略定制化 | 柔性事務(wù) | 數(shù)據(jù)脫敏 |
| - | 無(wú)中心化分布式主鍵 | 可視化鏈路追蹤 |
核心三套件
| Sharding-JDBC | Sharding-Proxy | Sharding-Sidecar | |
|---|---|---|---|
| 數(shù)據(jù)庫(kù) | 任意 | MySQL | MySQL |
| 連接消耗數(shù) | 高 | 低 | 高 |
| 異構(gòu)語(yǔ)言 | 僅Java | 任意 | 任意 |
| 性能 | 損耗低 | 損耗略高 | 損耗低 |
| 無(wú)中心化 | 是 | 否 | 是 |
| 靜態(tài)入口 | 無(wú) | 有 | 無(wú) |
Sharding-JDBC
客戶(hù)端直連數(shù)據(jù)庫(kù)进肯，以jar包形式提供服務(wù)，無(wú)需額外部署和依賴(lài)扫沼，可理解為增強(qiáng)版的 JDBC驅(qū)動(dòng)出爹，完全兼容JDBC和各種ORM框架。
- 適用于任何基于JDBC的ORM框架：JPA, Hibernate, Mybatis, Spring JDBC Template或直 接使用JDBC缎除。
- 支持任何第三方的數(shù)據(jù)庫(kù)連接池：DBCP, C3P0, BoneCP, Druid, HikariCP等严就。
- 支持任意實(shí)現(xiàn)JDBC規(guī)范的數(shù)據(jù)庫(kù)。支持MySQL器罐，Oracle梢为，SQLServer，PostgreSQL等遵循 SQL92標(biāo)準(zhǔn)的數(shù)據(jù)庫(kù)轰坊。
image.png
Sharding-Proxy
透明化的數(shù)據(jù)庫(kù)代理端铸董，兼容所有MySQL/PostgreSQL協(xié)議的訪問(wèn)客戶(hù)端。
- 向應(yīng)用程序完全透明肴沫，可直接當(dāng)做MySQL/PostgreSQL使用粟害。
- 適用于任何兼容MySQL/PostgreSQL協(xié)議的的客戶(hù)端。
image.png
二颤芬、核心概念
邏輯表
水平拆分的數(shù)據(jù)庫(kù)（表）的相同邏輯和數(shù)據(jù)結(jié)構(gòu)表的總稱(chēng)悲幅。例：訂單數(shù)據(jù)根據(jù)主鍵尾數(shù) 拆分為10張表，分別是t_order_0到t_order_9驻襟，他們的邏輯表名為t_order夺艰。
真實(shí)表
在分片的數(shù)據(jù)庫(kù)中真實(shí)存在的物理表。即上個(gè)示例中的t_order_0到t_order_9沉衣。
數(shù)據(jù)節(jié)點(diǎn)
數(shù)據(jù)分片的最小單元郁副。由數(shù)據(jù)源名稱(chēng)和數(shù)據(jù)表組成，例：ds_0.t_order_0豌习。
綁定表
分片規(guī)則一致的主表和子表存谎。例如：t_order表和t_order_item表，均按照order_id分 片肥隆，則此兩張表互為綁定表關(guān)系既荚。綁定表之間的多表關(guān)聯(lián)查詢(xún)不會(huì)出現(xiàn)笛卡爾積關(guān)聯(lián)，關(guān)聯(lián) 查詢(xún)效率將大大提升栋艳。
廣播表
指所有的分片數(shù)據(jù)源中都存在的表恰聘，表結(jié)構(gòu)和表中的數(shù)據(jù)在每個(gè)數(shù)據(jù)庫(kù)中均完全一致。 適用于數(shù)據(jù)量不大且需要與海量數(shù)據(jù)的表進(jìn)行關(guān)聯(lián)查詢(xún)的場(chǎng)景吸占。字典表就是典型的場(chǎng)景晴叨。
三、ShardingSphere快速啟動(dòng)
引入項(xiàng)目依賴(lài)
<dependencies>
<!‐‐ springboot‐‐>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring‐boot‐starter‐web</artifactId>
<version>2.0.5.RELEASE</version>
</dependency>
<!‐‐ mybatis ‐‐>
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis‐spring‐boot‐starter</artifactId>
<version>2.0.1</version>
<exclusions>
<exclusion>
<artifactId>spring‐boot‐starter</artifactId>
<groupId>org.springframework.boot</groupId>
</exclusion>
</exclusions>
</dependency>
<!‐‐ shardingsphere‐jdbc,這里使用的版本為apache孵化版本矾屯，4.0之前 都是沒(méi)有捐獻(xiàn)給apache基金會(huì)的版本,之前的版本都在‐‐>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding‐jdbc‐spring‐boot‐starter</artifactId>
<version>4.0.0‐RC2</version>
</dependency>
<!‐‐ mysql 驅(qū)動(dòng) ‐‐>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql‐connector‐java</artifactId>
<version>5.1.48</version>
</dependency>
<!‐‐ 可選兼蕊，工具類(lèi) ‐‐>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.16.20</version>
<scope>provided</scope>
</dependency>
</dependencies>
分庫(kù)不分表配置
#配置ds0和ds1兩個(gè)數(shù)據(jù)源,這里有個(gè)坑(使用下劃線可能會(huì)有異常產(chǎn)生，字符不支持,如：ds_0)
spring.shardingsphere.datasource.names=ds0,ds1
#ds0配置
spring.shardingsphere.datasource.ds0.type=com.zaxxer.hikari.HikariDataSource
#數(shù)據(jù)庫(kù)驅(qū)動(dòng)
spring.shardingsphere.datasource.ds0.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds0.jdbc‐url=jdbc:mysql://192.168.241.198:3306/shop_ds_0?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds0.username=root
spring.shardingsphere.datasource.ds0.password=root
#ds1配置
spring.shardingsphere.datasource.ds1.type=com.zaxxer.hikari.HikariDataSource
#數(shù)據(jù)庫(kù)驅(qū)動(dòng)
spring.shardingsphere.datasource.ds1.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds1.jdbc‐url=jdbc:mysql://192.168.241.198:3306/shop_ds_1?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds1.username=root
spring.shardingsphere.datasource.ds1.password=root
#分庫(kù)策略根據(jù)id取模確定數(shù)據(jù)進(jìn)哪個(gè)數(shù)據(jù)庫(kù)
spring.shardingsphere.sharding.default‐database‐strategy.inline.sharding‐column=user_id
spring.shardingsphere.sharding.default‐database‐strategy.inline.algorithm‐expression=ds$‐>{user_id % 2}
#綁定表
spring.shardingsphere.sharding.binding‐tables=t_order,t_order_item
#廣播表
spring.shardingsphere.sharding.broadcast‐tables=t_address
# t_order表策略
spring.shardingsphere.sharding.tables.t_order.actual‐data‐nodes=ds$‐>{0..1}.t_order
#使用SNOWFLAKE算法生成主鍵
spring.shardingsphere.sharding.tables.t_order.key‐generator.column=order_id
spring.shardingsphere.sharding.tables.t_order.key‐generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_order.key‐
generator.props.worker.id=123
# t_order_item表策略
spring.shardingsphere.sharding.tables.t_order_item.actual‐data‐nodes=ds$‐>{0..1}.t_order_item
#使用SNOWFLAKE算法生成主鍵
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.column=order_item_id
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.props.worker.id=123
分庫(kù)分表
#配置ds0和ds1兩個(gè)數(shù)據(jù)源
spring.shardingsphere.datasource.names=ds0,ds1
#ds0配置
spring.shardingsphere.datasource.ds0.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds0.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds0.jdbc‐url=jdbc:mysql://192.168.241.198:3306/shop_ds_0?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds0.username=root
spring.shardingsphere.datasource.ds0.password=root
#ds1配置
spring.shardingsphere.datasource.ds1.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds1.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds1.jdbc‐url=jdbc:mysql://192.168.241.198:3306/shop_ds_1?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds1.username=root
spring.shardingsphere.datasource.ds1.password=root
#分庫(kù)策略根據(jù)id取模確定數(shù)據(jù)進(jìn)哪個(gè)數(shù)據(jù)庫(kù)
spring.shardingsphere.sharding.default‐database‐strategy.inline.sharding‐column=user_id
spring.shardingsphere.sharding.default‐database‐strategy.inline.algorithm‐expression=ds$‐>{user_id % 2}
#綁定表（好像沒(méi)什么卵用）
spring.shardingsphere.sharding.binding‐tables=t_order,t_order_item
#廣播表
spring.shardingsphere.sharding.broadcast‐tables=t_address
#具體分表策略
#節(jié)點(diǎn) ds0.t_order_0,ds0.t_order_1,ds1.t_order_0,ds1.t_order_1
spring.shardingsphere.sharding.tables.t_order.actual‐data‐nodes=ds$‐>{0..1}.t_order_$‐>{0..1}
#分表字段id
spring.shardingsphere.sharding.tables.t_order.table‐strategy.inline.sharding‐column=order_id
#分表策略根據(jù)id取模,確定數(shù)據(jù)最終落在那個(gè)表中
spring.shardingsphere.sharding.tables.t_order.table‐strategy.inline.algorithm‐expression = t_order_$‐>{order_id % 2}
#使用SNOWFLAKE算法生成主鍵
spring.shardingsphere.sharding.tables.t_order.key‐generator.column=order_id
spring.shardingsphere.sharding.tables.t_order.key‐generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_order.key‐generator.props.worker.id=123
#節(jié)點(diǎn) ds0.t_order_item_0,ds0.t_order_item_1,ds1.t_order_item_0,ds1.t_order_item_1
spring.shardingsphere.sharding.tables.t_order_item.actual‐data‐nodes=ds$‐>{0..1}.t_order_item_$‐>{0..1}
#分表字段id
spring.shardingsphere.sharding.tables.t_order_item.table‐strategy.inline.sharding‐column=order_id
#分表策略根據(jù)id取模,確定數(shù)據(jù)最終落在那個(gè)表中
spring.shardingsphere.sharding.tables.t_order_item.table‐strategy.inline.algorithm‐expression=t_order_item_$‐>{order_id % 2}
#使用SNOWFLAKE算法生成主鍵
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.column=order_item_id
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.props.worker.id=123
讀寫(xiě)分離
#shardingsphere讀寫(xiě)分離,master‐slave,可以一主多從
spring.shardingsphere.datasource.names=ds‐master,ds‐slave0
#主庫(kù)
spring.shardingsphere.datasource.ds‐master.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds‐master.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds‐master.jdbc‐url=jdbc:mysql://192.168.241.198:3306/shop_ds_master?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds‐master.username=root
spring.shardingsphere.datasource.ds‐master.password=root
#從庫(kù)0
spring.shardingsphere.datasource.ds‐slave0.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds‐slave0.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds‐slave0.jdbc‐url=jdbc:mysql://192.168.241.199:3306/shop_ds_slave?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds‐slave0.username=root
spring.shardingsphere.datasource.ds‐slave0.password=root
#從庫(kù)1
#spring.shardingsphere.datasource.ds‐slave1.type=com.zaxxer.hikari.HikariDataSource
#spring.shardingsphere.datasource.ds‐slave1.driver‐class‐name=com.mysql.jdbc.Driver
#spring.shardingsphere.datasource.ds‐slave1.jdbc‐url=jdbc:mysql://192.168.241.199:3306/shop_ds_slave1serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
#spring.shardingsphere.datasource.ds‐slave1.username=root
#spring.shardingsphere.datasource.ds‐slave1.password=root
#讀寫(xiě)分離主從規(guī)則設(shè)置件蚕，當(dāng)有2個(gè)以上從庫(kù)時(shí)孙技，從庫(kù)讀采用輪詢(xún)的負(fù)載均衡機(jī)制(也可設(shè)置為隨機(jī)讀)
spring.shardingsphere.masterslave.load‐balance‐algorithm‐type=round_robin
spring.shardingsphere.masterslave.name=ds
spring.shardingsphere.masterslave.master‐data‐source‐name=ds‐master
#如果有多個(gè)從庫(kù),在本配置項(xiàng)后加:,ds‐slave1即可
spring.shardingsphere.masterslave.slave‐data‐source‐names=ds‐slave0
讀寫(xiě)分離+分庫(kù)分表
#shardingsphere讀寫(xiě)分離,master‐slave,可以一主多從
spring.shardingsphere.datasource.names=ds‐master0,ds‐slave0,ds‐master1,ds‐slave1
#主庫(kù)0
spring.shardingsphere.datasource.ds‐master0.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds‐master0.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds‐master0.jdbc‐url=jdbc:mysql://192.168.241.198:3306/shop_ds_master?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds‐master0.username=root
spring.shardingsphere.datasource.ds‐master0.password=root
#從庫(kù)0
spring.shardingsphere.datasource.ds‐slave0.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds‐slave0.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds‐slave0.jdbc‐url=jdbc:mysql://192.168.241.199:3306/shop_ds_slave?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds‐slave0.username=root
spring.shardingsphere.datasource.ds‐slave0.password=root
#主庫(kù)1
spring.shardingsphere.datasource.ds‐master1.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds‐master1.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds‐master1.jdbc‐url=jdbc:mysql://192.168.241.198:3306/shop_ds_master1?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds‐master1.username=root
spring.shardingsphere.datasource.ds‐master1.password=root
#從庫(kù)1
spring.shardingsphere.datasource.ds‐slave1.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds‐slave1.driver‐class‐name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds‐slave1.jdbc‐url=jdbc:mysql://192.168.241.199:3306/shop_ds_slave1?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF‐8
spring.shardingsphere.datasource.ds‐slave1.username=root
spring.shardingsphere.datasource.ds‐slave1.password=root
#分庫(kù)策略根據(jù)id取模確定數(shù)據(jù)進(jìn)哪個(gè)數(shù)據(jù)庫(kù)
spring.shardingsphere.sharding.default‐database‐strategy.inline.sharding‐column=user_id
spring.shardingsphere.sharding.default‐database‐strategy.inline.algorithm‐expression=ds_$‐>{user_id % 2}
#綁定表
sharding.jdbc.config.sharding.binding‐tables=t_order,t_order_item
spring.shardingsphere.sharding.broadcast‐tables=t_address
#分表策略
spring.shardingsphere.sharding.tables.t_order.actual‐data‐nodes=ds_$‐>{0..1}.t_order_$‐>{0..1}
spring.shardingsphere.sharding.tables.t_order.table‐strategy.inline.sharding‐column=order_id
spring.shardingsphere.sharding.tables.t_order.table‐strategy.inline.algorithm‐expression=t_order_$‐>{order_id % 2}
spring.shardingsphere.sharding.tables.t_order.key‐generator.column=order_id
spring.shardingsphere.sharding.tables.t_order.key‐generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_order.key‐generator.props.worker.id=123
spring.shardingsphere.sharding.tables.t_order_item.actual‐data‐nodes=ds_$‐>{0..1}.t_order_item_$‐>{0..1}
spring.shardingsphere.sharding.tables.t_order_item.table‐strategy.inline.sharding‐column=order_id
spring.shardingsphere.sharding.tables.t_order_item.table‐strategy.inline.algorithm‐expression=t_order_item_$‐>{order_id % 2}
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.column=order_item_id
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_order_item.key‐generator.props.worker.id=123
#讀寫(xiě)分離數(shù)據(jù)源0
spring.shardingsphere.sharding.master‐slave‐rules.ds_0.master‐data‐source‐name=ds‐master0
spring.shardingsphere.sharding.master‐slave‐rules.ds_0.slave‐data‐source‐names=ds‐slave0
#讀寫(xiě)分離主從規(guī)則設(shè)置产禾，當(dāng)有2個(gè)以上從庫(kù)時(shí)，從庫(kù)讀采用輪詢(xún)的負(fù)載均衡機(jī)制
spring.shardingsphere.sharding.master‐slave‐rules.ds_0.load‐balance‐algorithm‐type=ROUND_ROBIN
#讀寫(xiě)分離數(shù)據(jù)源1
spring.shardingsphere.sharding.master‐slave‐rules.ds_1.master‐data‐source‐name=ds‐master1
spring.shardingsphere.sharding.master‐slave‐rules.ds_1.slave‐data‐source‐names=ds‐slave1
#讀寫(xiě)分離主從規(guī)則設(shè)置牵啦，當(dāng)有2個(gè)以上從庫(kù)時(shí)亚情，從庫(kù)讀采用輪詢(xún)的負(fù)載均衡機(jī)制
spring.shardingsphere.sharding.master‐slave‐rules.ds_1.load‐balance‐algorithm‐type=ROUND_ROBIN
四、源碼
程序分區(qū)塊
| 名稱(chēng) | 作用 |
|---|---|
| sharding-core | sharding內(nèi)核模塊蕾久；定義了核心api势似，SQL解析，SQL重寫(xiě)僧著，SQL路由，spi障簿，引擎等等核心功能sharding-distribution |
| sharding-distribution | 部署盹愚、運(yùn)維相關(guān)zip包中的代碼 |
| sharding-integration-test | 整合測(cè)試 |
| sharding-jdbc | app-分庫(kù)分表jdbc增強(qiáng) |
| sharding-opentracing | 應(yīng)用性能監(jiān)控 |
| sharding-orchestration | 數(shù)據(jù)庫(kù)編排治理 |
| sharding-proxy | 服務(wù)端代理分庫(kù)分表模塊 |
| sharding-spring | 與spring集成 |
| sharding-sql-test | SQL測(cè)試用例 |
| sharding-transaction | 分布式事務(wù) |
image.png
shardingsphere-分庫(kù)分表底層SQL執(zhí)行過(guò)程分析.jpg
五踏烙、其它
- sharding-jdbc：主要應(yīng)用于應(yīng)用層芍躏，目前只支持Java。
- sharding-proxy：主要應(yīng)用于db代理層值戳，目前只支持Mysql數(shù)據(jù)庫(kù)西篓，類(lèi)似于Mycat愈腾，與zookeeper結(jié)合使用能實(shí)現(xiàn)動(dòng)態(tài)配置管理。
