以下教程為整合官方教程,主要是官方內(nèi)容锅纺,但是官方內(nèi)容有點(diǎn)亂,版本較低, 以下已完整測(cè)試及附帶個(gè)人測(cè)試結(jié)果和分析德绿。
MyCAT 1.6.7.1版本
1. 準(zhǔn)備
1.1. 操作系統(tǒng)
mac
注:操作系統(tǒng)的不同沒(méi)關(guān)系的献雅,mycat的安裝方式很簡(jiǎn)單,稍微把環(huán)境變量配上去就可以開(kāi)始測(cè)試了轿曙。
1.2. 環(huán)境
數(shù)據(jù)庫(kù):
Mysql:127.0.0.1:3306。
本人當(dāng)前版本為: mysql Ver 8.0.13 for macos10.14 on x86_64 (MySQL Community Server - GPL)
java
mycat是用Java開(kāi)發(fā)僻孝,需要有JAVA運(yùn)行環(huán)境导帝,mycat依賴(lài) >=jdk1.7的環(huán)境,若本機(jī)沒(méi)有穿铆,則需要下載安裝:<u>http://www.java.com/zh_CN/</u>.
本人版本為:
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
2. 下載并安裝mycat
2.1. 關(guān)于mycat
獲取MyCAT的最新開(kāi)源版本您单,項(xiàng)目主頁(yè)http://www.mycat.io/
2.2. 下載mycat
2.3. 安裝mycat
下載并解壓后,建議放在/usr/local/mycat目錄下, 并加入環(huán)境變量中:
mycat
export PATH=${PATH}:/usr/local/mycat/bin.
2.4. 目錄解釋
--bin 啟動(dòng)目錄
--conf 配置目錄存放配置文件:
--server.xml:是Mycat服務(wù)器參數(shù)調(diào)整和用戶(hù)授權(quán)的配置文件荞雏。
--schema.xml:是邏輯庫(kù)定義和表以及分片定義的配置文件虐秦。
--rule.xml: 是分片規(guī)則的配置文件,分片規(guī)則的具體一些參數(shù)信息單獨(dú)存放為文件凤优,也在這個(gè)目錄下悦陋,配置文件修改需要重啟MyCAT。
--log4j.xml: 日志存放在logs/log中筑辨,每天一個(gè)文件俺驶,日志的配置是在conf/log4j.xml中,根據(jù)自己的需要可以調(diào)整輸出級(jí)別為debug debug級(jí)別下挖垛,會(huì)輸出更多的信息痒钝,方便排查問(wèn)題。
--autopartition-long.txt,partition-hash-int.txt,sequence_conf.properties痢毒, sequence_db_conf.properties 分片相關(guān)的id分片規(guī)則配置文件
--lib MyCAT自身的jar包或依賴(lài)的jar包的存放目錄送矩。
--logs MyCAT日志的存放目錄。日志存放在logs/log中哪替,每天一個(gè)文件
| 附官方完整配置及個(gè)人注釋
我們識(shí)別下配置中重要結(jié)構(gòu)及名詞解釋對(duì)照
+ mycat
+ 邏輯庫(kù)(虛擬庫(kù))(schema)
+ 邏輯表(虛擬表)(table)
+ 數(shù)據(jù)庫(kù)實(shí)例-分片節(jié)點(diǎn)(dataNode)栋荸、表-分片規(guī)則(type='global' 或者 rule='xxx')
+ 數(shù)據(jù)庫(kù)實(shí)例
+ 數(shù)據(jù)庫(kù)實(shí)例-分片節(jié)(dataNode):對(duì)應(yīng)到多個(gè)數(shù)據(jù)庫(kù)實(shí)例
+ 節(jié)點(diǎn)主機(jī)(物理主機(jī))(dataHost):對(duì)應(yīng)到一個(gè)數(shù)據(jù)庫(kù)實(shí)例存在于多少臺(tái)主機(jī)(主主復(fù)制+主從復(fù)制))
+ 寫(xiě)主機(jī)(writeHost): 多個(gè)主主復(fù)制的主主機(jī)
+ 讀主機(jī)(readHost):多個(gè)主從復(fù)制的從主機(jī)
server.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
-server.xml:是Mycat服務(wù)器參數(shù)調(diào)整和用戶(hù)授權(quán)的配置文件。
-schema.xml:是邏輯庫(kù)定義和表以及分片定義的配置文件凭舶。
-rule.xml: 是分片規(guī)則的配置文件晌块,分片規(guī)則的具體一些參數(shù)信息單獨(dú)存放為文件,也在這個(gè)目錄下帅霜,配置文件修改需要重啟MyCAT匆背。
-log4j.xml: 日志存放在logs/log中,每天一個(gè)文件身冀,日志的配置是在conf/log4j.xml中钝尸,根據(jù)自己的需要可以調(diào)整輸出級(jí)別為debug。debug級(jí)別下搂根,會(huì)輸出更多的信息珍促,方便排查問(wèn)題。
-autopartition-long.txt,partition-hash-int.txt,sequence_conf.properties剩愧, sequence_db_conf.properties 分片相關(guān)的id分片規(guī)則配置文件
-lib MyCAT自身的jar包或依賴(lài)的jar包的存放目錄猪叙。
-logs MyCAT日志的存放目錄。日志存放在logs/log中仁卷,每天一個(gè)文件
-->
<!-- - - Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License. - You
may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0
- - Unless required by applicable law or agreed to in writing, software -
distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the
License for the specific language governing permissions and - limitations
under the License. -->
<!DOCTYPE mycat:server SYSTEM "server.dtd">
<mycat:server xmlns:mycat="http://io.mycat/">
<system>
<property name="nonePasswordLogin">0</property> <!-- 0為需要密碼登陸穴翩、1為不需要密碼登陸 ,默認(rèn)為0,設(shè)置為1則需要指定默認(rèn)賬戶(hù)-->
<property name="useHandshakeV10">1</property>
<property name="useSqlStat">0</property> <!-- 1為開(kāi)啟實(shí)時(shí)統(tǒng)計(jì)锦积、0為關(guān)閉 -->
<property name="useGlobleTableCheck">0</property> <!-- 1為開(kāi)啟全加班一致性檢測(cè)藏否、0為關(guān)閉 -->
<property name="sequnceHandlerType">2</property>
<!--<property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|\)|\s)*)+</property>-->
<!--必須帶有MYCATSEQ_或者 mycatseq_進(jìn)入序列匹配流程 注意MYCATSEQ_有空格的情況-->
<property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|\)|\s)*)+</property>
<property name="subqueryRelationshipCheck">false</property> <!-- 子查詢(xún)中存在關(guān)聯(lián)查詢(xún)的情況下,檢查關(guān)聯(lián)字段中是否有分片字段 .默認(rèn) false -->
<!-- <property name="useCompression">1</property>--> <!--1為開(kāi)啟mysql壓縮協(xié)議-->
<!-- <property name="fakeMySQLVersion">5.6.20</property>--> <!--設(shè)置模擬的MySQL版本號(hào)-->
<!-- <property name="processorBufferChunk">40960</property> -->
<!--
<property name="processors">1</property>
<property name="processorExecutor">32</property>
-->
<!--默認(rèn)為type 0: DirectByteBufferPool | type 1 ByteBufferArena | type 2 NettyBufferPool -->
<property name="processorBufferPoolType">0</property>
<!--默認(rèn)是65535 64K 用于sql解析時(shí)最大文本長(zhǎng)度 -->
<!--<property name="maxStringLiteralLength">65535</property>-->
<!--<property name="sequnceHandlerType">0</property>-->
<!--<property name="backSocketNoDelay">1</property>-->
<!--<property name="frontSocketNoDelay">1</property>-->
<!--<property name="processorExecutor">16</property>-->
<!--
<property name="serverPort">8066</property> <property name="managerPort">9066</property>
<property name="idleTimeout">300000</property> <property name="bindIp">0.0.0.0</property>
<property name="frontWriteQueueSize">4096</property> <property name="processors">32</property> -->
<!--分布式事務(wù)開(kāi)關(guān),0為不過(guò)濾分布式事務(wù)充包,1為過(guò)濾分布式事務(wù)(如果分布式事務(wù)內(nèi)只涉及全局表副签,則不過(guò)濾),2為不過(guò)濾分布式事務(wù),但是記錄分布式事務(wù)日志-->
<property name="handleDistributedTransactions">0</property>
<!--
off heap for merge/order/group/limit 1開(kāi)啟 0關(guān)閉
-->
<property name="useOffHeapForMerge">0</property>
<!--
單位為m
-->
<property name="memoryPageSize">64k</property>
<!--
單位為k
-->
<property name="spillsFileBufferSize">1k</property>
<property name="useStreamOutput">0</property>
<!--
單位為m
-->
<property name="systemReserveMemorySize">384m</property>
<!--是否采用zookeeper協(xié)調(diào)切換 -->
<property name="useZKSwitch">false</property>
<!-- XA Recovery Log日志路徑 -->
<!--<property name="XARecoveryLogBaseDir">./</property>-->
<!-- XA Recovery Log日志名稱(chēng) -->
<!--<property name="XARecoveryLogBaseName">tmlog</property>-->
<!--如果為 true的話(huà) 嚴(yán)格遵守隔離級(jí)別,不會(huì)在僅僅只有select語(yǔ)句的時(shí)候在事務(wù)中切換連接-->
<property name="strictTxIsolation">false</property>
<property name="useZKSwitch">true</property>
</system>
<!-- 全局SQL防火墻設(shè)置 -->
<!--白名單可以使用通配符%或著*-->
<!--例如<host host="127.0.0.*" user="root"/>-->
<!--例如<host host="127.0.*" user="root"/>-->
<!--例如<host host="127.*" user="root"/>-->
<!--例如<host host="1*7.*" user="root"/>-->
<!--這些配置情況下對(duì)于127.0.0.1都能以root賬戶(hù)登錄-->
<!--
<firewall>
<whitehost>
<host host="1*7.0.0.*" user="root"/>
</whitehost>
<blacklist check="false">
</blacklist>
</firewall>
-->
<!--
配置Mycat邏輯庫(kù)與用戶(hù)
-->
<user name="root" defaultAccount="true">
<property name="password">123456</property>
<property name="schemas">TESTDB</property>
<!-- 表級(jí) DML 權(quán)限設(shè)置 -->
<!--
<privileges check="false">
<schema name="TESTDB" dml="0110" >
<table name="tb01" dml="0000"></table>
<table name="tb02" dml="1111"></table>
</schema>
</privileges>
-->
</user>
<user name="user">
<property name="password">user</property>
<property name="schemas">TESTDB</property>
<property name="readOnly">true</property>
</user>
</mycat:server>
schema.xml
<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">
<!--
邏輯庫(kù)定義和表以及分片定義的配置文件
-->
<!--
邏輯庫(kù)
-->
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
<!--
邏輯表配置 name=xxx基矮;表分片規(guī)則 rule=xxx
-->
<!-- auto sharding by id (long) -->
<table name="travelrecord" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
<!-- global table is auto cloned to all defined data nodes ,so can join with any table whose sharding node is in the same data node -->
<table name="company" primaryKey="ID" type="global" dataNode="dn1,dn2,dn3" />
<table name="goods" primaryKey="ID" type="global" dataNode="dn1,dn2" />
<!-- 分片規(guī)則為取模時(shí)淆储,與邏輯表中定義的分片節(jié)點(diǎn)順序有關(guān),從左到右索引從0開(kāi)始遞增(分片節(jié)點(diǎn)個(gè)數(shù)-1) -->
<!-- random sharding using mod sharind rule -->
<table name="hotnews" primaryKey="ID" autoIncrement="true" dataNode="dn1,dn2,dn3" rule="mod-long" />
<!-- <table name="dual" primaryKey="ID" dataNode="dnx,dnoracle2" type="global" needAddLimit="false"/> <table name="worker" primaryKey="ID" dataNode="jdbc_dn1,jdbc_dn2,jdbc_dn3" rule="mod-long" /> -->
<table name="employee" primaryKey="ID" dataNode="dn1,dn2" rule="sharding-by-intfile" />
<table name="customer" primaryKey="ID" dataNode="dn1,dn2" rule="sharding-by-intfile">
<childTable name="orders" primaryKey="ID" joinKey="customer_id" parentKey="id">
<childTable name="order_items" joinKey="order_id" parentKey="id" />
</childTable>
<childTable name="customer_addr" primaryKey="ID" joinKey="customer_id" parentKey="id" />
</table>
<!-- <table name="oc_call" primaryKey="ID" dataNode="dn1$0-743" rule="latest-month-calldate" /> -->
</schema>
<!--
分片節(jié)點(diǎn):配置分片(dataNode)
-->
<!-- <dataNode name="dn1$0-743" dataHost="localhost1" database="db$0-743" /> -->
<dataNode name="dn1" dataHost="localhost1" database="db1" />
<dataNode name="dn2" dataHost="localhost1" database="db2" />
<dataNode name="dn3" dataHost="localhost1" database="db3" />
<!--<dataNode name="dn4" dataHost="sequoiadb1" database="SAMPLE" />
<dataNode name="jdbc_dn1" dataHost="jdbchost" database="db1" />
<dataNode name="jdbc_dn2" dataHost="jdbchost" database="db2" />
<dataNode name="jdbc_dn3" dataHost="jdbchost" database="db3" /> -->
<!--
節(jié)點(diǎn)主機(jī):配置物理庫(kù)分片映射(dataHost)
修改dataHost和schema對(duì)應(yīng)的連接信息家浇,user,pay 垂直切分后的配置如下所示:
schema 是實(shí)際邏輯庫(kù)的配置本砰,user,pay分別對(duì)應(yīng)兩個(gè)邏輯庫(kù)钢悲,多個(gè)schema代表多個(gè)邏輯庫(kù)点额。
dataNode是邏輯庫(kù)對(duì)應(yīng)的分片舔株,如果配置多個(gè)分片只需要多個(gè)dataNode即可。
dataHost是實(shí)際的物理庫(kù)配置地址还棱,可以配置多主主從等其他配置载慈,多個(gè)dataHost代表分片對(duì)應(yīng)的物理庫(kù)地址,下面的writeHost珍手、readHost代表該分片是否配置多寫(xiě)办铡,主從,讀寫(xiě)分離等高級(jí)特性琳要。
-->
<!--
(1)設(shè)置 balance="1"與writeType="0"
Balance參數(shù)設(shè)置:
1. balance=“0”, 所有讀操作都發(fā)送到當(dāng)前可用的writeHost上寡具。
2. balance=“1”,所有讀操作都隨機(jī)的發(fā)送到readHost稚补。
3. balance=“2”童叠,所有讀操作都隨機(jī)的在writeHost、readhost上分發(fā)
WriteType參數(shù)設(shè)置:
1. writeType=“0”, 所有寫(xiě)操作都發(fā)送到可用的writeHost上课幕。
2. writeType=“1”拯钻,所有寫(xiě)操作都隨機(jī)的發(fā)送到readHost。
3. writeType=“2”撰豺,所有寫(xiě)操作都隨機(jī)的在writeHost粪般、readhost分上發(fā)。
“readHost是從屬于writeHost的污桦,即意味著它從那個(gè)writeHost獲取同步數(shù)據(jù)亩歹,因此,當(dāng)它所屬的writeHost宕機(jī)了凡橱,則它也不會(huì)再參與到讀寫(xiě)分離中來(lái)小作,即“不工作了”,這是因?yàn)榇藭r(shí)稼钩,它的數(shù)據(jù)已經(jīng)“不可靠”了顾稀。基于這個(gè)考慮坝撑,目前mycat 1.3和1.4版本中静秆,若想支持MySQL一主一從的標(biāo)準(zhǔn)配置,并且在主節(jié)點(diǎn)宕機(jī)的情況下巡李,從節(jié)點(diǎn)還能讀取數(shù)據(jù)抚笔,則需要在Mycat里配置為兩個(gè)writeHost并設(shè)置banlance=1∏壤梗”
注意:請(qǐng)不要配置writeType=1殊橙,會(huì)導(dǎo)致數(shù)據(jù)庫(kù)不一致的嚴(yán)重問(wèn)題。
重點(diǎn)測(cè)試balance = 1 & wirteType = 0 時(shí)
| writeHost | 是否存在readHost | readHost | 查詢(xún) | 更新 | 注意 |
| 正常 | 是 | 正常 | 優(yōu)先readHost | 優(yōu)先writeHost優(yōu)先 | 當(dāng)只有一個(gè)writeHost時(shí),因?yàn)閞eadHost屬于writeHost膨蛮,當(dāng)writeHost不可用時(shí)叠纹,readHost同樣不可以使用。當(dāng)有多個(gè)writeHost時(shí)敞葛,其中一個(gè)writeHost有故障且包含readHost誉察,而其他writeHost可用時(shí),readHost可用到心跳檢測(cè)后制肮,因?yàn)橐呀?jīng)判定其父主機(jī)writeHost不可用冒窍。然后切換到可用的writeHost主機(jī)递沪。若其他writeHost有不可用的則所有的writeHost和readHost將不可用 |
| 正常 | 否 | 無(wú) | writeHost | writeHost | 當(dāng)有多個(gè)writeHost且有一個(gè)故障時(shí)豺鼻,在心跳檢測(cè)之前所有的writeHost都不可用,檢測(cè)過(guò)后可用的writeHost正常使用款慨。 |
| 正常 | 是 | 不正常 | 先嘗試readHost儒飒,等到心跳檢測(cè)到故障點(diǎn)后切換到writeHost | 無(wú)延遲,直接使用writeHost | 當(dāng)writeHost中包含多個(gè)readHost時(shí)檩奠,有其中一個(gè)readHost故障則桩了,切換到故障的readHost時(shí)會(huì)自動(dòng)切換到WriteHost中,而正常的readHost不受影響埠戳。 |
(2)設(shè)置 switchType="2" 與slaveThreshold="100"
- -1 表示不自動(dòng)切換
- 1 默認(rèn)值井誉,自動(dòng)切換
- 2 基于MySQL主從同步的狀態(tài)決定是否切換
“Mycat心跳檢查語(yǔ)句配置為 show slave status ,dataHost 上定義兩個(gè)新屬性: switchType="2" 與slaveThreshold="100"整胃,此時(shí)意味著開(kāi)啟MySQL主從復(fù)制狀態(tài)綁定的讀寫(xiě)分離與切換機(jī)制颗圣。Mycat心跳機(jī)制通過(guò)檢測(cè) show slave status 中的 "Seconds_Behind_Master", "Slave_IO_Running", "Slave_SQL_Running" 三個(gè)字段來(lái)確定當(dāng)前主從同步的狀態(tài)以及Seconds_Behind_Master主從復(fù)制時(shí)延∑ㄊ梗“
T谄瘛!蛮寂!注意:配置writeHost與readHost的使用蔽午,需要配合mysql服務(wù)器做主從或者主主或者集群!3晏!<袄稀!
-->
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- mycat連接的數(shù)據(jù)庫(kù)配置(mysql服務(wù)器) -->
<!-- 根據(jù)節(jié)點(diǎn)主機(jī) dataHost name="localhost1" 對(duì)應(yīng)的分片節(jié)點(diǎn) dataNode 包含三個(gè)數(shù)據(jù)庫(kù):db1范抓、db2写半、db3,則數(shù)據(jù)庫(kù)在建表時(shí)應(yīng)將數(shù)據(jù)庫(kù)建立好 -->
<!-- can have multi write hosts -->
<writeHost host="hostM4" url="192.168.99.100:33060" user="root" password="p" >
<!-- <readHost host="hostS4" url="192.168.99.100:33061" user="root" password="p" /> -->
<!-- <readHost host="hostS41" url="127.0.0.1:3306" user="root" password="leeprince" /> -->
</writeHost>
<writeHost host="hostM1" url="127.0.0.1:3306" user="root" password="leeprince">
<!-- <readHost host="hostM3" url="192.168.99.100:33060" user="root" password="p" />
<readHost host="hostS1" url="192.168.99.100:33061" user="root" password="p" /> -->
</writeHost>
<!-- <writeHost host="hostM2" url="192.168.99.100:33060" user="root" password="p">
<readHost host="hostS2" url="192.168.99.100:33061" user="root" password="p" />
</writeHost> -->
<!-- <writeHost host="hostM3" url="192.168.99.100:33060" user="root" password="p" /> -->
</dataHost>
<!-- <dataHost name="sequoiadb1" maxCon="1000" minCon="1" balance="0" dbType="sequoiadb" dbDriver="jdbc">
<heartbeat> </heartbeat>
<writeHost host="hostM1" url="sequoiadb://1426587161.dbaas.sequoialab.net:11920/SAMPLE" user="jifeng" password="jifeng">
</writeHost>
</dataHost> -->
<!-- <dataHost name="oracle1" maxCon="1000" minCon="1" balance="0" writeType="0" dbType="oracle" dbDriver="jdbc">
<heartbeat>select 1 from dual</heartbeat>
<connectionInitSql>alter session set nls_date_format='yyyy-mm-dd hh24:mi:ss'</connectionInitSql>
<writeHost host="hostM1" url="jdbc:oracle:thin:@127.0.0.1:1521:nange" user="base" password="123456" >
</writeHost>
</dataHost> -->
<!-- <dataHost name="jdbchost" maxCon="1000" minCon="1" balance="0" writeType="0" dbType="mongodb" dbDriver="jdbc">
<heartbeat>select user()</heartbeat>
<writeHost host="hostM" url="mongodb://192.168.0.99/test" user="admin" password="123456" >
</writeHost>
</dataHost> -->
<!-- <dataHost name="sparksql" maxCon="1000" minCon="1" balance="0" dbType="spark" dbDriver="jdbc">
<heartbeat> </heartbeat>
<writeHost host="hostM1" url="jdbc:hive2://feng01:10000" user="jifeng" password="jifeng">
</writeHost>
</dataHost> -->
<!-- <dataHost name="jdbchost" maxCon="1000" minCon="10" balance="0" dbType="mysql" dbDriver="jdbc">
<heartbeat>select user()</heartbeat>
<writeHost host="hostM1" url="jdbc:mysql://localhost:3306" user="root" password="123456"> </writeHost>
</dataHost> -->
</mycat:schema>
rule.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
分片規(guī)則的配置文件,分片規(guī)則的具體一些參數(shù)信息單獨(dú)存放為文件尉咕,也在這個(gè)目錄下叠蝇,配置文件修改需要重啟MyCAT
-->
<!-- - - Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License. - You
may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0
- - Unless required by applicable law or agreed to in writing, software -
distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the
License for the specific language governing permissions and - limitations
under the License. -->
<!DOCTYPE mycat:rule SYSTEM "rule.dtd">
<mycat:rule xmlns:mycat="http://io.mycat/">
<!--
name 為schema.xml 中table 標(biāo)簽中對(duì)應(yīng)的 rule="sharding-by-hour" ,也就是配置表的分片規(guī)則,
columns 是表的切分字段: createTime 創(chuàng)建日期。
algorithm 是規(guī)則對(duì)應(yīng)的切分規(guī)則:映射到function 的name悔捶。
-->
<tableRule name="rule1">
<rule>
<columns>id</columns>
<algorithm>func1</algorithm>
</rule>
</tableRule>
<tableRule name="rule2">
<rule>
<columns>user_id</columns>
<algorithm>func1</algorithm>
</rule>
</tableRule>
<tableRule name="sharding-by-intfile">
<rule>
<columns>sharding_id</columns>
<algorithm>hash-int</algorithm>
</rule>
</tableRule>
<tableRule name="auto-sharding-long">
<rule>
<columns>id</columns>
<algorithm>rang-long</algorithm>
</rule>
</tableRule>
<tableRule name="mod-long">
<rule>
<columns>id</columns>
<algorithm>mod-long</algorithm>
</rule>
</tableRule>
<tableRule name="sharding-by-murmur">
<rule>
<columns>id</columns>
<algorithm>murmur</algorithm>
</rule>
</tableRule>
<tableRule name="crc32slot">
<rule>
<columns>id</columns>
<algorithm>crc32slot</algorithm>
</rule>
</tableRule>
<tableRule name="sharding-by-month">
<rule>
<columns>create_time</columns>
<algorithm>partbymonth</algorithm>
</rule>
</tableRule>
<tableRule name="latest-month-calldate">
<rule>
<columns>calldate</columns>
<algorithm>latestMonth</algorithm>
</rule>
</tableRule>
<tableRule name="auto-sharding-rang-mod">
<rule>
<columns>id</columns>
<algorithm>rang-mod</algorithm>
</rule>
</tableRule>
<tableRule name="jch">
<rule>
<columns>id</columns>
<algorithm>jump-consistent-hash</algorithm>
</rule>
</tableRule>
<function name="murmur" class="io.mycat.route.function.PartitionByMurmurHash">
<property name="seed">0</property><!-- 默認(rèn)是0 -->
<property name="count">2</property><!-- 要分片的數(shù)據(jù)庫(kù)節(jié)點(diǎn)數(shù)量铃慷,必須指定,否則沒(méi)法分片 -->
<property name="virtualBucketTimes">160</property><!-- 一個(gè)實(shí)際的數(shù)據(jù)庫(kù)節(jié)點(diǎn)被映射為這么多虛擬節(jié)點(diǎn)蜕该,默認(rèn)是160倍犁柜,也就是虛擬節(jié)點(diǎn)數(shù)是物理節(jié)點(diǎn)數(shù)的160倍 -->
<!-- <property name="weightMapFile">weightMapFile</property> 節(jié)點(diǎn)的權(quán)重,沒(méi)有指定權(quán)重的節(jié)點(diǎn)默認(rèn)是1堂淡。以properties文件的格式填寫(xiě)馋缅,以從0開(kāi)始到count-1的整數(shù)值也就是節(jié)點(diǎn)索引為key,以節(jié)點(diǎn)權(quán)重值為值绢淀。所有權(quán)重值必須是正整數(shù)萤悴,否則以1代替 -->
<!-- 用于測(cè)試時(shí)觀察各物理節(jié)點(diǎn)與虛擬節(jié)點(diǎn)的分布情況,如果指定了這個(gè)屬性皆的,會(huì)把虛擬節(jié)點(diǎn)的murmur hash值與物理節(jié)點(diǎn)的映射按行輸出到這個(gè)文件覆履,沒(méi)有默認(rèn)值,如果不指定费薄,就不會(huì)輸出任何東西 -->
<property name="bucketMapPath">/etc/mycat/bucketMapPath</property>
</function>
<function name="crc32slot" class="io.mycat.route.function.PartitionByCRC32PreSlot">
</function>
<function name="hash-int" class="io.mycat.route.function.PartitionByFileMap">
<property name="mapFile">partition-hash-int.txt</property>
</function>
<function name="rang-long" class="io.mycat.route.function.AutoPartitionByLong">
<property name="mapFile">autopartition-long.txt</property>
</function>
<function name="mod-long" class="io.mycat.route.function.PartitionByMod">
<!-- how many data nodes -->
<property name="count">3</property>
</function>
<function name="func1" class="io.mycat.route.function.PartitionByLong">
<property name="partitionCount">8</property>
<property name="partitionLength">128</property>
</function>
<function name="latestMonth" class="io.mycat.route.function.LatestMonthPartion">
<property name="splitOneDay">24</property>
</function>
<function name="partbymonth" class="io.mycat.route.function.PartitionByMonth">
<property name="dateFormat">yyyy-MM-dd</property>
<property name="sBeginDate">2015-01-01</property>
</function>
<function name="rang-mod" class="io.mycat.route.function.PartitionByRangeMod">
<property name="mapFile">partition-range-mod.txt</property>
</function>
<function name="jump-consistent-hash" class="io.mycat.route.function.PartitionByJumpConsistentHash">
<property name="totalBuckets">3</property>
</function>
</mycat:rule>
2.4.1. bin 程序目錄
存放了window版本硝全、linux版本 、mac楞抡,除了提供封裝成服務(wù)的版本之外伟众,也提供了nowrap的shell腳本命令,方便大家選擇和修改
2.4.2. conf目錄下存放配置文件
server.xml是Mycat服務(wù)器參數(shù)調(diào)整和用戶(hù)授權(quán)的配置文件召廷,schema.xml是邏輯庫(kù)定義和表以及分片定義的配置文件凳厢,rule.xml是分片規(guī)則的配置文件,分片規(guī)則的具體一些參數(shù)信息單獨(dú)存放為文件柱恤,也在這個(gè)目錄下数初,配置文件修改,需要重啟Mycat或者通過(guò)9066端口reload梗顺。
提示:訪問(wèn)MyCAT的用戶(hù)賬號(hào)和授權(quán)信息是在conf/server.xml文件中配置泡孩,而MyCAT用來(lái)連接后端MySQL庫(kù)的用戶(hù)名密碼信息則在conf/schema.xml中,這是兩套完全獨(dú)立的系統(tǒng)寺谤,類(lèi)似的還有MyCAT的邏輯庫(kù)(schema)仑鸥,邏輯表(table)也是類(lèi)似的。
2.4.3. 日志存放在logs/mycat.log
每天一個(gè)文件变屁,日志的配置是在conf/log4j.xml中眼俊,根據(jù)自己的需要,可以調(diào)整輸出級(jí)別為debug粟关,debug級(jí)別下疮胖,會(huì)輸出更多的信息,方便排查問(wèn)題。
注意澎灸,wrap方式的程序院塞,其JVM配置參數(shù)在conf/wrap.conf中,可以修改為合適的參數(shù)性昭,參數(shù)調(diào)整參照http://wrapper.tanukisoftware.com/doc/english/properties.html拦止。用下面是一段實(shí)例:
注:mycat必須依賴(lài) >=jdk1.7,在1.6的情景下會(huì)報(bào)錯(cuò)糜颠,如果機(jī)器未升級(jí)可以指定jdk的目錄汹族,我考了一個(gè)jdk的包出來(lái)的,添加的絕對(duì)路徑其兴,根據(jù)情況定顶瞒。
wrapper.java.command=/usr/local/Mycat/jdk1.7.0/bin/java
Java Additional Parameters
wrapper.java.additional.5=-XX:MaxDirectMemorySize=2G
wrapper.java.additional.6=-Dcom.sun.management.jmxremote
Initial Java Heap Size (in MB)
wrapper.java.initmemory=2048
Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=2048
若啟動(dòng)報(bào)內(nèi)存不夠,可以試著將上述內(nèi)存都改小忌警,改為1G或500M搁拙。
3. 關(guān)于mycat命令秒梳,并啟動(dòng)
./mycat start 啟動(dòng)
./mycat stop 停止
./mycat console 前臺(tái)運(yùn)行
./mycat install 添加到系統(tǒng)自動(dòng)啟動(dòng)(暫未實(shí)現(xiàn))
./mycat remove 取消隨系統(tǒng)自動(dòng)啟動(dòng)(暫未實(shí)現(xiàn))
./mycat restart 重啟服務(wù)
./mycat pause 暫停
./mycat status 查看啟動(dòng)狀態(tài)
4. 啟動(dòng)mysql
用命令行工具或圖形化客戶(hù)端法绵,連接MYSQL,創(chuàng)建DEMO所用三個(gè)分片數(shù)據(jù)庫(kù)酪碘;
CREATE database db1;
CREATE database db2;
CREATE database db3;
注意:若是LINUX版本的MYSQL朋譬,則需要設(shè)置為Mysql大小寫(xiě)不敏感,否則可能會(huì)發(fā)生表找不到的問(wèn)題兴垦。windows與mac不需要
注:在MySQL的配置文件中my.ini [mysqld] 中增加一行
lower_case_table_names = 1
5. 修改配置并啟動(dòng)mycat
5.1. 修改配置
編輯MYCAT_HOME/conf/schema.xml文件徙赢,修改節(jié)點(diǎn)主機(jī)dataHost對(duì)應(yīng)的連接信息:
<dataNode name="dn1" dataHost="localhost1" database="db1" />
<dataNode name="dn2" dataHost="localhost1" database="db2" />
<dataNode name="dn3" dataHost="localhost1" database="db3" />
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- mycat連接的數(shù)據(jù)庫(kù)配置(mysql服務(wù)器) -->
<!-- 根據(jù)節(jié)點(diǎn)主機(jī) dataHost name="localhost1" 對(duì)應(yīng)的分片節(jié)點(diǎn) dataNode 包含三個(gè)數(shù)據(jù)庫(kù):db1、db2探越、db3,則數(shù)據(jù)庫(kù)在建表時(shí)應(yīng)將數(shù)據(jù)庫(kù)建立好 -->
<!-- can have multi write hosts -->
<writeHost host="hostM1" url="127.0.0.1:3306" user="root" password="leeprince">
<!-- can have multi read hosts -->
<!-- <readHost host="hostS2" url="192.168.1.200:3306" user="root" password="xxx" /> -->
</writeHost>
<!-- <writeHost host="hostS1" url="localhost:3316" user="root" password="123456" /> -->
<!-- <writeHost host="hostM2" url="localhost:3316" user="root" password="123456"/> -->
</dataHost>
注意writeHost/readHost中的location,user,password的值符合你所采用的Mysql的連接信息狡赐。
修改完成后保存
5.2. 啟動(dòng)mycat
進(jìn)入到MYCAT_HOME/bin(如果已經(jīng)配置則直接mycat)目錄下,執(zhí)行啟動(dòng)命令:mycat start
注意钦幔,默認(rèn)數(shù)據(jù)端口為8066,管理端口為9066。
6. 連接mycat
6.1.客戶(hù)端連接
客戶(hù)端也可以用圖形化的客戶(hù)端如:mysqlworkbench巷疼、 navicat 汽绢、以及一些基于Java的數(shù)據(jù)庫(kù)客戶(hù)端來(lái)訪問(wèn),注意要填寫(xiě)端口號(hào)8066
客戶(hù)端連接之后如果報(bào)如下錯(cuò)誤卷玉,請(qǐng)忽略哨颂,原因mycat與mysql正在連接或者有延遲
3000 - No MyCAT Database selected
1064 - find no Route:SELECT user, host, password, ssl_type, ssl_cipher, x509_issuer, x509_subject, max_questions, max_updates, max_connections, super_priv, max_user_connections FROM mysql.user ORDER BY user
6.2.命令連接
命令行運(yùn)行:mysql -uroot -p123456 -h127.0.0.1 -P8066 -DTESTDB --default_auth=mysql_native_password 就能訪問(wèn)了,以下操作都在此命令行里執(zhí)行(JDBC則將mysql的URL中的端口3306改為8066即可)
mysql -uroot -p123456 -h127.0.0.1 -P8066 -DTESTDB --default_auth=mysql_native_password
說(shuō)明:
--default_auth=mysql_native_password
本人數(shù)據(jù)庫(kù)是mysql8.0.13,由于版本原因相种,不加則報(bào)錯(cuò):ERROR 1045 (HY000): Access denied for user 'root', because password is error威恼;這是因?yàn)榕c5.7的驗(yàn)證插件有關(guān),mysql8默認(rèn)為:caching_sha2_password;5.7默認(rèn)為:mysql_native_password
127.0.0.1 不能替換為localhost
具體原因請(qǐng)查看源碼
注:9066端口是管理端口箫措,相關(guān)命令在連接進(jìn)入之后缭黔,通過(guò)show @@help查看
7. 測(cè)試邏輯庫(kù)、邏輯表蒂破、分片節(jié)點(diǎn)馏谨、分片規(guī)則、節(jié)點(diǎn)主機(jī)
7.1. employee表
Employee表附迷,是根據(jù)規(guī)則sharding-by-intfile (分片字段為sharding_id)進(jìn)行分片惧互。創(chuàng)建employee表:輸入如下SQL
create table employee (id int not null primary key,name varchar(100),sharding_id int not null);
運(yùn)行explain指令,查看該SQL被發(fā)往哪些分片節(jié)點(diǎn)執(zhí)行:
explaincreate table employee (id int not null primary key,name varchar(100),sharding_id int not null);
建議參照schema.xml中employee表的定義喇伯,以及其分片規(guī)則喊儡,來(lái)看看什么數(shù)據(jù)會(huì)出現(xiàn)在dn1上,什么數(shù)據(jù)會(huì)出現(xiàn)在dn2上稻据。
溫馨提示:explain可以用于任何正確的SQL上艾猜,其作用是告訴你,這條SQL會(huì)路由到哪些分片節(jié)點(diǎn)上執(zhí)行捻悯,這對(duì)于診斷分片相關(guān)的問(wèn)題很有幫助匆赃。另外,explain可以安全的執(zhí)行多次今缚,它僅僅是告訴你SQL的路由分片算柳,而不會(huì)執(zhí)行該SQL。
插入數(shù)據(jù):
insert into employee(id,name,sharding_id) values(1,'leader us',10000);
insert into employee(id,name,sharding_id) values(2, 'me',10010);
insert into employee(id,name,sharding_id) values(3, 'mycat',10000);
insert into employee(id,name,sharding_id) values(4, 'mydog',10010);
7.2. company表
company表是根據(jù)規(guī)則auto-sharding-long(主鍵范圍)進(jìn)行分片姓言。
創(chuàng)建company表:輸入如下SQL
create table company(id int not null primary key,name varchar(100));
錄入數(shù)據(jù):
insert into company(id,name) values(1,'hp');
insert into company(id,name) values(2,'ibm');
insert into company(id,name) values(3,'oracle');
你會(huì)看到三個(gè)分片上都插入了3條數(shù)據(jù)瞬项,因?yàn)閏ompany定義為全局表,用explain來(lái)確認(rèn)這個(gè)情況:
explain insert into company(id,name) values(1,'hp')
返回3個(gè)節(jié)點(diǎn)的信息:
| DATA_NODE | SQL |
+-----------+---------------------------------------------+
| dn1 | insert into company(id,name) values(1,'hp') |
| dn2 | insert into company(id,name) values(1,'hp') |
| dn3 | insert into company(id,name) values(1,'hp') |
+-----------+---------------------------------------------+
7.3. customer****表
創(chuàng)建表:
create table customer(id int not null primary key,name varchar(100),company_id int not null,sharding_id int not null);
插入數(shù)據(jù):
insert into customer (id,name,company_id,sharding_id )values(1,'wang',1,10000); //stored in db1;
insert into customer (id,name,company_id,sharding_id )values(2,'xue',2,10010); //stored in db2;
insert into customer (id,name,company_id,sharding_id )values(3,'feng',3,10000); //stored in db1;
查詢(xún)結(jié)果:
Select * from customer;
explain Select * from customer; (確認(rèn)數(shù)據(jù)是分片存儲(chǔ))
7.4. orders****表
創(chuàng)建表格orders何荚,并插入數(shù)據(jù):
create table orders (id int not null primary key ,customer_id int not null,sataus int ,note varchar(100) );
insert into orders(id,customer_id) values(1,1); //stored in db1 because customer table with id=1 stored in db1
insert into orders(id,customer_id) values(2,2); //stored in db2 because customer table with id=1 stored in db2
查詢(xún)囱淋,支持JOIN!
select customer.name ,orders.* from customer ,orders where customer.id=orders.customer_id;
select t1.name ,t2.* from customer t1 ,orders t2where t1.id = t2.customer_id;
selectt1.name ,t2.* from customer t1 join orders t2 on t1.id = t2.customer_id
7.5. travelrecord****表
travelrecord根據(jù)ID主鍵的范圍進(jìn)行分片:
create travelrecord: create table travelrecord (id bigint not null primary key,user_id varchar(100),traveldate DATE, fee decimal,days int);
insert into travelrecord (id,user_id,traveldate,fee,days) values(1,'wang','2014-01-05',510.5,3);這個(gè)ID就存放在分片0上了
explain insert into travelrecord (id,user_id,traveldate,fee,days) values(7000001,'wang','2014-01-05',510.5,3); 這個(gè)ID就存放在分片1上了
7.6. hotnews****表
熱點(diǎn)新聞,用取摸的方式隨機(jī)分配到dn1,dn2,dn3上
create table hotnews(id int not null primary key ,title varchar(400) ,created_time datetime);
插入數(shù)據(jù)
insert into hotnews(id,title,created_time) values(1,'first',now()); 在分片1上
而Id為5餐塘,則到dn3上妥衣,5%3=2 ,即對(duì)應(yīng)dn3的 index
7.7. goods表
全局表是自動(dòng)克隆到所有已定義的數(shù)據(jù)節(jié)點(diǎn)上的,因此可以與其位于分片節(jié)點(diǎn)中同一數(shù)據(jù)節(jié)點(diǎn)中的任何表連接(join)唠倦。
創(chuàng)建表
create table goods(id int not null primary key,name varchar(200),good_type tinyint,good_img_url varchar(200),good_created date,good_desc varchar(500), price double);
插入數(shù)據(jù)
INSERT INTO goods
(id
, name
, good_type
, good_img_url
, good_created
, good_desc
, price
) VALUES (1, 'googsprince01', 1, 'http://www.xx.com/imgs.jpg', '2019-10-22', 'princedesc01', 100.11);
注:測(cè)試參考官方鏈接:https://github.com/MyCATApache/Mycat-Server