一降允、眾所周知,MySQL在RR隔離級別下萍程,會出現(xiàn)幻讀的問題幢妄。
出現(xiàn)幻讀的前提條件:
?Innodb存儲引擎,在RR隔離級別下茫负,并且使用了當前讀蕉鸳;
出現(xiàn)幻讀的表現(xiàn):
?一個事務在前后兩次查詢同一范圍數(shù)據(jù)的時候(當前讀),后一次查詢看到了前一次查詢沒有看到的行忍法。兩點需要說明:
1 置吓、在可重復讀隔離級別下,普通查詢是快照讀缔赠,不會看到其他事務插入的數(shù)據(jù)衍锚。幻讀只在當前讀才會出現(xiàn)嗤堰;
2 戴质、幻讀專指新插入的行度宦。當前讀的作用就是能讀到其他事務已經(jīng)提交的新插入的記錄。
二告匠、幻讀帶來的影響是會導致主從之間數(shù)據(jù)不一致戈抄,是很嚴重的問題。
三后专、如何解決幻讀的問題:
?產(chǎn)生幻讀的原因是:mysql的record lock(行鎖)只能鎖住行划鸽,但是新插入記錄這個動作,是更新了記錄之間的“間隙”戚哎。因此裸诽,為了解決幻讀,innodb引入了新的鎖型凳,也就是間隙鎖(Gap Lock)丈冬。顧名思義,間隙鎖甘畅,鎖的就是兩個值之間的空隙埂蕊。
四、以上介紹了間隙鎖疏唾,它能幫我們解決了幻讀的問題蓄氧,但同時也會給我們帶來一些“困擾”。接下來結合一個線上的故障case槐脏,講解一下間隙鎖匀们,給我們帶來了什么樣的困擾:
1、故障現(xiàn)象描述:
用戶報障准给,一個insert的SQL語句被堵住了,從DBbrain看到的現(xiàn)象update的SQL持有鎖重抖,導致insert語句無法寫入:
2露氮、分析:
1)、從上圖可以看到钟沛,insert語句處于鎖等待狀態(tài):LOCK WAIT畔规。Dbbrain顯示持有鎖的事務為:“UPDATE t_push_task SET status=9 WHERE push_id=1384715944290652160 AND access_id=1500015064 limit 1” 以及
“UPDATE t_push_task SET status=9 WHERE push_id=1384729752719482880 AND access_id=1500015064 limit 1”
insert語句為:
INSERT INTO t_push_task SET status=0, access_id=1600007315, type=8, push_req='?????(:??[{"Ids":["zhihuishu_class_204117682"],"IdsStru":null,"InnerOperator":1,"OuterOperator":1,"IsNot":false,"TagType":"xg_user_define"}]J?00:4895P?Z?????????\n??{"alert":{"title":"通知","subtitle":"","body":"劉婷班 ......
2)、是否這兩個update語句有性能問題呢恨统?
從explain執(zhí)行計劃分析叁扫,該SQL掃描的行數(shù)只有一行,并且是走主鍵索引掃描畜埋,所以update語句并沒有性能問題莫绣;
3)、那為什么update語句會把insert語句堵住呢悠鞍?
我們知道对室,insert插入記錄,更新的是記錄之間的“間隙”。那么是否有可能是由于間隙鎖的原因掩宜,導致insert無法插入呢蔫骂?
4)、我們來看看表結構牺汤,如下:
CREATE TABLE t_push_task
(
push_id
bigint(20) unsigned NOT NULL AUTO_INCREMENT,
group_id
varchar(256) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
status
int(8) NOT NULL,
access_id
bigint(11) NOT NULL,
type
bigint(20) NOT NULL,
target_list
mediumtext COLLATE utf8mb4_unicode_ci,
push_req
blob,
create_time
datetime DEFAULT CURRENT_TIMESTAMP,
push_time
datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
push_node
varchar(256) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
expire_sec
int(11) NOT NULL DEFAULT '259200',
start_time
datetime DEFAULT NULL,
finish_time
datetime DEFAULT NULL,
source
int(8) NOT NULL,
msg_type
int(8) NOT NULL,
msg_status
int(8) NOT NULL DEFAULT '0',
push_content
mediumtext COLLATE utf8mb4_unicode_ci,
last_modify_time
datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
global_push_type
varchar(32) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
upload_id
bigint(20) unsigned NOT NULL DEFAULT '0',
already_send_num
bigint(20) unsigned NOT NULL DEFAULT '0',
queue_id
varchar(256) COLLATE utf8mb4_unicode_ci DEFAULT '',
collapse_id
int(8) NOT NULL DEFAULT '0' COMMENT '????push_id',
expect_send_num
bigint(20) unsigned DEFAULT '0',
current_index
int(11) DEFAULT '-1' COMMENT '?????????',
PRIMARY KEY (push_id
,push_time
),
KEY idx_status_create
(access_id
,status
,create_time
),
KEY idx_status_push
(access_id
,status
,start_time
),
KEY idx_status_push_queenid
(queue_id
,status
,push_time
),
KEY idx_source_type_content_push
(access_id
,create_time
,source
,msg_type
,push_content
(512)),
KEY idx_push_time
(push_time
),
KEY idx_id_pushtime_type_msgtype_source
(access_id
,push_time
,type
,msg_type
,source
),
KEY idx_id_type_status_pushtime
(access_id
,type
,status
,push_time
),
KEY idx_id_collapseid
(access_id
,collapse_id
),
KEY id_idx_status
(push_id
,access_id
,status
)
) ENGINE=InnoDB AUTO_INCREMENT=500534759 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
/*!50100 PARTITION BY RANGE (to_days(push_time))
(PARTITION p201904 VALUES LESS THAN (737545) ENGINE = InnoDB,
PARTITION p201905 VALUES LESS THAN (737576) ENGINE = InnoDB,
PARTITION p201906 VALUES LESS THAN (737606) ENGINE = InnoDB,
PARTITION p201907 VALUES LESS THAN (737637) ENGINE = InnoDB,
PARTITION p201908 VALUES LESS THAN (737668) ENGINE = InnoDB,
PARTITION p201909 VALUES LESS THAN (737698) ENGINE = InnoDB,
PARTITION p201910 VALUES LESS THAN (737729) ENGINE = InnoDB,
PARTITION p201911 VALUES LESS THAN (737759) ENGINE = InnoDB,
PARTITION p201912 VALUES LESS THAN (737790) ENGINE = InnoDB,
PARTITION p202001 VALUES LESS THAN (737821) ENGINE = InnoDB,
PARTITION p202002 VALUES LESS THAN (737850) ENGINE = InnoDB,
PARTITION p202003 VALUES LESS THAN (737881) ENGINE = InnoDB,
PARTITION p202004 VALUES LESS THAN (737911) ENGINE = InnoDB,
PARTITION p202005 VALUES LESS THAN (737942) ENGINE = InnoDB,
PARTITION p202006 VALUES LESS THAN (737972) ENGINE = InnoDB,
PARTITION p202007 VALUES LESS THAN (738003) ENGINE = InnoDB,
PARTITION p202008 VALUES LESS THAN (738034) ENGINE = InnoDB,
PARTITION p202009 VALUES LESS THAN (738064) ENGINE = InnoDB,
PARTITION p202010 VALUES LESS THAN (738095) ENGINE = InnoDB,
PARTITION p202011 VALUES LESS THAN (738125) ENGINE = InnoDB,
PARTITION p202012 VALUES LESS THAN (738156) ENGINE = InnoDB,
PARTITION p202101 VALUES LESS THAN (738187) ENGINE = InnoDB,
PARTITION p202102 VALUES LESS THAN (738215) ENGINE = InnoDB,
PARTITION p202103 VALUES LESS THAN (738246) ENGINE = InnoDB,
PARTITION p202104 VALUES LESS THAN (738276) ENGINE = InnoDB,
PARTITION p202105 VALUES LESS THAN (738307) ENGINE = InnoDB,
PARTITION p202106 VALUES LESS THAN (738337) ENGINE = InnoDB,
PARTITION p202107 VALUES LESS THAN (738368) ENGINE = InnoDB,
PARTITION p202108 VALUES LESS THAN (738399) ENGINE = InnoDB,
PARTITION p202109 VALUES LESS THAN (738429) ENGINE = InnoDB,
PARTITION p202110 VALUES LESS THAN (738460) ENGINE = InnoDB,
PARTITION p202111 VALUES LESS THAN (738490) ENGINE = InnoDB,
PARTITION p202112 VALUES LESS THAN (738521) ENGINE = InnoDB,
PARTITION p2022 VALUES LESS THAN (741443) ENGINE = InnoDB) */
從表結構可以知道辽旋,該表主鍵為:PRIMARY KEY (push_id
,push_time
)。而update語句的where條件為:push_id=1384729752719482880 AND access_id=1500015064檐迟,故update語句走的是主鍵索引的前綴索引补胚,長度為8個字節(jié)(push_id類型為bigint,長度為8個字節(jié)锅减;push_time類型為datetime糖儡,長度也是8個字節(jié));
由于push_id是自增字段auto_increment怔匣,故insert語句插入的時候握联,push_id會進行自增,會在當前最大max(push_id)的基礎上每瞒,自增加1金闽;
5)、用戶提供的線索:
用戶反饋剿骨,由于業(yè)務邏輯設計不合理的原因代芜,其update語句中的push_id實際上在表中是并不存在的,并且遠遠大于當前表中的最大push_id,當前表中最大push_id為:
這就很奇怪了浓利,update語句where 條件的push_id=1384715944290652160在表當中根本就不存在挤庇,也就是說該update語句實際上沒生效。update語句和insert語句風馬牛不相及贷掖,為啥update會阻塞insert呢嫡秕??苹威?
6)昆咽、測試實例實驗一把,看看是否會出現(xiàn)同樣的情況牙甫?
a掷酗、創(chuàng)建表、寫入記錄
MySQL [(none)]> use test;
MySQL [test]> CREATE TABLE t
( id
int(11) NOT NULL, c
int(11) DEFAULT NULL, d
int(11) DEFAULT NULL, PRIMARY KEY (id
), KEY c
(c
)) ENGINE=InnoDB;
Query OK, 0 rows affected (0.00 sec)
MySQL [test]> show create table t\G;
*************************** 1. row ***************************
Table: t
Create Table: CREATE TABLE t
(
id
int(11) NOT NULL,
c
int(11) DEFAULT NULL,
d
int(11) DEFAULT NULL,
PRIMARY KEY (id
),
KEY c
(c
)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
insert into t values(0,0,0),(5,5,5),(10,10,10),(15,15,15),(20,20,20),(25,25,25);
MySQL [test]> select * from t;
+----+------+------+
| id | c | d |
+----+------+------+
| 0 | 0 | 0 |
| 5 | 5 | 5 |
| 10 | 10 | 10 |
| 15 | 15 | 15 |
| 20 | 20 | 20 |
| 25 | 25 | 25 |
+----+------+------+
6 rows in set (0.00 sec)
b窟哺、開啟兩個會話泻轰,分別是session A、session B
session A:
MySQL [test]> begin;
Query OK, 0 rows affected (0.00 sec)
MySQL [test]> update t set d=99 where id=100;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
session A更新了表中不存在的id且轨,并且比當前max(id)還要大糕殉。此時session A的事務還未提交亩鬼;
session B:
MySQL [(none)]> use test;
MySQL [test]> insert into t values(26,26,26);
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
session B想要插入比當前比當前max(id)=25要大的一條記錄26,結果發(fā)現(xiàn)被堵住了阿蝶,產(chǎn)生了鎖等待雳锋;
實驗結果現(xiàn)象和上述用戶的故障case是一樣的,為什么會出現(xiàn)這種情況呢羡洁?
7)玷过、next-key lock(record lock + gap lock)加鎖原則:
原則 1:加鎖的基本單位是 next-key lock。next-key lock 是前開后閉區(qū)間筑煮;
原則 2:查找過程中訪問到的對象才會加鎖辛蚊;
優(yōu)化 1:索引上的等值查詢,給唯一索引加鎖的時候真仲,next-key lock 退化為行鎖袋马;
優(yōu)化 2:索引上的等值查詢,向右遍歷時且最后一個值不滿足等值條件的時候秸应,next-key lock 退化為間隙鎖虑凛;
一個 bug:唯一索引上的范圍查詢會訪問到不滿足條件的第一個值為止;
由于用戶的表當中并沒有push_id=1384715944290652160或者push_id=1384729752719482880软啼,用上述加鎖規(guī)則來判斷:
a桑谍、加鎖的單位是next-key lock,故“UPDATE t_push_task SET status=9 WHERE push_id=1384715944290652160 AND access_id=1500015064 limit 1”的加鎖范圍為:
(max(push_id),+∞]祸挪;
b锣披、根據(jù)優(yōu)化2原則:update語句是一個等值查詢(push_id=1384715944290652160),向右遍歷比1384715944290652160大的是+∞,而+∞不滿足查詢條件贿条,故next-key lock退化成間隙鎖雹仿,鎖住的范圍為(max(push_id),+∞);
c整以、由于insert語句寫入胧辽,push_id會自增,比當前最大的max(push_id)自增加1悄蕾,所有落入了加鎖范圍(max(push_id),+∞)內(nèi),導致用戶的insert語句被阻塞础浮。
d帆调、即:如果索引沒有命中,會有間隙鎖豆同, 向左掃描掃到第一個比給定參數(shù)小的值番刊, 向右掃描掃描到第一個比給定參數(shù)大的值, 然后以此為界影锈,構建一個區(qū)間芹务, 鎖住整個區(qū)間內(nèi)的數(shù)據(jù)
由此就能解釋蝉绷,為什么insert會被update阻塞了。將原因同步給用戶之后枣抱,用戶承諾會優(yōu)化業(yè)務代碼熔吗,規(guī)避間隙鎖的問題。
8)佳晶、思考:
a桅狠、上述case是更新了一個比表中max(push_id)還要大的push_id,那如果更新的push_id剛好是表中存在的一條記錄轿秧,比如更新的是當前的max(push_id)=500536561中跌,那么會如何加鎖呢?
根據(jù)加鎖原則優(yōu)化1:索引上的等值查詢菇篡,給唯一索引加鎖的時候漩符,next-key lock 退化為行鎖。此時只會對push_id=500536561加鎖驱还;
b嗜暴、如果更新的push_id在表中不存在,但是比max(push_id)要小铝侵,例如更新的push_id=447689301灼伤,介于447689203 和 448344009之間。此時會如何加鎖呢咪鲜?
根據(jù)原則1狐赡,加鎖單位是next-key lock,故加鎖的范圍是(447689203,448344009]疟丙;
根據(jù)優(yōu)化2颖侄,這是一個等值查詢,push_id=447689301享郊,向右遍歷且最后一個值不滿足等值條件的時候览祖,next-key lock 退化為間隙鎖,故最終加鎖的范圍為(447689203,448344009)炊琉。