業(yè)務(wù)節(jié)點(diǎn)將緩存 or 持久化數(shù)據(jù)寫入Redis之后，在多數(shù)DBA or DBA DevOps的工作場景中骂删，不可避免的要涉及Redis緩存分析，其主要包含大Key和熱key的分析。
基于redis-cli的bigkeys分析
redis-cli -h ${HowUger_redis_addr} -p ${HowUger_redis_port} -a ${HowUger_redis_auth} --bigkeys
# Scanning the entire keyspace to find biggest keys as well as
# average sizes per key type. You can use -i 0.1 to sleep 0.1 sec
# per 100 SCAN commands (not usually needed).
[00.00%] Biggest zset found so far 'testzset' with 129 members
[00.00%] Biggest hash found so far 'h2' with 513 fields
[00.00%] Biggest set found so far 'si1' with 5 members
[00.00%] Biggest hash found so far 'h4' with 514 fields
[00.00%] Biggest string found so far 'key' with 9 bytes
-------- summary -------
Sampled 9 keys in the keyspace!
Total key length in bytes is 27 (avg len 3.00)
Biggest string found 'key' has 9 bytes
Biggest set found 'si1' has 5 members
Biggest hash found 'h4' has 514 fields
Biggest zset found 'testzset' has 129 members
1 strings with 9 bytes (11.11% of keys, avg size 9.00)
0 lists with 0 items (00.00% of keys, avg size 0.00)
2 sets with 8 members (22.22% of keys, avg size 4.00)
4 hashs with 1541 fields (44.44% of keys, avg size 385.25)
2 zsets with 132 members (22.22% of keys, avg size 66.00)
0 streams with 0 entries (00.00% of keys, avg size 0.00)
簡單總結(jié)下：
- Bigkeys實(shí)際是使用
scan方式對Redis中所有key進(jìn)行掃描統(tǒng)計(jì)魔市，因此無需擔(dān)心其對Redis造成阻塞休讳。 - Bigkeys主要應(yīng)用于統(tǒng)計(jì)以
string為主要存儲類型的key分析場景讲婚。 - Bigkeys對于list,set,zset和hash等其他Redis數(shù)據(jù)類型的統(tǒng)計(jì)是以元素個(gè)數(shù)作為衡量標(biāo)準(zhǔn)的，可以說對于非string類型key的統(tǒng)計(jì)bigkeys的結(jié)論是模糊且沒有意義的俊柔。
基于debug object key的序列化分析
HowUger_redis:orz> hmset myhash k1 v1 k2 v2 k3 v3
OK
HowUger_redis:orz> debug object myhash
Value at:0x7f005c6920a0 refcount:1 encoding:ziplist serializedlength:36 lru:3341677 lru_seconds_idle:2
debug object key基本輸出說明如下：
- Value at：key的內(nèi)存地址
- refcount：引用次數(shù)
- encoding：編碼類型
- serializedlength：序列化長度
- lru_seconds_idle：空閑時(shí)間
官方不建議在客戶端使用debug object key命令筹麸，原話是這樣?jì)饍旱危?/p>
DEBUG OBJECT is a debugging command that should not be used by clients.
說幾個(gè)問題：
- serializedlength是key序列化后的長度活合，并不是key在內(nèi)存中的真正長度。就像一個(gè)數(shù)組在json_encode后的長度與其在內(nèi)存中的真正長度并不相同物赶。
- serializedlength會對string做一些可能的壓縮白指。如果有些string的壓縮比特別高，那么在比較時(shí)就會出事情
- 計(jì)算serializedlength的代價(jià)相對高酵紫，少嘗試告嘲，慎用別開玩笑。（多數(shù)云供應(yīng)商的DCS服務(wù)會直接禁用客戶端調(diào)用debug object命令哦）
基于rdbtools的rdb文件分析
rdb -c memory /tmp/HowUger_redis_dump.rdb
database,type,key,size_in_bytes,encoding,num_elements,len_largest_element,expiry
0,hash,data:index_flow_yingshi,10492,hashtable,1,8992,2019-01-14T08:20:10.236000
0,hash,data:index_movie,22068,hashtable,7,2896,2019-01-14T07:29:19.685000
0,string,block:index_module_novel,8296,string,7694,7694,2019-01-13T00:27:46.128000
0,string,block:index_bottom_baike_aikan,8296,string,7632,7632,2019-01-14T02:27:11.850000
0,string,block:index_bottom_tools,5224,string,4549,4549,2019-01-13T01:02:09.171000
0,string,block:index_module_travel,7272,string,6408,6408,2019-01-13T00:43:39.478000
...
總結(jié)說明：
- python2.4以上版本
- git倉
- 基于離線數(shù)據(jù)rdb文件
- 包含但不僅限于rdb奖地，還有redis-memory-for-key橄唬，RdbParser等待
- 加分依賴包python-lzf
- rdb文件分析、生成json或csv格式報(bào)告参歹、單key內(nèi)存分析仰楚、rdb解析器等等，更多具體功能看git吧犬庇。
- 通過分析rdb文件中的key及value僧界，反算出該kv在內(nèi)存中的大小。計(jì)算時(shí)考慮了數(shù)據(jù)類型的影響臭挽，key本身長度的影響捂襟，內(nèi)存分配等多種因素。雖然得出的大小不是真實(shí)值埋哟，但用于key大小的統(tǒng)計(jì)已經(jīng)是完全足夠了笆豁。
