問(wèn)題
在基于 Redis 的優(yōu)先級(jí)隊(duì)列這篇文章中诱鞠，我們使用 sorted set 實(shí)現(xiàn)了一個(gè)優(yōu)先級(jí)隊(duì)列】嫱冢現(xiàn)在，考慮如下問(wèn)題：
在一個(gè)任務(wù)調(diào)度系統(tǒng)中般甲，任務(wù)進(jìn)入隊(duì)列排隊(duì)肋乍，然后等待被調(diào)度出隊(duì)列鹅颊。為了衡量調(diào)度效率敷存，一個(gè)指標(biāo)是任務(wù)的平均排隊(duì)時(shí)間。那么堪伍，如何統(tǒng)計(jì)每個(gè)任務(wù)的排隊(duì)時(shí)間以及這些任務(wù)的平均排隊(duì)時(shí)間呢锚烦？
解決方案
為了統(tǒng)計(jì)每個(gè)任務(wù)的排隊(duì)時(shí)間，我們需要知道任務(wù)的入隊(duì)時(shí)間以及出隊(duì)時(shí)間帝雇，這兩個(gè)時(shí)間的差值就是這個(gè)任務(wù)的排隊(duì)時(shí)間涮俄。我們?nèi)匀皇褂?sorted set 來(lái)記錄任務(wù)的入隊(duì)或出隊(duì)時(shí)間，只不過(guò)這個(gè)時(shí)候 sorted set 中元素的 score 不再是優(yōu)先級(jí)尸闸，而是任務(wù)的入隊(duì)或出隊(duì)時(shí)間彻亲。
代碼實(shí)現(xiàn)
下面是一個(gè)可能的代碼實(shí)現(xiàn)：
class QueuingTimeStat(object):
def __init__(self, redis_client, redis_key, namespace='schedule'):
self.client = redis_client
self.key = '%s:%s' % (namespace, redis_key)
def _clean(self):
"""清理一天之前的記錄"""
now = time.time()
self.client.zremrangebyscore(self.key, 0, now - 3600 * 24)
def add_start(self, task_id, timestamp=None):
self._clean()
member = '%s:%s' % (task_id, 'start')
timestamp = time.time() if timestamp is None else timestamp
self.client.zadd(self.key, timestamp, member)
def add_end(self, task_id, timestamp=None):
self._clean()
member = '%s:%s' % (task_id, 'end')
timestamp = time.time() if timestamp is None else timestamp
self.client.zadd(self.key, timestamp, member)
def average_delay(self, window):
now = time.time()
results = self.client.zrangebyscore(self.key, now - window, now, withscores=True)
# [(value, score), ...]
sorted_results = sorted(results, key=lambda x: x[0])
group_results = groupby(sorted_results, key=lambda x: x[0].rsplit(':', 1)[0])
total = 0.0
count = 0
for task_id, sub_group in group_results:
pairs = tuple(sub_group)
if len(pairs) == 2:
count += 1
total += abs(pairs[0][1] - pairs[1][1])
else:
if pairs[0][0].endswith('start'):
count += 1
total += now - pairs[0][1]
return total / count if count > 0 else 0
在上面的代碼中，task_id 是任務(wù)的唯一標(biāo)識(shí)吮廉。task_id:start 用于記錄任務(wù)的入隊(duì)時(shí)間苞尝，task_id:end 用于記錄任務(wù)的出隊(duì)時(shí)間。average_delay 用于統(tǒng)計(jì)最近任務(wù)的平均排隊(duì)時(shí)間宦芦，該方法提供了時(shí)間窗口 window 參數(shù)宙址，時(shí)間窗口越大，則平均排隊(duì)時(shí)間變化曲線越平滑调卑。另外抡砂，為了降低對(duì)內(nèi)存的消耗，每次記錄入隊(duì)或出隊(duì)時(shí)間的時(shí)候恬涧，會(huì)清理一天之前的入隊(duì)出隊(duì)時(shí)間記錄注益。
