--- 2017.11.28
動機(jī)
有一批相似的函數(shù),代碼邏輯是重復(fù)的,只有幾個參數(shù)是不同的凭语;比如在bigflow里輸出counter,counter.increase("my_counter_a") 需要定義成函數(shù):
def my_counter_a(r):
counter.increase("my_counter_a")
return r
my_pcollection.map(my_counter_a)
當(dāng)要輸出很多counter時薄啥,就很煩了辕羽。
解法
可以定義一個Callable對象,在運(yùn)行時生成函數(shù)對象(其實(shí)類似C++里的仿函數(shù)的用法):
from collections import Callable
from bigflow import counter
class Counter(Callable):
def __init__(self, counter_name):
self.counter_name = counter_name
def __call__(self, record):
counter.increase(self.counter_name)
return record
my_pcollection.map(Counter("my_counter_a"))
collections.Callable 的實(shí)現(xiàn)在 lib/python2.7/_abcoll.py, 使用了 metaclass垄惧,后續(xù)研究一下再補(bǔ)充刁愿。
---2017.11.29
今天發(fā)現(xiàn)Callable對象跟function還是不一樣,不能用在pcollection.map()里到逊;
又想了另一個方法:
def _do_counter(record, counter_name):
counter.increase(counter_name)
return record