一:往數(shù)據(jù)庫里插入數(shù)據(jù)
先生成一些簡單的數(shù)據(jù)
from pymongoimport MongoClient
from randomimport randint
import datetime
client = MongoClient('localhost',27017)
db = client.get_database('229_taobao')
order = db.order_info
status = ['A','B','C']
cust_id = ['A123','B123','C123']
price = [500,200,250,300]
sku = ['mmm','nnn']
for iin range(1,100):
????items = []
????item_count =randint(2,6)
????for nin range(item_count):
????????items.append({"sku":sku[randint(0,1)],"qty":randint(1,10),"price":randint(0,5)})
????new = {
????"status":status[randint(0,2)],
????"cust_id":cust_id[randint(0,2)],
????"price":price[randint(0,3)],
????"ord_date":datetime.datetime.utcnow(),
????"items":items
}
print new
order.insert_one(new)
print i
print order.estimated_document_count()
二:進入數(shù)據(jù)庫進行mapreduce操作
? ?查看數(shù)據(jù)類型
(1) 查詢每個cust_id 的所有price總和
map函數(shù)
var mapFunction1 = function() {?
? ? ? ? ? ? ? ?emit(this.cust_id, this.price);
? ? ? ? ? ? ? };
emit函數(shù)是把數(shù)據(jù)按照cust_id和price進行分組错英,可以將參數(shù)傳給reduce函數(shù)
reduce函數(shù)
var reduceFunction1 = function(keyCustId, valuesPrices) {
? ? ? ? ? ? return Array.sum(valuesPrices);
? ? ? };
執(zhí)行mapreduce函數(shù)并把結(jié)果輸入map_reduce_example中
db.order_info.mapReduce( mapFunction1, reduceFunction1, { out: "map_reduce_example" } )
查詢輸出的結(jié)果
(2)計算所有items 的平均庫存
? map函數(shù)
var mapFunction2 = function() {?
for (var idx = 0; idx < this.items.length; idx++) {
? ? ? ? ? ? ?var key = this.items[idx].sku;?
? ? ? ? ? ? ?var value = { count: 1, qty: this.items[idx].qty };?
? ? ? ? ? ? emit(key, value);
?}
?};
reduce函數(shù)
var reduceFunction2 = function(keySKU, countObjVals) {?
????reducedVal = { count: 0, qty: 0 };?
????for (var idx = 0; idx < countObjVals.length; idx++) {?
????????reducedVal.count += countObjVals[idx].count;?
????????reducedVal.qty += countObjVals[idx].qty;
?}?
????return reducedVal;?
};
finalize函數(shù)
var finalizeFunction2 = function (key, reducedVal) {?
????reducedVal.avg = reducedVal.qty/reducedVal.count;?
????return reducedVal;
?};
執(zhí)行mapreduce函數(shù)并把結(jié)果輸入map_reduce_example中
db.order_info.mapReduce( mapFunction2, reduceFunction2, { out: { merge: "map_reduce_example" }, finalize: finalizeFunction2 } )
查詢輸出結(jié)果
通過此實驗可以看出太示,mapreduce比aggregate會更靈活