JSON格式的數(shù)據(jù)在http接口的自動(dòng)化測(cè)試中是一種常見(jiàn)的數(shù)據(jù)結(jié)構(gòu)柬焕,在接口自動(dòng)化腳本里解析JSON返回結(jié)果,驗(yàn)證數(shù)據(jù)的正確性是非常關(guān)鍵的一步,但JSON結(jié)構(gòu)有簡(jiǎn)單有復(fù)雜册烈,不同的接口返回不同的結(jié)果倘屹,自動(dòng)化腳本中解析響應(yīng)JSON數(shù)據(jù)占了一大部分的工作量候址,而且隨著接口的變動(dòng),維護(hù)腳本也特別麻煩族扰,今天在這里介紹一種通用的解析所有JSON 的通用算法厌丑,來(lái)解決這一問(wèn)題定欧。
算法解析:
首先來(lái)看一段較為復(fù)雜的JSON:
從圖-1可以看出,這段JSON第一層包含了resultCode怒竿,resultDesc砍鸠,data,dataList四個(gè)key值耕驰,其中resultCode爷辱,resultDesc,data 都是單個(gè)的字符串朦肘,而dataList 的值是一個(gè)列表饭弓,有倆元素,其中每個(gè)元素又是一個(gè)字典媒抠,包括id,name,caption,subjectId,notes,items,其中items中又是一個(gè)list,每個(gè)元素為一個(gè)字典弟断,key值為 id,name,caption等....不難發(fā)現(xiàn)這段JSON中第二層和第三層中都包含相同的key: id,name, capiton,這也是選擇這段JSON的原因趴生。大多數(shù)的JSON中可能都會(huì)有相同的key,解析的時(shí)候如何精確的獲取想要的key對(duì)應(yīng)的值這也是需要考慮的場(chǎng)景之一夫嗓。
其實(shí)JSON的整個(gè)結(jié)構(gòu)就是 “樹(shù)”,可以將上述JSON中的 第一個(gè)節(jié)點(diǎn)[JSON]看成樹(shù)的根節(jié)點(diǎn)冲秽, resultCode舍咖,resultDesc 等節(jié)點(diǎn)看成葉子節(jié)點(diǎn),而dataList 則可以看成父節(jié)點(diǎn)锉桑,dataList里的元素可以看成是子節(jié)點(diǎn)排霉。
來(lái)一張更容易理解的圖:A 為根節(jié)點(diǎn),黃色的節(jié)點(diǎn)稱是父節(jié)點(diǎn)民轴,綠色節(jié)點(diǎn)則是葉子節(jié)點(diǎn)攻柠,圖中的D 類比為圖-1中的dataList節(jié)點(diǎn),B,C為圖一種的resultCode后裸,resultDesc
解析JSON主要分為以下幾步
第一步:
- 將JSON 轉(zhuǎn)化為樹(shù)瑰钮,從根節(jié)點(diǎn)(A)開(kāi)始遍歷每一路徑,即從根節(jié)點(diǎn)到葉子節(jié)點(diǎn)每一條路徑作為一條數(shù)據(jù)微驶,并且獲取每一深度(JSON中的每一層)葉子節(jié)點(diǎn)的{key:value}浪谴,其中父節(jié)點(diǎn)的值不取(通常父節(jié)點(diǎn)可能是一個(gè)list,或者一個(gè)字典),當(dāng)前深度的葉子節(jié)點(diǎn)的值全取因苹,與下一深度的葉子節(jié)點(diǎn)的值加到同一字典中苟耻。
- 如圖-2,B,C,D,E是深度為2的節(jié)點(diǎn)扶檐,第一次遍歷凶杖,第一條數(shù)據(jù)為{B,C},以此遞歸,直至遍歷所有路徑款筑,可到圖-2的遍歷結(jié)果為(其中B節(jié)點(diǎn)理解為 {'B':'B'},圖-3 為圖-2的JSON格式)
按第一步解析結(jié)果如下:
1. [{'B': 'B'}, {'C': 'C'}]
2. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}]
3. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]
4. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}]
5. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}]
6. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}, {'T': 'T'}]
7. [{'B': 'B'}, {'C': 'C'}, {'U': 'U'}]
分析上述結(jié)果可以看出 第3組結(jié)果包含了第1,2組值智蝠,6包含了5,4的結(jié)果腾么,因此可以將重復(fù)的數(shù)據(jù)進(jìn)行一次過(guò)濾和篩選
第二步:
過(guò)濾重復(fù)數(shù)據(jù),步驟一的結(jié)果可以過(guò)濾為:
3. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]
6.[{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}, {'T': 'T'}]
7. [{'B': 'B'}, {'C': 'C'}, {'U': 'U'}]
在這一步驟中杈湾,如果不同深度的葉子節(jié)點(diǎn)名稱重復(fù)解虱,則按照深度,依次在key后面加入序號(hào)1,2...
即如果有3個(gè)B,則解析結(jié)果為:
[{'B': 'B'}, {'B1': 'B1'}, {'B2: 'B2'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]
第三步:
根據(jù)想要獲取的KEY,在第二步獲取到的list中取出對(duì)應(yīng)的value毛秘,KEY以列表的方式給出
keys = [key1,key2,key3....]饭寺,如要獲取[B,C,F,G]
則最終結(jié)果為:
[{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}]
這樣就輕而易舉的解析處理我們想要的結(jié)果!
代碼 -python算法實(shí)現(xiàn)
def responseComplexJson2LD(responseJson,assertKeys):
'''
:param responseJson: 類型:JSONObject 接口返回:JSON 格式
:param assertKeys: 類型:list 需要校驗(yàn)的key ['resultCode','resultDesc','id','name','caption','id1','notes','name1','caption1','cubeName','functions']
說(shuō)明:如果key里有重復(fù)的解析為 key1,key2...
:return: list[dict]
'''
try:
if not isinstance(responseJson,dict):
raise Exception('responseJson 類型錯(cuò)誤,必須為JSON格式')
result_list = []
def isInclude(dict1, dict2):
'''
:function: 判斷 dict1 是否包含于dict2
:param dict1:
:param dict2:
:return: True False
'''
for key1 in dict1:
if key1 in dict2:
pass
else:
return False
return True
def parse_dict(responseJson, parent):
'''
:遞歸解析json,將json樹(shù)解析成list[dict,dict....]
:param responseJson:
:param parent:
:return:
'''
if isinstance(responseJson, dict):
dp = parent[:]
not_l_d = [responseKey for responseKey in responseJson if not isinstance(responseJson[responseKey], (dict, list))]
for i in not_l_d:
dp.append({i: responseJson[i]})
dp_1 = dp[:]
result_list.append(dp)
for responseKey in responseJson:
if isinstance(responseJson[responseKey], (dict, list)):
parse_dict(responseJson[responseKey], dp_1)
elif isinstance(responseJson, list):
for i in responseJson:
dd = parent[:]
if not isinstance(i, (dict, list)):
dd.append(i)
result_list.append(dd)
else:
parse_dict(i, dd)
else:
dx = parent[:]
dx.append(responseJson)
result_list.append(dx)
parse_dict(responseJson, parent=[])
if not result_list:
return []
'''獲取給定的assertKeys'''
last_result =[]
templist ={}
if assertKeys:
if isinstance(assertKeys,list):
for iter in result_list:
for key in iter:
thiskey = list(key.keys())[0]
'''如果有重復(fù)的key,解析為key1,key2...'''
i = 1
temp = thiskey
while True:
if thiskey in list(templist.keys()):
thiskey = thiskey + str(i)
i = i + 1
else:
templist.setdefault(thiskey,key.get(temp))
break
last_result.append(templist)
templist={}
else:
raise Exception("assertKeys 類型錯(cuò)誤叫挟,必須為list")
'''將assertKeys里的定義的key加入dict,不在assertKeys 定義的除去'''
assert_result = []
templist = {}
for meta in last_result:
keys = list(meta.keys())
if isInclude(assertKeys,keys):
for key in assertKeys:
templist.setdefault(key,meta.get(key))
assert_result.append(templist)
templist={}
else:
pass
'''對(duì)返回的last_result中的元素去重'''
result = []
for meta in assert_result:
if meta not in result:
result.append(meta)
else:
pass
return result
except Exception as err:
Log.error(err)
return []
結(jié)果驗(yàn)證
以圖-1中的json 為例測(cè)試
測(cè)試1:
keylist = ['resultCode']
responseComplexJson2LD(test,keylist)
返回:
{'resultCode': 100}
測(cè)試2:
keylist = ['resultCode','resultDesc']
responseComplexJson2LD(test,keylist)
返回:
{'resultCode': 100, 'resultDesc': '成功'}
測(cè)試3:
keylist = ['resultCode','resultDesc','id']
responseComplexJson2LD(test,keylist)
返回:
{'resultCode': 100, 'resultDesc': '成功', 'id': 1}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2}
測(cè)試4:--重復(fù)id,第二用id1標(biāo)識(shí)
keylist = ['resultCode', 'resultDesc', 'id','id1']
responseComplexJson2LD(test,keylist)
返回:
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 1}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 2}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 20}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 3}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 4}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 5}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 6}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 21}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 22}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 23}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 7}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 8}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 9}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 10}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 24}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 25}
測(cè)試5:取第二個(gè)id
keylist = ['resultCode', 'resultDesc','id1']
responseComplexJson2LD(test,keylist)
返回:
{'resultCode': 100, 'resultDesc': '成功', 'id1': 1}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 2}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 20}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 3}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 4}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 5}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 6}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 21}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 22}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 23}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 7}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 8}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 9}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 10}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 24}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 25}
測(cè)試6:只取caption 字段艰匙,倆caption 時(shí)第二個(gè)寫caption1
keylist = ['caption','caption1]
responseComplexJson2LD(test,keylist)
返回:
{'caption': '交易', 'caption1': '交易金額'}
{'caption': '交易', 'caption1': '交易用戶數(shù)'}
{'caption': '交易', 'caption1': '人均交易金額'}
{'caption': '交易', 'caption1': '訂單數(shù)'}
{'caption': '交易', 'caption1': '訂單金額'}
{'caption': '交易', 'caption1': '大訂單數(shù)'}
{'caption': '交易', 'caption1': '大訂單金額'}
{'caption': '交易', 'caption1': '平均訂單金額'}
{'caption': '交易', 'caption1': '大訂單數(shù)占比'}
{'caption': '交易', 'caption1': '平均大訂單金額'}
{'caption': '盈虧', 'caption1': '盈利用戶數(shù)'}
{'caption': '盈虧', 'caption1': '虧損用戶數(shù)'}
{'caption': '盈虧', 'caption1': '凈盈利值'}
{'caption': '盈虧', 'caption1': '凈虧損值'}
{'caption': '盈虧', 'caption1': '盈利用戶數(shù)占比'}
{'caption': '盈虧', 'caption1': '虧損用戶數(shù)占比'}
附上圖-1完整的Json報(bào)文:
data = {
"resultCode": 100,
"resultDesc": "成功",
"data": "",
"dataList": [
{
"id": 1,
"subjectId": "",
"name": "trading",
"caption": "交易",
"notes": "1.人均交易金額=用戶累積交易金額/交易用戶數(shù)<br>2.平均訂單金額=用戶累計(jì)訂單金額/訂單數(shù)<br>3.大訂單:?jiǎn)喂P手續(xù)費(fèi)大于50元的訂單(手續(xù)費(fèi)=每筆訂單金額*萬(wàn)分之八)<br>4.大訂單數(shù)占比=大訂單數(shù)/訂單數(shù)<br>5.平均大訂單金額=累計(jì)大訂單金額/大訂單數(shù)<br>",
"items": [
{
"id": 1,
"name": "pay_money",
"caption": "交易金額",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY",
"itemType": "SINGLE",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 2,
"name": "pay_user_num",
"caption": "交易用戶數(shù)",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "ACCOUNT_ID",
"itemType": "SINGLE",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT_DISTINCT"
},
{
"id": 20,
"name": "pay_money,pay_user_num",
"caption": "人均交易金額",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY,ACCOUNT_ID",
"itemType": "COMBINE",
"calType": "/",
"valCalType": "MONEY",
"functions": "SUM,COUNT_DISTINCT"
},
{
"id": 3,
"name": "order_num",
"caption": "訂單數(shù)",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "*",
"itemType": "SINGLE",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT"
},
{
"id": 4,
"name": "order_money",
"caption": "訂單金額",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY",
"itemType": "SINGLE",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 5,
"name": "big_order_num",
"caption": "大訂單數(shù)",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "*",
"itemType": "CONDITION",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT"
},
{
"id": 6,
"name": "big_order_money",
"caption": "大訂單金額",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY",
"itemType": "CONDITION",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 21,
"name": "order_money,order_num",
"caption": "平均訂單金額",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY,*",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "MONEY",
"functions": "SUM,COUNT"
},
{
"id": 22,
"name": "big_order_num,order_num",
"caption": "大訂單數(shù)占比",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "*,*",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "PERCENT",
"functions": "COUNT,COUNT"
},
{
"id": 23,
"name": "big_order_money,big_order_num",
"caption": "平均大訂單金額",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY,*",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "MONEY",
"functions": "SUM,COUNT"
}
]
},
{
"id": 2,
"subjectId": "",
"name": "profit&loss",
"caption": "盈虧",
"notes": "<br>1.盈利用戶數(shù):當(dāng)日用戶中,凈利潤(rùn)>0的用戶數(shù)<br>2.虧損用戶數(shù):當(dāng)日用戶中抹恳,凈利潤(rùn)<0的用戶數(shù)<br>3.盈利用戶占比=當(dāng)日盈利用戶數(shù)/當(dāng)日總用戶數(shù)<br>4.虧損用戶占比=當(dāng)日虧損用戶數(shù)/當(dāng)日總用戶數(shù)<br>5.凈盈利值:當(dāng)日用戶中员凝,凈利潤(rùn)>0的值求和<br>6.凈虧損值:當(dāng)日用戶中,凈利潤(rùn)<0的值求和<br>7.日凈利潤(rùn)=當(dāng)日凈資產(chǎn)-前一日凈資產(chǎn)-當(dāng)日入金+當(dāng)日出金<br>8.月凈利潤(rùn)=月末凈資產(chǎn)-月初凈資產(chǎn)-月內(nèi)總?cè)虢?月內(nèi)總出金<br>其他時(shí)間粒度計(jì)算方法類似月粒度<br>",
"items": [
{
"id": 7,
"name": "profit_user_count",
"caption": "盈利用戶數(shù)",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "ACCOUNT_ID",
"itemType": "CONDITION",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT_DISTINCT"
},
{
"id": 8,
"name": "loss_user_count",
"caption": "虧損用戶數(shù)",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "ACCOUNT_ID",
"itemType": "CONDITION",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT_DISTINCT"
},
{
"id": 9,
"name": "net_profit_num",
"caption": "凈盈利值",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "NETPROFIT",
"itemType": "CONDITION",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 10,
"name": "net_loss_num",
"caption": "凈虧損值",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "NETPROFIT",
"itemType": "CONDITION",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 24,
"name": "profit_user_count,profit_total_user_count",
"caption": "盈利用戶數(shù)占比",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "ACCOUNT_ID,ACCOUNT_ID",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "PERCENT",
"functions": "COUNT_DISTINCT,COUNT_DISTINCT"
},
{
"id": 25,
"name": "loss_user_count,profit_total_user_count",
"caption": "虧損用戶數(shù)占比",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "ACCOUNT_ID,ACCOUNT_ID",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "PERCENT",
"functions": "COUNT_DISTINCT,COUNT_DISTINCT"
}
]
}
]
}