大家好禁谦，我是辣條。
現(xiàn)在短視頻可謂是一騎絕塵废封，吃飯的時候州泊、休息的時候、躺在床上都在刷短視頻漂洋，今天給大家?guī)韕ython爬蟲進階 ：美拍視頻地址加密解析遥皂。
短視頻js逆向解析
抓取目標
工具使用
開發(fā)環(huán)境：win10力喷、python3.7
開發(fā)工具：pycharm、Chrome
工具包：requests演训、xpath弟孟、base64
重點學(xué)習內(nèi)容
爬蟲采集數(shù)據(jù)的解析過程
js代碼調(diào)試技巧
js逆向解析代碼
Python代碼的轉(zhuǎn)換
項目思路解析
進入到網(wǎng)站的首頁
挑選你感興趣的分類根據(jù)首頁地址獲取到進入詳情頁面的超鏈接的跳轉(zhuǎn)地址
# eplace()方法用于在字符串中用一些字符替換另一些字符# parseInt 數(shù)據(jù)轉(zhuǎn)換成對應(yīng)的整型# base64.atob 對base64編碼過的字符串進行解碼# substring 方法可在字符串中抽取從 start 下標開始的指定數(shù)目的字符
將js代碼轉(zhuǎn)換成Python代碼
import base64def decode(data):def getHex(a):return {'str': a[4:],'hex': ''.join(list(a[:4])[::-1]),}def getDec(a):b = str(int(a, 16))return {'pre': list(b[:2]),'tail': list(b[2:]),}def substr(a, b):c = a[0: int(b[0])]d = a[int(b[0]): int(b[0]) + int(b[1])]return c + a[int(b[0]):].replace(d, "")def getPos(a, b):b[0] = len(a) - int(b[0]) - int(b[1])return bb = getHex(data)c = getDec(b['hex'])d = substr(b['str'], c['pre'])return base64.b64decode(substr(d, getPos(d, c['tail'])))print(decode("e121Ly9tBrI84RdnZpZGVvMTAubWVpdHVkYXRhLmNvbS82MGJjZDcwNTE3NGZieXBueG5udnRwMTA5N19IMjY0XzFfNWY3YThmM2U0MTEwNy5tc2JVjAu3EDQ="))
簡易源碼分享
import requestsfrom lxml import etreeimport base64def decode_mp4(data):def getHex(a):return {'str': a[4:],'hex': ''.join(list(a[:4])[::-1]),}def getDec(a):b = str(int(a, 16))return {'pre': list(b[:2]),'tail': list(b[2:]),}def substr(a, b):c = a[0: int(b[0])]d = a[int(b[0]): int(b[0]) + int(b[1])]return c + a[int(b[0]):].replace(d, "")def getPos(a, b):b[0] = len(a) - int(b[0]) - int(b[1])return bb = getHex(data)c = getDec(b['hex'])d = substr(b['str'], c['pre'])return base64.b64decode(substr(d, getPos(d, c['tail'])))# 運行主函數(shù)def main():url = 'https://www.meipai.com'headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',}response = requests.get(url=url, headers=headers)html_data = etree.HTML(response.text)href_list = html_data.xpath('//div/a/@href')# print(href_list)for href in href_list:res = requests.get('https://www.meipai.com' + href, headers=headers)html = etree.HTML(res.text)name = html.xpath('//div[@id="detailVideo"]/img/@alt')[0]mp4_data = html.xpath('//div[@id="detailVideo"]/@data-video')[0]# print(name, mp4_data)mp4_url = decode_mp4(mp4_data).decode('utf-8')print(mp4_url)result = requests.get("http:" + mp4_url)with open(name + ".mp4", 'wb') as f:f.write(result.content)f.close()if __name__ == '__main__':main()
