在說這個m3u8下載庫之前性锭，首先得感謝github上面的qq494257084的m3u8Download項目列牺，讓我獲取了靈感贬堵，同時也參考了這項目的部分代碼带族。萬分感謝前人的輪子锁荔。
吉特哈布地址：https://github.com/qq494257084/m3u8Download
回歸正題，當時做這個下載器的目的就是為了抓取某些網(wǎng)站的電視劇以及電影下載觀看蝙砌，上期我們說到了我自己也封裝了jsoup的框架阳堕，就是那框架讓我抓到了不少m3u8是視頻資源，一開始就打算用個WebView進行瀏覽就算了择克，但是后面想了一下恬总，瀏覽器播放其實挺卡的，外出的時候也不方便觀看肚邢。于是研究了一下m3u8的視頻結(jié)構(gòu)壹堰，就著手進行了這個m3u8視頻下載器的開發(fā)。
首先骡湖，我們先來了解一下什么是m3u8視頻文件贱纠。個人的理解就是m3u8文件相當于一個視頻分片（ts格式的視頻文件）列表索引，用來確定每個分片視頻信息响蕴，視頻分片的順序以及加密信息等等谆焊。也可以說類似于Lrc格式的歌詞文件。m3u8既可以用來作為直播浦夷，也可以用來作為點播辖试，是一種不錯的視頻直播+點播的方案。直播的時候劈狐，服務(wù)器會不斷更新m3u8文件罐孝，不斷的生成視頻分片（ts格式文件）；如果不是直播的時候懈息，也可以當做一個視頻分片的合集肾档。
下載m3u8視頻完整流程需要分為幾步：
第一步：獲取m3u8文件（相當于列表索引）；
第二步：解析m3u8文件辫继，獲取到視頻分片是否有加密怒见，加密方式是什么類型，分片的下載路徑以及時長等等姑宽。
第三步：根據(jù)解析得到的信息遣耍，去下載路徑把全部ts文件都下載下來（如果ts文件已加密，那就需要解密后寫入文件）炮车。
第四步：按照m3u8的文件索引順序舵变，將ts文件合并成一個mp4文件酣溃。
第五步：合并文件成功后，刪除ts分片的視頻文件纪隙。
前文提到的m3u8Download項目實際上已經(jīng)解決了大部分的需求了赊豌，但是，我發(fā)現(xiàn)绵咱，該視頻網(wǎng)站的m3u8文件里面同時包含有加密的ts文件和未加密的ts文件碘饼，這個m3u8Download項目框架就不夠用了。所以我只能參考后悲伶，重新進行下載器的封裝艾恼。
下圖是整個框架的結(jié)構(gòu)：
image.png
1.先把下載任務(wù)的監(jiān)聽器內(nèi)容放出來。
public interface DownloadM3U8Listener {
/**
* 開始下載
*
* @param downloadModel
* @param noAd
*/
void startDownload(M3U8FileInfoModel downloadModel, boolean noAd);
/**
* 下載進度回調(diào)
*
* @param consume 耗時
* @param downloadUrl 下載地址
* @param finished 已完成的數(shù)量
* @param sum 總文件數(shù)量
* @param percent 下載百分比
* @param speedPerSecond 下載速度
* @param noAd 是否去廣告
*/
void process(int consume, String downloadUrl, int finished, int sum, float percent, String speedPerSecond, boolean noAd);
/**
* 合并文件開始
*
* @param downloadModel
* @param noAd
*/
void mergeStart(M3U8FileInfoModel downloadModel, boolean noAd);
/**
* 合并文件結(jié)束
*
* @param downloadModel
* @param noAd
*/
void mergeEnd(M3U8FileInfoModel downloadModel, boolean noAd);
/**
* 刪除ts文件開始
*
* @param downloadModel
* @param noAd
*/
void deleteStart(M3U8FileInfoModel downloadModel, boolean noAd);
/**
* 刪除ts文件結(jié)束
*
* @param downloadModel
* @param noAd
*/
void deleteEnd(M3U8FileInfoModel downloadModel, boolean noAd);
/**
* 下載錯誤
*
* @param downloadModel
* @param e
* @param noAd
*/
void error(M3U8FileInfoModel downloadModel, Exception e, boolean noAd);
/**
* 下載完成（不管成功與否都會回調(diào)）
*
* @param downloadModel
* @param isSuccess
* @param totalSize 總文件數(shù)
* @param finishedCount 完成的數(shù)量
* @param noAd
*/
void complate(M3U8FileInfoModel downloadModel, boolean isSuccess, int totalSize, int finishedCount, boolean noAd);
}
2.定義下載的數(shù)據(jù)類麸锉，有單個m3u8文件的信息類（M3U8FileInfoModel）和單一部分的ts文件數(shù)據(jù)（M3U8TsPartInfoModel）
public class M3U8FileInfoModel {
//m3u8的最外層url
public String url;
//m3u8的真實內(nèi)容URL
public String trueUrl;
//保存的文件名
public String fileName;
//ts文件部分的合集钠绍，為了防止一個m3u8文件出現(xiàn)多個加密算法
public ArrayList<M3U8TsPartInfoModel> tsPartInfoModels = new ArrayList<>();
//合并后的文件存儲目錄
public String dir;
}
public class M3U8TsPartInfoModel {
//key的地址
public String keyUrl = "";
//加密方法
public String method = "";
//iv
public String iv = "";
//key
public String key = "";
//密鑰字節(jié)
public byte[] keyBytes = new byte[16];
//key是否為字節(jié)
public boolean isByte = false;
//ts文件的集合
public Set<TsInfoModel> tsSet = new LinkedHashSet<>();
public void addTsContent(String tsFileStr, double duration, int index) {
tsSet.add(new TsInfoModel(tsFileStr, duration, index));
}
public static class TsInfoModel {
/**
* 地址
*/
public String tsUrl = "";
/**
* 時長
*/
public double duration = 0;
/**
* 索引位置
*/
public int index = 0;
public TsInfoModel(String tsUrl, double duration, int index) {
this.tsUrl = tsUrl;
this.duration = duration;
this.index = index;
}
}
}
3.編寫全局下載的管理類（M3U8DownloadManager），使用了線程池進行多任務(wù)下載管理花沉。
public class M3U8DownloadManager {
private static M3U8DownloadManager m3u8Download;
//最大同時下載的M3U8文件柳爽，默認是3
private int maxDownloadTask = 3;
private ExecutorService fixedThreadPool;
private Set<String> downloadTask = new HashSet<>();
public static M3U8DownloadManager getInstance() {
if (m3u8Download == null) {
synchronized (M3U8DownloadManager.class) {
if (m3u8Download == null) {
m3u8Download = new M3U8DownloadManager();
}
}
}
return m3u8Download;
}
/**
* 設(shè)置最大同時下載的數(shù)量
*
* @param maxDownloadTask
* @return
*/
public M3U8DownloadManager setMaxDownloadTask(int maxDownloadTask) {
this.maxDownloadTask = maxDownloadTask;
return this;
}
/**
* 初始化配置
*
* @return
*/
private M3U8DownloadManager initConfig() {
fixedThreadPool = Executors.newFixedThreadPool(maxDownloadTask);
return this;
}
/**
* 將任務(wù)加入線程池
*
* @param runnable
* @return
*/
public M3U8DownloadManager pushTask(M3U8SingleFileDownloadManager runnable) {
if (fixedThreadPool == null) {
initConfig();
}
fixedThreadPool.execute(runnable);
return this;
}
/**
* 停止全部任務(wù)
*/
public void stopAllTask() {
fixedThreadPool.shutdownNow();
fixedThreadPool = null;
}
}
5.（核心）編寫單個m3u8視頻下載的管理類（M3U8SingleFileDownloadManager）
public abstract class M3U8SingleFileDownloadManager implements Runnable {
//監(jiān)聽
public DownloadM3U8Listener listener;
private M3U8FileInfoModel downloadModel;
private int retryCount = 10;//重試次數(shù)
private boolean noAd = false;//無廣告，true代表無廣告主穗，false代表有廣告
private int maxDownloadCount = 50;//線程池大小
//自定義請求頭
private Map<String, Object> requestHeaderMap = new HashMap<>();
private ExecutorService fixedThreadPool; //優(yōu)化內(nèi)存占用
private static final BlockingQueue<byte[]> BLOCKING_QUEUE = new LinkedBlockingQueue<>();
//已經(jīng)下載的文件大小
private BigDecimal downloadBytes = new BigDecimal(0);
//已完成ts片段個數(shù)
private int finishedCount = 0;
//開始時間,-1表示從0開始
private int startTime = -1;
//結(jié)束時間泻拦，-1表示直到結(jié)尾
private int endTime = -1;
//解密后的片段，用于合并
private Set<File> finishedFiles = new ConcurrentSkipListSet<>(Comparator.comparingInt(o -> Integer.parseInt(o.getName().replace(".xyz", ""))));
//是否是獲取m3u8忽媒，如果是，那么將不執(zhí)行下載
private boolean isGetM3U8 = false;
private boolean isSelectDown = false;
protected M3U8SingleFileDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd) {
this(url, dir, fileName, listener, 30, 100, noAd, false, -1, -1, new ArrayList<>());
}
protected M3U8SingleFileDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd, int startTime, int endTime) {
this(url, dir, fileName, listener, 30, 100, noAd, false, startTime, endTime, new ArrayList<>());
}
protected M3U8SingleFileDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd, boolean isGetM3U8) {
this(url, dir, fileName, listener, 30, 100, noAd, isGetM3U8, -1, -1, new ArrayList<>());
}
protected M3U8SingleFileDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd, ArrayList<M3U8TsPartInfoModel> tsPartInfoModels) {
this(url, dir, fileName, listener, 30, 100, noAd, false, -1, -1, tsPartInfoModels);
}
private M3U8SingleFileDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener,
int retryCount, int maxDownloadCount, boolean noAd, boolean isGetM3U8, int startTime, int endTime, ArrayList<M3U8TsPartInfoModel> tsPartInfoModels) {
downloadModel = new M3U8FileInfoModel();
this.listener = listener;
downloadModel.dir = dir;
downloadModel.fileName = fileName;
downloadModel.url = url;
downloadModel.tsPartInfoModels = tsPartInfoModels;
this.retryCount = retryCount;
this.noAd = noAd;
this.isGetM3U8 = isGetM3U8;
this.startTime = startTime;
this.endTime = endTime;
setThreadCount(maxDownloadCount);
requestHeaderMap.put("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36");
}
/**
* 設(shè)置線程數(shù)量
*
* @param maxDownloadCount
*/
public void setThreadCount(int maxDownloadCount) {
if (BLOCKING_QUEUE.size() < maxDownloadCount) {
for (int i = BLOCKING_QUEUE.size(); i < maxDownloadCount * Constant.FACTOR; i++) {
try {
BLOCKING_QUEUE.put(new byte[Constant.BYTE_COUNT]);
} catch (InterruptedException ignored) {
}
}
}
this.maxDownloadCount = maxDownloadCount;
}
@Override
public void run() {
boolean isSuccess = false;
try {
if (isGetM3U8) {
//校驗字段
checkM3U8Field();
//獲取真正下載的url和m3u8內(nèi)容
getTrueDownloadUrlAndContent(false);
//將m3u8內(nèi)容放進數(shù)據(jù)庫
saveM3U8InfoToDb(downloadModel.fileName, downloadModel.tsPartInfoModels);
} else {
//開始下載
this.listener.startDownload(downloadModel, noAd);
//校驗字段
checkField();
if (downloadModel.tsPartInfoModels != null && downloadModel.tsPartInfoModels.size() > 0) {
//說明是指定片段下載
isSelectDown = true;
} else {
//如果是片段就需要去拿m3u8數(shù)據(jù)
if (startTime != -1 || endTime != -1) {
//片段下載,m3u8信息不入庫
//獲取真正下載的url和m3u8內(nèi)容
getTrueDownloadUrlAndContent(true);
} else {
//全量下載
if (hasDbData(downloadModel.fileName)) {
downloadModel.tsPartInfoModels = getM3U8InfoByName(downloadModel.fileName);
} else {
//獲取真正下載的url和m3u8內(nèi)容
getTrueDownloadUrlAndContent(false);
//將m3u8內(nèi)容放進數(shù)據(jù)庫
saveM3U8InfoToDb(downloadModel.fileName, downloadModel.tsPartInfoModels);
}
}
}
// 下載文件
downloadTsFile();
//開始合并視頻
this.listener.mergeStart(downloadModel, noAd);
mergeTs();
this.listener.mergeEnd(downloadModel, noAd);
// 刪除多余的ts片段
this.listener.deleteStart(downloadModel, noAd);
deleteFiles();
this.listener.deleteEnd(downloadModel, noAd);
}
isSuccess = true;
} catch (Exception e) {
e.printStackTrace();
this.listener.error(downloadModel, e, noAd);
} finally {
this.listener.complate(downloadModel, isSuccess, totalSize, finishedCount, noAd);
}
}
private int totalSize = 0;
/**
* 下載ts文件
*/
private void downloadTsFile() {
fixedThreadPool = Executors.newFixedThreadPool(maxDownloadCount);
int i = 0;
//如果生成目錄不存在腋粥，則創(chuàng)建
File file1 = new File(downloadModel.dir);
if (!file1.exists())
file1.mkdirs();
//將任務(wù)加入線程池
for (int j = 0; j < downloadModel.tsPartInfoModels.size(); j++) {
M3U8TsPartInfoModel partInfoModel = downloadModel.tsPartInfoModels.get(j);
if (partInfoModel != null && partInfoModel.tsSet != null && partInfoModel.tsSet.size() > 0) {
for (M3U8TsPartInfoModel.TsInfoModel tsModel : partInfoModel.tsSet) {
i++;
fixedThreadPool.execute(getThread(tsModel.tsUrl, i, partInfoModel.method, partInfoModel.key, partInfoModel.iv, partInfoModel.isByte, partInfoModel.keyBytes));
}
}
}
fixedThreadPool.shutdown();
totalSize = i;
int consume = 0;
while (!fixedThreadPool.isTerminated()) {
try {
consume++;
BigDecimal bigDecimal = new BigDecimal(downloadBytes.toString());
Thread.sleep(1000L);
listener.process(consume,
downloadModel.url,
finishedCount,
totalSize,
new BigDecimal(finishedCount)
.divide(new BigDecimal(i), 4, BigDecimal.ROUND_HALF_UP)
.multiply(new BigDecimal(100))
.setScale(2, BigDecimal.ROUND_HALF_UP)
.floatValue(),
StringUtils.convertToDownloadSpeed(new BigDecimal(downloadBytes.toString()).subtract(bigDecimal), 3) + "/s",
noAd);
} catch (Exception e) {
e.printStackTrace();
}
}
}
/**
* 開啟下載線程
*
* @param urls ts片段鏈接
* @param i ts片段序號
* @return 線程
*/
private Thread getThread(final String urls, final int i, final String method,
final String key, final String iv, boolean isByte, byte[] keyBytes) {
return new Thread() {
@Override
public void run() {
int count = 1;
HttpURLConnection httpURLConnection = null;
//xy為未解密的ts片段晦雨，如果存在，則刪除
File file2 = new File(downloadModel.dir + FILESEPARATOR + i + ".xy");
if (file2.exists())
file2.delete();
OutputStream outputStream = null;
InputStream inputStream1 = null;
FileOutputStream outputStream1 = null;
byte[] bytes;
try {
bytes = BLOCKING_QUEUE.take();
} catch (InterruptedException e) {
bytes = new byte[Constant.BYTE_COUNT];
}
//重試次數(shù)判斷
while (count <= retryCount) {
try {
//模擬http請求獲取ts片段文件
URL url = new URL(urls);
httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setConnectTimeout((int) 20000L);
for (Map.Entry<String, Object> entry : requestHeaderMap.entrySet())
httpURLConnection.addRequestProperty(entry.getKey(), entry.getValue().toString());
httpURLConnection.setUseCaches(false);
httpURLConnection.setReadTimeout((int) 20000L);
httpURLConnection.setDoInput(true);
InputStream inputStream = httpURLConnection.getInputStream();
try {
outputStream = new FileOutputStream(file2);
} catch (FileNotFoundException e) {
e.printStackTrace();
continue;
}
int len;
//將未解密的ts片段寫入文件
while ((len = inputStream.read(bytes)) != -1) {
outputStream.write(bytes, 0, len);
synchronized (this) {
downloadBytes = downloadBytes.add(new BigDecimal(len));
}
}
outputStream.flush();
inputStream.close();
inputStream1 = new FileInputStream(file2);
int available = inputStream1.available();
if (bytes.length < available)
bytes = new byte[available];
inputStream1.read(bytes);
File file = new File(downloadModel.dir + FILESEPARATOR + i + ".xyz");
outputStream1 = new FileOutputStream(file);
//去廣告版
if ("NONE".equals(method) && noAd) {
break;
}
//開始解密ts片段隘冲，這里我們把ts后綴改為了xyz闹瞧，改不改都一樣
if ("NONE".equals(method) || "".equals(method)) {
SystemLogUtil.printSysDebugLog("解密", "NONE或者正常");
// 這里是一個m3u8文件出現(xiàn)多個解密方式，需要區(qū)別
outputStream1.write(bytes, 0, available);
} else {
byte[] decrypt = decrypt(file2.getName(), bytes, available, key, iv, method, isByte, keyBytes);
if (decrypt == null) {
SystemLogUtil.printSysDebugLog("解密", "正常");
outputStream1.write(bytes, 0, available);
} else {
SystemLogUtil.printSysDebugLog("解密", "AES" + isByte + " " + key + " keyBytes" + LwNetWorkTool.toHexString(keyBytes) + " " + file2.getName());
outputStream1.write(decrypt);
}
}
finishedFiles.add(file);
break;
} catch (Exception e) {
if (e instanceof InvalidKeyException || e instanceof InvalidAlgorithmParameterException) {
SystemLogUtil.printSysDebugLog("M3U8DOWNLOAD", "解密失斦勾恰奥邮！");
break;
}
SystemLogUtil.printSysDebugLog("M3U8DOWNLOAD", e.getMessage().toString() + " 第" + count + "次獲取鏈接重試！\t" + urls);
count++;
e.printStackTrace();
} finally {
try {
if (inputStream1 != null)
inputStream1.close();
if (outputStream1 != null)
outputStream1.close();
if (outputStream != null)
outputStream.close();
BLOCKING_QUEUE.put(bytes);
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
if (httpURLConnection != null) {
httpURLConnection.disconnect();
}
}
}
if (count > retryCount)
//自定義異常
throw new M3u8Exception("連接超時罗珍！");
finishedCount++;
SystemLogUtil.printSysDebugLog("M3U8DOWNLOAD", urls + "下載完畢洽腺！\t已完成" + finishedCount + "個，還剩" + (totalSize - finishedCount) + "個覆旱！");
}
};
}
/**
* 獲取真正的m3u8內(nèi)容的url以及要下載數(shù)據(jù)的models
*
* @param isTimePartDownload 是否是視頻片段下載（true是部分內(nèi)容下載蘸朋，false表示全部內(nèi)容下載）
*/
private void getTrueDownloadUrlAndContent(boolean isTimePartDownload) {
StringBuilder content = getUrlContent(downloadModel.url);
//判斷是否是m3u8鏈接
String conStr = content.toString();
if (!conStr.contains("#EXTM3U"))
throw new M3u8Exception(downloadModel.url + "不是m3u8鏈接！");
//如果含有此字段扣唱，則說明ts片段鏈接需要從第二個m3u8鏈接獲取
if (conStr.contains(".m3u8")) {
String[] split = conStr.split("\\n");
for (String s : split) {
if (s.contains(".m3u8")) {
//如果是url就需要拼接藕坯，如果帶有http開頭就直接賦值
if (StringUtils.isUrl(s)) {
downloadModel.trueUrl = s;
break;
}
//獲取服務(wù)器域名部分
String relativeUrl = downloadModel.url.substring(0, downloadModel.url.lastIndexOf("/") + 1);
if (s.startsWith("/"))
s = s.replaceFirst("/", "");
downloadModel.trueUrl = mergeUrl(relativeUrl, s);
break;
}
}
getTsPartModels(isTimePartDownload);
} else {
downloadModel.trueUrl = downloadModel.url;
//初始化model
getTsPartModels(isTimePartDownload);
}
}
/**
* 獲取TsPart數(shù)據(jù)
*
* @param isTimePartDownload 是否是視頻片段下載（true是部分內(nèi)容下載团南，false表示全部內(nèi)容下載）
*/
private void getTsPartModels(boolean isTimePartDownload) {
SystemLogUtil.printSysDebugLog("M3U8SingleFile", downloadModel.toString());
//獲取m3u8文本內(nèi)容
StringBuilder content = getUrlContent(downloadModel.trueUrl);
//判斷是否是m3u8鏈接
String conStr = content.toString();
if (!conStr.contains("#EXTM3U"))
throw new M3u8Exception(downloadModel.trueUrl + "不是m3u8鏈接！");
String relativeUrl = downloadModel.trueUrl.substring(0, downloadModel.trueUrl.lastIndexOf("/") + 1);
double curDuration = 0;
downloadModel.tsPartInfoModels = new ArrayList<>();
String[] split = conStr.split("\\n");
//判斷是否有加密的部分
boolean hasKey = conStr.contains("#EXT-X-KEY");
M3U8TsPartInfoModel partInfoModel = null;
boolean isContent = false;
double duration = 0;
int index = 0;
if (!hasKey) {
partInfoModel = new M3U8TsPartInfoModel();
}
for (String s : split) {
//結(jié)束了
if (s.contains("#EXT-X-ENDLIST")) {
if (partInfoModel != null) {
//如果不為空炼彪，說明上一批已經(jīng)有數(shù)據(jù)了吐根，需要加入downloadModel
downloadModel.tsPartInfoModels.add(partInfoModel);
}
break;
}
if (hasKey) {
//獲取key
if (s.contains("#EXT-X-KEY")) {
//獲取密鑰
if (partInfoModel != null) {
//如果不為空，說明上一批已經(jīng)有數(shù)據(jù)了辐马，需要加入downloadModel
downloadModel.tsPartInfoModels.add(partInfoModel);
}
//拆分密鑰的詳情
String[] split1 = s.split(",");
partInfoModel = new M3U8TsPartInfoModel();
index = 0;
for (String s1 : split1) {
if (s1.contains("METHOD")) {
partInfoModel.method = s1.split("=", 2)[1].replaceAll("\"", "");
continue;
}
if (s1.contains("URI")) {
partInfoModel.keyUrl = s1.split("=", 2)[1].replaceAll("\"", "");
continue;
}
if (s1.contains("IV")) {
partInfoModel.iv = s1.split("=", 2)[1].replaceAll("\"", "");
continue;
}
}
initKeyByUrl(partInfoModel, StringUtils.isUrl(partInfoModel.keyUrl) ? partInfoModel.keyUrl : mergeUrl(relativeUrl, partInfoModel.keyUrl));
isContent = false;
continue;
}
//獲取內(nèi)容
if (s.contains("#EXTINF")) {
isContent = true;
duration = Double.parseDouble(s.replaceAll("#EXTINF:", "").replaceAll(",", ""));
//統(tǒng)計
curDuration += duration;
continue;
}
if (curDuration < startTime
&& startTime != -1
&& isTimePartDownload) {
SystemLogUtil.printSysDebugLog("片段下載", "還未到時間" + curDuration + " " + startTime + " " + endTime);
continue;
}
setTsContent(relativeUrl, partInfoModel, isContent, s, duration, index);
if (curDuration > endTime
&& endTime != -1
&& isTimePartDownload) {
//超過了時間就退出
SystemLogUtil.printSysDebugLog("片段下載", "跳出循環(huán)" + curDuration + " " + startTime + " " + endTime);
if (partInfoModel != null) {
//如果不為空佑惠，說明上一批已經(jīng)有數(shù)據(jù)了，需要加入downloadModel
downloadModel.tsPartInfoModels.add(partInfoModel);
}
break;
}
index++;
isContent = false;
} else {
//獲取內(nèi)容
if (s.contains("#EXTINF")) {
isContent = true;
duration = Double.parseDouble(s.replaceAll("#EXTINF:", "").replaceAll(",", ""));
//統(tǒng)計
curDuration += duration;
continue;
}
if (curDuration < startTime
&& startTime != -1
&& isTimePartDownload) {
SystemLogUtil.printSysDebugLog("片段下載", "還未到時間" + curDuration + " " + startTime + " " + endTime);
continue;
}
setTsContent(relativeUrl, partInfoModel, isContent, s, duration, index);
if (curDuration > endTime
&& endTime != -1
&& isTimePartDownload) {
//超過了時間就退出
SystemLogUtil.printSysDebugLog("片段下載", "跳出循環(huán)" + curDuration + " " + startTime + " " + endTime);
if (partInfoModel != null) {
//如果不為空齐疙，說明上一批已經(jīng)有數(shù)據(jù)了膜楷，需要加入downloadModel
downloadModel.tsPartInfoModels.add(partInfoModel);
}
break;
}
index++;
isContent = false;
}
}
SystemLogUtil.printSysDebugLog("M3U8SingleFile", "獲取文件部分：" + downloadModel.toString());
}
/**
* 設(shè)置ts文件內(nèi)容
*
* @param relativeUrl
* @param partInfoModel
* @param isContent
* @param s
* @return
*/
private void setTsContent(String relativeUrl, M3U8TsPartInfoModel partInfoModel, boolean isContent, String s, double duration, int index) {
if (isContent) {
partInfoModel.addTsContent(StringUtils.isUrl(s) ? s : mergeUrl(relativeUrl, s), duration, index);
}
}
/**
* 去url獲取key內(nèi)容
*
* @param partInfoModel
* @param keyUrl
* @return
*/
private void initKeyByUrl(M3U8TsPartInfoModel partInfoModel, String keyUrl) {
if (!StringUtils.isEmpty(partInfoModel.method) && !"NONE".equals(partInfoModel.method)) {
int count = 1;
HttpURLConnection httpURLConnection = null;
while (count <= retryCount) {
try {
URL url = new URL(keyUrl);
httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setConnectTimeout((int) 10000L);
httpURLConnection.setReadTimeout((int) 10000L);
httpURLConnection.setUseCaches(false);
httpURLConnection.setDoInput(true);
for (Map.Entry<String, Object> entry : requestHeaderMap.entrySet())
httpURLConnection.addRequestProperty(entry.getKey(), entry.getValue().toString());
InputStream inputStream = httpURLConnection.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
byte[] bytes = new byte[128];
int len;
len = inputStream.read(bytes);
if (len == 1 << 4) {
partInfoModel.isByte = true;
partInfoModel.key = "isByte";
partInfoModel.keyBytes = Arrays.copyOf(bytes, 16);
} else {
partInfoModel.isByte = false;
partInfoModel.key = new String(Arrays.copyOf(bytes, len));
}
bufferedReader.close();
inputStream.close();
SystemLogUtil.printSysDebugLog("M3U8SingleFile", "獲取key內(nèi)容： ");
break;
} catch (Exception e) {
SystemLogUtil.printSysDebugLog("M3U8SingleFile", "獲取key異常" + e.getMessage().toString() + " 第" + count + "獲取鏈接重試！\t" + keyUrl);
count++;
e.printStackTrace();
} finally {
if (httpURLConnection != null) {
httpURLConnection.disconnect();
}
}
}
if (count > retryCount)
throw new M3u8Exception("獲取key連接超時贞奋！");
}
}
/**
* 模擬http請求獲取內(nèi)容
*
* @param urls http鏈接
* @return 內(nèi)容
*/
private StringBuilder getUrlContent(String urls) {
int count = 1;
HttpURLConnection httpURLConnection = null;
StringBuilder content = new StringBuilder();
while (count <= retryCount) {
try {
URL url = new URL(urls);
httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setConnectTimeout((int) 10000L);
httpURLConnection.setReadTimeout((int) 10000L);
httpURLConnection.setUseCaches(false);
httpURLConnection.setDoInput(true);
for (Map.Entry<String, Object> entry : requestHeaderMap.entrySet())
httpURLConnection.addRequestProperty(entry.getKey(), entry.getValue().toString());
String line;
InputStream inputStream = httpURLConnection.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
while ((line = bufferedReader.readLine()) != null)
content.append(line).append("\n");
bufferedReader.close();
inputStream.close();
SystemLogUtil.printSysDebugLog("M3U8SingleFile", "getUrlContent " + content.toString());
break;
} catch (Exception e) {
SystemLogUtil.printSysDebugLog("M3U8SingleFile", e.getMessage().toString() + " 第" + count + "獲取鏈接重試赌厅！\t" + urls);
count++;
e.printStackTrace();
} finally {
if (httpURLConnection != null) {
httpURLConnection.disconnect();
}
}
}
if (count > retryCount)
throw new M3u8Exception("連接超時！");
return content;
}
/**
* 字段校驗
*/
private void checkField() throws M3u8Exception {
if ("m3u8".compareTo(MediaFormat.getMediaFormat(downloadModel.url)) != 0)
throw new M3u8Exception(downloadModel.url + "不是一個完整m3u8鏈接轿塔！");
if (StringUtils.isEmpty(downloadModel.dir))
throw new M3u8Exception("視頻存儲目錄不能為空特愿！");
if (StringUtils.isEmpty(downloadModel.fileName))
throw new M3u8Exception("視頻名稱不能為空！");
finishedFiles.clear();
downloadBytes = new BigDecimal(0);
finishedCount = 0;
}
/**
* 字段校驗
*/
private void checkM3U8Field() throws M3u8Exception {
if ("m3u8".compareTo(MediaFormat.getMediaFormat(downloadModel.url)) != 0)
throw new M3u8Exception(downloadModel.url + "不是一個完整m3u8鏈接勾缭！");
if (StringUtils.isEmpty(downloadModel.fileName))
throw new M3u8Exception("視頻名稱不能為空揍障！");
finishedFiles.clear();
downloadBytes = new BigDecimal(0);
finishedCount = 0;
}
/**
* 解密ts
*
* @param name
* @param sSrc ts文件字節(jié)數(shù)組
* @param length
* @param sKey 密鑰
* @return 解密后的字節(jié)數(shù)組
*/
private byte[] decrypt(String name, byte[] sSrc, int length, String sKey, String iv, String method, boolean isByte, byte[] keyBytes) throws Exception {
if (StringUtils.isNotEmpty(method) && !method.contains("AES"))
throw new M3u8Exception("未知的算法！" + method);
// 判斷Key是否正確
if (StringUtils.isEmpty(sKey)) {
SystemLogUtil.printSysDebugLog("解密", name + " skey= " + sKey);
return null;
}
// 判斷Key是否為16位
if (sKey.length() != 16 && !isByte) {
throw new M3u8Exception("Key長度不是16位俩由！");
}
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7Padding");
SecretKeySpec keySpec = new SecretKeySpec(isByte ? keyBytes : sKey.getBytes("UTF-8"), "AES");
byte[] ivByte;
if (iv.startsWith("0x"))
ivByte = StringUtils.hexStringToByteArray(iv.substring(2));
else ivByte = iv.getBytes();
if (ivByte.length != 16)
ivByte = new byte[16];
//如果m3u8有IV標簽毒嫡，那么IvParameterSpec構(gòu)造函數(shù)就把IV標簽后的內(nèi)容轉(zhuǎn)成字節(jié)數(shù)組傳進去
AlgorithmParameterSpec paramSpec = new IvParameterSpec(ivByte);
cipher.init(Cipher.DECRYPT_MODE, keySpec, paramSpec);
return cipher.doFinal(sSrc, 0, length);
}
/**
* 組合url
*
* @param start
* @param end
* @return
*/
private String mergeUrl(String start, String end) {
if (end.startsWith("/"))
end = end.replaceFirst("/", "");
int position = 0;
String subEnd, tempEnd = end;
while ((position = end.indexOf("/", position)) != -1) {
subEnd = end.substring(0, position + 1);
if (start.endsWith(subEnd)) {
tempEnd = end.replaceFirst(subEnd, "");
break;
}
++position;
}
return start + tempEnd;
}
public static final SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss");
/**
* 合并文件
*/
private void mergeTs() {
try {
String saveName = downloadModel.fileName;
if (isSelectDown) {
saveName = downloadModel.fileName + sdf.format(new Date());
} else {
if (startTime == -1 && endTime == -1) {
//說明是下載完整的視頻
saveName = downloadModel.fileName;
} else if (startTime != -1 && endTime != -1) {
saveName = "(" + StringUtils.timeParseBySecond(startTime) + "-" + StringUtils.timeParseBySecond(endTime) + ")" + downloadModel.fileName;
} else if (startTime == -1) {
saveName = "(00:00-" + StringUtils.timeParseBySecond(endTime) + ")" + downloadModel.fileName;
} else if (endTime == -1) {
saveName = "(" + StringUtils.timeParseBySecond(startTime) + "toEnd)" + downloadModel.fileName;
}
}
File file = new File(downloadModel.dir + FILESEPARATOR
+ saveName + ".mp4");
System.gc();
if (file.exists())
file.delete();
else file.createNewFile();
FileOutputStream fileOutputStream = new FileOutputStream(file);
byte[] b = new byte[4096];
for (File f : finishedFiles) {
SystemLogUtil.printSysDebugLog("合并文件", "合并了 " + f.getName());
FileInputStream fileInputStream = new FileInputStream(f);
int len;
while ((len = fileInputStream.read(b)) != -1) {
fileOutputStream.write(b, 0, len);
}
fileInputStream.close();
fileOutputStream.flush();
}
fileOutputStream.close();
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 合并完成后刪除ts文件
*/
private void deleteFiles() {
File file = new File(downloadModel.dir);
for (File f : file.listFiles()) {
if (f.getName().endsWith(".xy") || f.getName().endsWith(".xyz"))
f.delete();
}
}
/**
* 判斷是否有數(shù)據(jù)
*
* @param fileName
* @return
*/
protected abstract boolean hasDbData(String fileName);
/**
* 獲取
*
* @param fileName
* @return
*/
protected abstract ArrayList<M3U8TsPartInfoModel> getM3U8InfoByName(String fileName) throws Exception;
/**
* 保存m3u8信息
*
* @param fileName
* @param tsPartInfoModels
*/
protected abstract void saveM3U8InfoToDb(String fileName, ArrayList<M3U8TsPartInfoModel> tsPartInfoModels);
}
至此，下載的庫就封裝完成了幻梯，接下來我們看看怎么樣使用這個下載庫兜畸。
參考下面的幫助類：
/**
* <pre>
* author : lawwing
* time : 2021年03月25日
* desc : 添加保存功能的
* version: 1.0
* </pre>
*/
public class M3U8SaveDbAndDownloadManager extends M3U8SingleFileDownloadManager {
public M3U8SaveDbAndDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd) {
super(url, dir, fileName, listener, noAd);
}
public M3U8SaveDbAndDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd, int startTime, int endTime) {
super(url, dir, fileName, listener, noAd, startTime, endTime);
}
public M3U8SaveDbAndDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd, ArrayList<M3U8TsPartInfoModel> tsPartInfoModels) {
super(url, dir, fileName, listener, noAd, tsPartInfoModels);
}
public M3U8SaveDbAndDownloadManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd, boolean isGetM3U8) {
super(url, dir, fileName, listener, noAd, isGetM3U8);
}
/**
* 獲取m3u8內(nèi)容的manager
*
* @param url
* @param fileName
* @param listener
* @return
*/
public static M3U8SingleFileDownloadManager getFindM3U8DataManager(String url, String fileName, DownloadM3U8Listener listener) {
return new M3U8SaveDbAndDownloadManager(url, "", fileName, listener, false, true);
}
/**
* 獲取下載任務(wù)的manager
*
* @param url
* @param fileName
* @param listener
* @return
*/
public static M3U8SingleFileDownloadManager getDownloadDataManager(String url, String dir, String fileName, DownloadM3U8Listener listener, boolean noAd) {
return new M3U8SaveDbAndDownloadManager(url, dir, fileName, listener, noAd);
}
/**
* 獲取下載指定時間任務(wù)的manager
*
* @param url
* @param dir
* @param fileName
* @param listener
* @param noAd
* @param startTime
* @param endTime
* @return
*/
public static M3U8SingleFileDownloadManager getDownloadDataByTimeManager(String url, String dir, String fileName,
DownloadM3U8Listener listener, boolean noAd,
Integer startTime, Integer endTime) {
return new M3U8SaveDbAndDownloadManager(url, dir, fileName, listener, noAd, startTime.intValue(), endTime.intValue());
}
/**
* 獲取下載指定任務(wù)片的manager
*
* @param url
* @param dir
* @param fileName
* @param listener
* @param noAd
* @return
*/
public static M3U8SingleFileDownloadManager getDownloadDataSelectManager(String url, String dir, String fileName,
DownloadM3U8Listener listener, boolean noAd, ArrayList<M3U8TsPartInfoModel> tsPartInfoModels) {
return new M3U8SaveDbAndDownloadManager(url, dir, fileName, listener, noAd, tsPartInfoModels);
}
@Override
protected boolean hasDbData(String fileName) {
try {
int count = VideoDatabase.Companion.getDBInstace().getM3U8PartDao().checkCountByName(fileName);
if (count > 0) {
return true;
} else {
return false;
}
} catch (Exception e) {
e.printStackTrace();
}
return false;
}
@Override
protected ArrayList<M3U8TsPartInfoModel> getM3U8InfoByName(String fileName) throws Exception {
ArrayList<M3U8TsPartInfoModel> result = new ArrayList<>();
List<M3U8FilePartDbInfo> datasByNameAsc = VideoDatabase.Companion.getDBInstace().getM3U8PartDao().findDatasByNameAsc(fileName);
if (datasByNameAsc != null && datasByNameAsc.size() > 0) {
for (int i = 0; i < datasByNameAsc.size(); i++) {
M3U8FilePartDbInfo m3U8FilePartDbInfo = datasByNameAsc.get(i);
if (m3U8FilePartDbInfo != null) {
int m3u8id = m3U8FilePartDbInfo.getId();
M3U8TsPartInfoModel partModel = new M3U8TsPartInfoModel();
partModel.method = m3U8FilePartDbInfo.getMethod();
partModel.key = m3U8FilePartDbInfo.getKey();
partModel.keyUrl = m3U8FilePartDbInfo.getKeyUrl();
partModel.keyBytes = m3U8FilePartDbInfo.getKeyBytes();
partModel.iv = m3U8FilePartDbInfo.getIv();
partModel.isByte = m3U8FilePartDbInfo.isByte();
List<TsFileDbInfo> tsFileDbInfoList = VideoDatabase.Companion.getDBInstace().getTsFileDao().findDatasByPartIdAsc(m3u8id);
if (tsFileDbInfoList != null && tsFileDbInfoList.size() > 0) {
for (int j = 0; j < tsFileDbInfoList.size(); j++) {
TsFileDbInfo tsFileDbInfo = tsFileDbInfoList.get(j);
partModel.tsSet.add(new M3U8TsPartInfoModel.TsInfoModel(tsFileDbInfo.getTsPath(),
tsFileDbInfo.getDuration(),
tsFileDbInfo.getPartindex()));
}
}
result.add(partModel);
}
}
}
return result;
}
@Override
public void saveM3U8InfoToDb(String fileName, ArrayList<M3U8TsPartInfoModel> tsPartInfoModels) {
//如果已經(jīng)有了，就刪除數(shù)據(jù)庫之前的數(shù)據(jù)
List<M3U8FilePartDbInfo> datasByNameAsc = VideoDatabase.Companion.getDBInstace().getM3U8PartDao().findDatasByNameAsc(fileName);
if (datasByNameAsc != null && datasByNameAsc.size() > 0) {
for (int i = 0; i < datasByNameAsc.size(); i++) {
M3U8FilePartDbInfo m3U8FilePartDbInfo = datasByNameAsc.get(i);
VideoDatabase.Companion.getDBInstace().getTsFileDao().deleteByPartId(m3U8FilePartDbInfo.getId());
VideoDatabase.Companion.getDBInstace().getM3U8PartDao().deleteById(m3U8FilePartDbInfo.getId());
}
}
// 保存到數(shù)據(jù)庫
if (tsPartInfoModels != null && tsPartInfoModels.size() > 0) {
int partIndex = 0;
for (int j = 0; j < tsPartInfoModels.size(); j++) {
M3U8TsPartInfoModel partInfoModel = tsPartInfoModels.get(j);
M3U8FilePartDbInfo dbPartInfo = new M3U8FilePartDbInfo();
dbPartInfo.setPartindex(partIndex);
dbPartInfo.setName(fileName);
dbPartInfo.setKey(partInfoModel.key);
dbPartInfo.setKeyUrl(partInfoModel.keyUrl);
dbPartInfo.setKeyBytes(partInfoModel.keyBytes);
dbPartInfo.setByte(partInfoModel.isByte);
dbPartInfo.setIv(partInfoModel.iv);
dbPartInfo.setMethod(partInfoModel.method);
//先插入數(shù)據(jù)再獲取到id
VideoDatabase.Companion.getDBInstace().getM3U8PartDao().insert(dbPartInfo);
int partId = VideoDatabase.Companion.getDBInstace().getM3U8PartDao().findPartIdByNameMaxPartindex(fileName);
ArrayList<TsFileDbInfo> tsList = new ArrayList<>();
if (partInfoModel != null && partInfoModel.tsSet != null && partInfoModel.tsSet.size() > 0) {
for (M3U8TsPartInfoModel.TsInfoModel tsInfo : partInfoModel.tsSet) {
TsFileDbInfo tsModel = new TsFileDbInfo();
tsModel.setM3u8id(partId);
tsModel.setTsPath(tsInfo.tsUrl);
tsModel.setDuration(tsInfo.duration);
tsModel.setPartindex(tsInfo.index);
tsList.add(tsModel);
}
}
VideoDatabase.Companion.getDBInstace().getTsFileDao().insertAll(tsList);
partIndex++;
}
}
}
}
由于將m3u8信息保存到數(shù)據(jù)庫中碘梢，可減少服務(wù)器請求次數(shù)咬摇，防止ip被封禁，所以下面附上數(shù)據(jù)庫的建表相關(guān)代碼（使用了room數(shù)據(jù)庫框架）：
@Entity(tableName = "M3U8FilePartInfo")
class M3U8FilePartDbInfo : BaseObservable() {
@PrimaryKey(autoGenerate = true)
@ColumnInfo(name = "id")
var id: Int? = null
/**
* 這個塊所在的位置
*/
@ColumnInfo(name = "partindex")
var partindex: Int? = null
/**
* 名字
*/
@ColumnInfo(name = "name")
var name: String? = null
/**
*key的地址
*/
@ColumnInfo(name = "keyUrl")
var keyUrl: String? = ""
/**
*加密方法
*/
@ColumnInfo(name = "method")
var method: String? = ""
/**
*iv
*/
@ColumnInfo(name = "iv")
var iv: String? = ""
/**
*key
*/
@ColumnInfo(name = "key")
var key: String? = ""
/**
*密鑰字節(jié)
*/
@ColumnInfo(name = "keyBytes")
var keyBytes: ByteArray? = ByteArray(16)
/**
*key是否為字節(jié)
*/
@ColumnInfo(name = "isByte")
var isByte: Boolean? = false
}
@Entity(tableName = "TsFileInfo")
class TsFileDbInfo : BaseObservable() {
@PrimaryKey(autoGenerate = true)
@ColumnInfo(name = "id")
var id: Int? = null
/**
* 關(guān)聯(lián)的表M3U8FilePartInfo的id
*/
@ColumnInfo(name = "m3u8id")
var m3u8id: Int? = null
/**
* 這個ts塊所在的位置
*/
@ColumnInfo(name = "partindex")
var partindex: Int? = null
/**
* 分片路徑
*/
@ColumnInfo(name = "tsPath")
var tsPath: String? = null
/**
* 分片時長
*/
@ColumnInfo(name = "duration")
var duration: Double? = null
}
網(wǎng)上的輪子縱然好用煞躬，但是也不會完全適合自己的需求肛鹏，所以還得去修改，去創(chuàng)新恩沛。
自己封裝的框架不僅可以下載完整的m3u8視頻的文件在扰，還可以選擇分片，選擇指定時間去下載所需要的視頻片段复唤。除此之外健田，研究了該網(wǎng)站的視頻，發(fā)現(xiàn)廣告都是插在不加密的視頻部分佛纫，所以不加密的視頻不下載妓局，就成了去除廣告的功能了总放。
