前段時(shí)間由于業(yè)務(wù)需要愈涩,需要做一個(gè)視頻下載的功能栽渴,包括m3u8視頻和mp4視頻等,于是在Github上找了幾個(gè)相關(guān)的下載庫溪窒,發(fā)現(xiàn)要不是太久沒有更新了,要不就是不太符合我們的需求冯勉,所以干脆就手?jǐn)]了一個(gè)
M3U8Downloader
Github
地址:https://github.com/xuqingquan1995/M3U8Downloader
Gitee
地址:https://gitee.com/xuqingquan/M3U8Downloader
M3U8文件結(jié)構(gòu)
開始擼代碼之前澈蚌,先預(yù)備一下相關(guān)知識(shí),M3U8視頻其實(shí)主要就一個(gè)文件灼狰,文件里面寫明了視頻片段ts的地址宛瞄,我們獲得這個(gè)m3u8文件就可以通過文件內(nèi)的內(nèi)容,分析出世紀(jì)的ts交胚,然后下載相對(duì)應(yīng)的ts文件份汗,就可以做到下載m3u8視頻了
最直接的m3u8文件
https://135zyv5.xw0371.com/2018/10/29/X05c7CG3VB91gi1M/playlist.m3u8
這個(gè)鏈接的m3u8文件下載后內(nèi)容如下
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:19
#EXTINF:12.640000,
out000.ts
#EXTINF:7.960000,
out001.ts
#EXTINF:12.280000,
out002.ts
#EXTINF:7.520000,
out003.ts
#EXTINF:10.240000,
out004.ts
#EXTINF:15.520000,
out005.ts
#EXTINF:8.600000,
out006.ts
#EXTINF:7.440000,
out007.ts
#EXTINF:8.240000,
out008.ts
#EXTINF:10.000000,
out009.ts
#EXTINF:13.120000,
out010.ts
。蝴簇。杯活。。熬词。旁钧。。
可以很直觀的看出荡澎,其實(shí)這個(gè)文件里面是一系列的ts文件
需要重定向的m3u8
還有例如以下這兩個(gè)鏈接的m3u8文件下載后內(nèi)容如下,只有簡單的一行
http://youku.cdn-iqiyi.com/20180523/11112_b1fb9d8b/index.m3u8
#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=800000,RESOLUTION=1080x608
1000k/hls/index.m3u8
#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1000000,RESOLUTION=1280x720
/ppvod/1F94756C565EC42C5735D57272032622.m3u8
對(duì)于這一類的m3u8文件均践,其實(shí)是需要重定向的,重定向后可以獲得真實(shí)的m3u8地址摩幔,從而獲取到對(duì)應(yīng)的ts地址
根據(jù)url規(guī)則彤委,以上兩個(gè)m3u8的實(shí)際地址為:
http://youku.cdn-iqiyi.com/20180523/11112_b1fb9d8b/index.m3u8 轉(zhuǎn)為:http://youku.cdn-iqiyi.com/20180523/11112_b1fb9d8b/1000k/hls/index.m3u8
https://v8.yongjiu8.com/20180321/V8I5Tg8p/index.m3u8 轉(zhuǎn)為:https://v8.yongjiu8.com/ppvod/1F94756C565EC42C5735D57272032622.m3u8
ts文件分析
對(duì)于獲取到的ts文件主要有以下幾種類型:
- 只有文件名
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:9
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:4.276000,
65f7a658c87000.ts
#EXTINF:4.170000,
65f7a658c87001.ts
#EXTINF:5.754600,
65f7a658c87002.ts
#EXTINF:4.170000,
65f7a658c87003.ts
#EXTINF:4.170000,
- 帶有路徑的
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10,
/20180321/V8I5Tg8p/1000kb/hls/bdmhnU1119000.ts
#EXTINF:10,
/20180321/V8I5Tg8p/1000kb/hls/bdmhnU1119001.ts
#EXTINF:10,
/20180321/V8I5Tg8p/1000kb/hls/bdmhnU1119002.ts
#EXTINF:10,
/20180321/V8I5Tg8p/1000kb/hls/bdmhnU1119003.ts
#EXTINF:7.8,
/20180321/V8I5Tg8p/1000kb/hls/bdmhnU1119004.ts
其實(shí)也是根據(jù)url規(guī)則進(jìn)行替換,對(duì)于只有文件名的ts文件或衡,只要把它對(duì)應(yīng)的m3u8地址最后的文件名替換成ts文件名就行了焦影,對(duì)于帶有路徑的,根據(jù)url規(guī)則封断,如果以/開頭的斯辰,則代表是在域名根目錄下的,不是/開頭的坡疼,則代表是在當(dāng)前目錄下的彬呻,進(jìn)行相應(yīng)替換就可以得到ts文件的url地址了
技術(shù)選型
既然是下載,免不了的是涉及到網(wǎng)絡(luò)請(qǐng)求的實(shí)現(xiàn),其實(shí)就是具體的下載怎么去做闸氮,在Github
上有找到一個(gè)okdownload這個(gè)庫剪况,之所以選擇它,一方面是他是下載庫star最多的FileDownloader的升級(jí)版,另一方面是它的批下載功能符合我下載m3u8這樣多個(gè)ts文件的場景
代碼實(shí)現(xiàn)
數(shù)據(jù)類型準(zhǔn)備
VideoDownloadEntity
主要是存儲(chǔ)過程中的數(shù)據(jù)蒲跨,并且方便之后操作的
const val NO_START = 0
const val PREPARE = 1
const val DOWNLOADING = 2
const val PAUSE = 3
const val COMPLETE = 4
const val ERROR = 5
const val DELETE = -1
class VideoDownloadEntity(
var originalUrl: String,//原始下載鏈接
var name: String = "",//視頻名稱
var subName: String = "",//視頻子名稱
var redirectUrl: String = "",//重定向后的下載鏈接
var fileSize: Long = 0,//文件總大小
var currentSize: Long = 0,//當(dāng)前已下載大小
var currentProgress: Double = 0.0,//當(dāng)前進(jìn)度
var currentSpeed: String = "",//當(dāng)前速率
var tsSize: Int = 0,//ts的數(shù)量
var createTime: Long = System.currentTimeMillis()//創(chuàng)建時(shí)間
) : Parcelable, Comparable<VideoDownloadEntity> {
//狀態(tài)
var status: Int = NO_START
set(value) {
if (field != DELETE) {
field = value
}
if (value == DELETE) {
startDownload = null
downloadContext?.stop()
downloadTask?.cancel()
}
}
var downloadContext: DownloadContext? = null
var downloadTask: DownloadTask? = null
var startDownload: (() -> Unit)? = null
constructor(parcel: Parcel) : this(
parcel.readString() ?: "",
parcel.readString() ?: "",
parcel.readString() ?: "",
parcel.readString() ?: "",
parcel.readLong(),
parcel.readLong(),
parcel.readDouble(),
parcel.readString() ?: "",
parcel.readInt(),
parcel.readLong()
) {
this.status = parcel.readInt()
}
override fun writeToParcel(parcel: Parcel, flags: Int) {
parcel.writeString(originalUrl)
parcel.writeString(name)
parcel.writeString(subName)
parcel.writeString(redirectUrl)
parcel.writeLong(fileSize)
parcel.writeLong(currentSize)
parcel.writeDouble(currentProgress)
parcel.writeString(currentSpeed)
parcel.writeInt(tsSize)
parcel.writeLong(createTime)
parcel.writeInt(status)
}
override fun describeContents(): Int {
return 0
}
companion object CREATOR : Parcelable.Creator<VideoDownloadEntity> {
override fun createFromParcel(parcel: Parcel): VideoDownloadEntity {
return VideoDownloadEntity(parcel)
}
override fun newArray(size: Int): Array<VideoDownloadEntity?> {
return arrayOfNulls(size)
}
}
override fun toString(): String {
val json = JSONObject()
json.put("originalUrl", originalUrl)
json.put("name", name)
json.put("subName", subName)
json.put("redirectUrl", redirectUrl)
json.put("fileSize", fileSize)
json.put("currentSize", currentSize)
json.put("currentProgress", currentProgress)
json.put("currentSpeed", currentSpeed)
json.put("tsSize", tsSize)
json.put("createTime", createTime)
json.put("status", status)
return json.toString()
}
fun toFile() {
val path = FileDownloader.getDownloadPath(originalUrl)
val config = File(path, "video.config")
if (!config.exists() && this.createTime == 0L) {
this.createTime = System.currentTimeMillis()
}
config.writeText(toString())
}
override fun compareTo(other: VideoDownloadEntity) =
(other.createTime - this.createTime).toInt()
}
fun parseJsonToVideoDownloadEntity(jsonString: String): VideoDownloadEntity? {
if (jsonString.isEmpty()) {
return null
}
return try {
val json = JSONObject(jsonString)
val entity = VideoDownloadEntity(
json.getString("originalUrl"),
json.getString("name"),
json.getString("subName"),
json.getString("redirectUrl"),
json.getLong("fileSize"),
json.getLong("currentSize"),
json.getDouble("currentProgress"),
json.getString("currentSpeed"),
json.getInt("tsSize"),
json.getLong("createTime")
)
entity.status = json.getInt("status")
entity
} catch (t: Throwable) {
t.printStackTrace()
null
}
}
獲取真實(shí)ts路徑
下載m3u8文件译断,最開始是獲取到真實(shí)的ts文件,那么先創(chuàng)建一個(gè)M3U8ConfigDownloader
進(jìn)行配置文件的獲取
internal object M3U8ConfigDownloader {
private val downloadList = arrayListOf<String>()
private val TAG = "M3U8ConfigDownloader"
//清楚所有任務(wù)或悲,
fun clear() {
downloadList.clear()
}
/**
* @return 如果返回空則不需要下載孙咪,如果返回的文件存在了,則開始下載巡语,否則等待下載完成
*/
fun start(entity: VideoDownloadEntity): File? {
if (entity.status == DELETE) {
return null
}
if (downloadList.contains(entity.originalUrl)) {
return null
}
if (entity.createTime == 0L) {
entity.createTime = System.currentTimeMillis()
}
entity.redirectUrl = ""
val path = FileDownloader.getDownloadPath(entity.originalUrl)
val config = FileDownloader.getConfigFile(entity.originalUrl)
val realEntity = if (!config.exists()) {
entity.toFile()
entity
} else {
parseJsonToVideoDownloadEntity(config.readText()) ?: entity
}
if (entity.status == DELETE) {
path.deleteRecursively()
return null
}
val m3u8ListFile = File(path, "m3u8.list")
return if (realEntity.status != COMPLETE) {//沒有完成的才有必要下載
Log.d(TAG, "init")
if (m3u8ListFile.exists()) {
Log.d(TAG, "從文件下載")
} else {
Log.d(TAG, "從0開始下載")
realEntity.status = PREPARE
FileDownloader.downloadCallback.postValue(realEntity)
entity.toFile()
//進(jìn)入下載m3u8
downloadM3U8File(path, realEntity)
}
m3u8ListFile
} else {
null
}
}
/**
* 下載單個(gè)文件
*/
private fun downloadM3U8File(path: File, entity: VideoDownloadEntity) {
if (entity.status == DELETE) {
return
}
val fileName: String
val url = if (entity.redirectUrl.isNotEmpty()) {//如果有了重定向的url
fileName = "real.m3u8"
entity.redirectUrl
} else {//否則就用初始的url
fileName = "original.m3u8"
entity.originalUrl
}
Log.d(TAG, "downloadM3U8File-url=$url,fileName=$fileName")
val downloadFile = File(path, fileName)
DownloadTask.Builder(url, downloadFile.parentFile)
.setFilename(downloadFile.name)
.build()
.enqueue(object : DownloadListener1() {
override fun taskStart(task: DownloadTask, model: Listener1Assist.Listener1Model) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
Log.d(TAG, "taskStart-->")
downloadList.add(task.url)
}
override fun taskEnd(
task: DownloadTask, cause: EndCause, realCause: Exception?,
model: Listener1Assist.Listener1Model
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
Log.d(TAG, "taskEnd-->${cause.name},${realCause?.message}")
if (cause == EndCause.COMPLETED) {
getFileContent(path, entity)
} else {
entity.status = ERROR
downloadList.remove(entity.originalUrl)
entity.startDownload = {
start(entity)
}
entity.toFile()
FileDownloader.downloadCallback.postValue(entity)
}
}
override fun progress(task: DownloadTask, currentOffset: Long, totalLength: Long) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
}
override fun connected(
task: DownloadTask, blockCount: Int, currentOffset: Long, totalLength: Long
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
Log.d(TAG, "connected-->")
}
override fun retry(task: DownloadTask, cause: ResumeFailedCause) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
}
})
}
/**
* 分析文件內(nèi)容
*/
private fun getFileContent(path: File, entity: VideoDownloadEntity) {
if (entity.status == DELETE) {
return
}
Log.d(TAG, "getFileContent---$entity")
val url = if (entity.redirectUrl.isNotEmpty()) {//如果有了重定向的url
entity.redirectUrl
} else {//否則就用初始的url
entity.originalUrl
}
val uri = Uri.parse(url)
val realM3U8File = File(path, "real.m3u8")
var file = realM3U8File
if (!file.exists()) {//直接判斷真實(shí)的m3u8文件是否存在翎蹈,存在則讀取
file = File(path, "original.m3u8")
}
Log.d(TAG, "getFileContent---${file.name}")
val list = file.readLines().filter { !it.startsWith("#") }//讀取m3u8文件
if (list.size > 1) {//直接的m3u8的ts鏈接
entity.tsSize = list.size
entity.toFile()
if (file != realM3U8File) {
file.copyTo(realM3U8File)
}
val m3u8ListFile = File(path, "m3u8.list")
list.forEach {
val ts = if (!it.startsWith("/")) {
url.substring(0, url.lastIndexOf("/") + 1) + it
} else {
"${uri.scheme}://${uri.host}$it"
}
m3u8ListFile.appendText("$ts\n")
}
val localPlaylist = File(path, "localPlaylist.m3u8")
file.readLines().forEach {
var str = it
if (!str.startsWith("#")) {
str = if (str.contains("/")) {
".ts${it.substring(it.lastIndexOf("/"))}"
} else {
".ts/$it"
}
}
localPlaylist.appendText("$str\n")
}
Log.d(TAG, "start--->$entity")
} else {//重定向
val newUrl = list[0]
entity.redirectUrl = if (newUrl.startsWith("/")) {
"${uri.scheme}://${uri.host}$newUrl"
} else {
url.substring(0, url.lastIndexOf("/") + 1) + newUrl
}
entity.toFile()
downloadM3U8File(path, entity)
}
}
}
在以上代碼中,從一個(gè)最初始的url開始捌臊,下載對(duì)應(yīng)的m3u8文件杨蛋,分析如果這個(gè)m3u8是最終的ts流,將ts流的完整url寫入m3u8.list
這個(gè)文件理澎,之后下載的都從這個(gè)文件進(jìn)行下,如果這個(gè)m3u8需要重定向,那么就重組鏈接曙寡,再一次下載糠爬,以此循環(huán)得到最終的ts流,同時(shí)举庶,在獲取到最終ts流到時(shí)候执隧,會(huì)構(gòu)造一個(gè)本地可以播放到m3u8文件localPlaylist.m3u8
,當(dāng)視頻下載完成之后就可以通過這個(gè)文件打開本地的播放器進(jìn)行播放
下載ts文件
之前已經(jīng)獲取到真實(shí)的ts路徑了户侥,并且將這些路徑保存在m3u8.list
文件里面了镀琉,所以之后就是通過這個(gè)文件里面的路徑,使用okdownload
進(jìn)行批量下載了蕊唐,具體實(shí)現(xiàn)如下
internal object M3U8Downloader {
private val downloadList = arrayListOf<String>()
private const val TAG = "---M3U8Downloader---"
//清楚所有任務(wù)
fun clear() {
downloadList.clear()
}
//批下載
fun bunchDownload(path: File) {
val config = FileDownloader.getConfigFile(path)
Log.d(TAG, "config==>${config.readText()}")
val entity = parseJsonToVideoDownloadEntity(config.readText())
if (entity == null) {//獲取到的實(shí)體類為空的忽略
Log.d(TAG, "entity==null${config.readText()}")
return
}
//如果狀態(tài)是刪除的就忽略
if (entity.status == DELETE) {
path.deleteRecursively()
return
}
//避免重復(fù)進(jìn)入下載
if (downloadList.contains(entity.originalUrl)) {
Log.d(TAG, "contains")
return
}
var lastCallback = 0L
val CURRENT_PROGRESS = entity.originalUrl.hashCode()
val speedCalculator = SpeedCalculator()
val listener = object : DownloadListener1() {
override fun taskStart(
task: DownloadTask, model: Listener1Assist.Listener1Model
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
}
override fun taskEnd(
task: DownloadTask, cause: EndCause, realCause: Exception?,
model: Listener1Assist.Listener1Model
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
}
override fun progress(
task: DownloadTask, currentOffset: Long, totalLength: Long
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
val preOffset = (task.getTag(CURRENT_PROGRESS) as Long?) ?: 0
speedCalculator.downloading(currentOffset - preOffset)
val now = System.currentTimeMillis()
if (now - lastCallback > 1000) {
entity.currentSpeed = speedCalculator.speed() ?: ""
entity.status = DOWNLOADING
entity.toFile()
FileDownloader.downloadCallback.postValue(entity)
lastCallback = now
}
task.addTag(CURRENT_PROGRESS, currentOffset)
}
override fun connected(
task: DownloadTask, blockCount: Int, currentOffset: Long, totalLength: Long
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
}
override fun retry(task: DownloadTask, cause: ResumeFailedCause) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
}
}
Log.d(TAG, "bunchDownload")
val m3u8ListFile = File(path, "m3u8.list")
var urls = m3u8ListFile.readLines()
var times = 5
while (times > 0 && urls.size != entity.tsSize) {//如果還有重試機(jī)會(huì)且ts數(shù)量還不完全對(duì)的話,等待100ms
urls = m3u8ListFile.readLines()
times--
Thread.sleep(100)
}
val tsDirectory = File(path, ".ts")
if (!tsDirectory.exists()) {
tsDirectory.mkdir()
}
val builder = DownloadContext.QueueSet()
.setParentPathFile(tsDirectory)
.setMinIntervalMillisCallbackProcess(1000)
.setPassIfAlreadyCompleted(true)
.commit()
Log.d(TAG, "ts.size===>${urls.size}")
urls.forEachIndexed { index, url ->
builder.bind(url).addTag(1, index)
}
val downloadContext = builder.setListener(object : DownloadContextListener {
override fun taskEnd(
context: DownloadContext, task: DownloadTask, cause: EndCause,
realCause: Exception?, remainCount: Int
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
if (entity.downloadContext == null) {
entity.downloadContext = context
}
if (context.isStarted && cause == EndCause.COMPLETED) {
val progress = 1 - (remainCount * 1.0) / urls.size
entity.status = DOWNLOADING
entity.currentProgress = progress
entity.fileSize += task.file?.length() ?: 0
entity.currentSize += task.file?.length() ?: 0
val now = System.currentTimeMillis()
if (now - lastCallback > 1000) {
FileDownloader.downloadCallback.postValue(entity)
lastCallback = now
}
entity.toFile()
}
}
override fun queueEnd(context: DownloadContext) {
Log.d(TAG, "queueEnd")
if (entity.downloadContext == null) {
entity.downloadContext = context
}
when (entity.currentProgress) {
1.0 -> entity.status = COMPLETE
0.0 -> entity.status = ERROR
else -> entity.status = PAUSE
}
entity.toFile()
FileDownloader.downloadCallback.postValue(entity)
FileDownloader.subUseProgress(entity.originalUrl)//已使用的線程數(shù)減少
}
}).build()
entity.downloadContext = downloadContext
entity.startDownload = { downloadContext.startOnSerial(listener) }
downloadContext.startOnSerial(listener)
FileDownloader.addUseProgress(entity.originalUrl)//已使用的線程數(shù)增加
downloadList.add(entity.originalUrl)
}
}
通過以上代碼就可以進(jìn)行批量下載的實(shí)現(xiàn)了
MP4下載
既然對(duì)于復(fù)雜的m3u8都能下載,那么單個(gè)文件的mp4之類的肯定要支持下載的屋摔,以下為mp4的下載方案
internal object SingleVideoDownloader {
private val downloadList = arrayListOf<String>()
private const val TAG = "SingleVideoDownloader"
//清理所有任務(wù)
fun clear() {
downloadList.clear()
}
//下載任務(wù)的初始化
fun initConfig(entity: VideoDownloadEntity): File {
val config = FileDownloader.getConfigFile(entity.originalUrl)
if (!config.exists()) {
if (entity.createTime == 0L) {
entity.createTime = System.currentTimeMillis()
}
entity.status = PREPARE
entity.fileSize = 0
entity.currentSize = 0
entity.toFile()
Log.d(TAG, "config==>${config.readText()}")
FileDownloader.downloadCallback.postValue(entity)
}
return config
}
//下載任務(wù)的入口
fun fileDownloader(entity: VideoDownloadEntity) {
val path = FileDownloader.getDownloadPath(entity.originalUrl)
if (entity.status == DELETE) {//如果是刪除狀態(tài)的則忽略
path.deleteRecursively()
return
}
if (downloadList.contains(entity.originalUrl)) {//避免重復(fù)下載
Log.d(TAG, "contains---${entity.originalUrl},${entity.name}")
return
}
entity.status = PREPARE
entity.fileSize = 0
entity.currentSize = 0
FileDownloader.downloadCallback.postValue(entity)
var lastCallback = 0L
val CURRENT_PROGRESS = entity.originalUrl.hashCode()
val speedCalculator = SpeedCalculator()
Log.d(TAG, "fileDownloader")
val fileName = if (entity.name.isNotEmpty()) {//主標(biāo)題有
if (entity.subName.isNotEmpty()) {//副標(biāo)題也有
"${entity.name}-${entity.subName}.mp4"
} else {//只有主標(biāo)題
"${entity.name}.mp4"
}
} else {//沒有主標(biāo)題
if (entity.subName.isNotEmpty()) {//只有副標(biāo)題
"${entity.subName}.mp4"
} else {//標(biāo)題都沒有
"index.mp4"
}
}
val downloadFile = File(path, fileName)
Log.d(TAG, "downloadFile===>${downloadFile.absolutePath}")
val task = DownloadTask.Builder(entity.originalUrl, downloadFile.parentFile)
.setFilename(downloadFile.name)
.setPassIfAlreadyCompleted(true)
.setMinIntervalMillisCallbackProcess(1000)
.setConnectionCount(3)
.build()
task.enqueue(object : DownloadListener1() {
override fun taskStart(task: DownloadTask, model: Listener1Assist.Listener1Model) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
Log.d(TAG, "taskStart-->")
entity.status = PREPARE
entity.fileSize = 0
entity.currentSize = 0
entity.toFile()
FileDownloader.downloadCallback.postValue(entity)
}
override fun taskEnd(
task: DownloadTask, cause: EndCause, realCause: Exception?,
model: Listener1Assist.Listener1Model
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
Log.d(TAG, "taskEnd-->${cause.name},${realCause?.message}")
when (cause) {
EndCause.COMPLETED -> entity.status = COMPLETE
EndCause.CANCELED -> {
entity.status = PAUSE
entity.startDownload = {
fileDownloader(entity)
}
}
else -> {
entity.status = ERROR
entity.startDownload = {
fileDownloader(entity)
}
}
}
entity.toFile()
FileDownloader.downloadCallback.postValue(entity)
downloadList.remove(entity.originalUrl)
FileDownloader.subUseProgress(task.url)//已使用的線程數(shù)減少
}
override fun progress(task: DownloadTask, currentOffset: Long, totalLength: Long) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
val preOffset = (task.getTag(CURRENT_PROGRESS) as Long?) ?: 0
speedCalculator.downloading(currentOffset - preOffset)
entity.currentSize = currentOffset
val now = System.currentTimeMillis()
if (now - lastCallback > 1000) {
entity.currentProgress = (currentOffset * 1.0) / (totalLength * 1.0)
entity.currentSpeed = speedCalculator.speed() ?: ""
entity.status = DOWNLOADING
entity.toFile()
FileDownloader.downloadCallback.postValue(entity)
lastCallback = now
}
task.addTag(CURRENT_PROGRESS, currentOffset)
}
override fun connected(
task: DownloadTask, blockCount: Int, currentOffset: Long, totalLength: Long
) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
entity.currentSize += currentOffset
entity.fileSize += totalLength
entity.toFile()
FileDownloader.downloadCallback.postValue(entity)
}
override fun retry(task: DownloadTask, cause: ResumeFailedCause) {
if (entity.downloadTask == null) {
entity.downloadTask = task
}
}
})
entity.downloadTask = task
downloadList.add(entity.originalUrl)
FileDownloader.addUseProgress(entity.originalUrl)//已使用的線程數(shù)增加
}
}
多任務(wù)管理
以上代碼出現(xiàn)了不少的FileDownloader
這個(gè)類,這個(gè)類的主要作用是進(jìn)行多任務(wù)的管理替梨,實(shí)現(xiàn)順序任務(wù)下載钓试,限制同時(shí)下載數(shù)量等功能,具體代碼如下:
object FileDownloader {
private val TAG = "FileDownloader"
val downloadCallback = MutableLiveData<VideoDownloadEntity>()//下載進(jìn)度回調(diào)
private var MAX_PROGRESS = -1
//最終計(jì)算結(jié)果至少為1
get() {
if (field == -1) {
field = Runtime.getRuntime().availableProcessors() / 2//可用線程數(shù)的一半
if (Build.VERSION.SDK_INT < 23) {//如果小于Android6的副瀑,可用線程數(shù)再減2
field -= 2
}
}
if (field > 5) {//最多只能有5個(gè)并行
field = 5
}
if (field <= 0) {//最少也要有1個(gè)任務(wù)
field = 1
}
return field
}
private var useProgress = 0
//已使用的線程數(shù),始終大于0
set(value) {
if (value >= 0) {
field = value
}
}
private var downloadingList = arrayListOf<String>()//下載中的列表弓熏,為統(tǒng)計(jì)線程使用
private var waitDownloadList = arrayListOf<String>()//等待下載的url列表
private val downloadList = arrayListOf<VideoDownloadEntity>()//排隊(duì)列表
private val waitList = arrayListOf<VideoDownloadEntity>()//等待下載的隊(duì)列
private var wait = false//m3u8等待狀態(tài)
/**
* 停止全部任務(wù)
*/
fun clearAllDownload() {
OkDownload.with().downloadDispatcher().cancelAll()
downloadingList.clear()
waitDownloadList.clear()
downloadList.clear()
waitList.clear()
M3U8ConfigDownloader.clear()
M3U8Downloader.clear()
SingleVideoDownloader.clear()
MAX_PROGRESS = -1
useProgress = 0
}
/**
* 減少已使用線程數(shù)
*/
fun subUseProgress(url: String) {
if (downloadingList.contains(url)) {
useProgress--
downloadingList.remove(url)
Log.d(TAG, "釋放線程---$useProgress")
if (downloadList.isNotEmpty()) {
Log.d(TAG, "subUseProgress---新增任務(wù)")
waitDownloadList.removeAt(0)
downloadVideo(downloadList.removeAt(0))
}
}
}
/**
* 增加使用線程數(shù)
*/
fun addUseProgress(url: String) {
if (!downloadingList.contains(url)) {
useProgress++
downloadingList.add(url)
}
}
/**
* 獲取最頂層的下載目錄
*/
@JvmStatic
fun getBaseDownloadPath(): File {
val file = File(Environment.getExternalStorageDirectory(), "m3u8Downloader")
if (!file.exists()) {
file.mkdirs()
}
return file
}
/**
* 獲取根據(jù)鏈接得到的下載存儲(chǔ)路徑
*/
@JvmStatic
fun getDownloadPath(url: String): File {
val file = File(getBaseDownloadPath(), md5(url))
if (!file.exists()) {
file.mkdir()
}
return file
}
/**
* 獲取相關(guān)配置文件
*/
@JvmStatic
fun getConfigFile(url: String): File {
val path = getDownloadPath(url)
return File(path, "video.config")
}
/**
* 獲取相關(guān)配置文件
*/
@JvmStatic
fun getConfigFile(path: File): File {
return File(path, "video.config")
}
/**
* 下載的入口
*/
@JvmStatic
fun downloadVideo(entity: VideoDownloadEntity) {
if (entity.status == DELETE) {
return
}
if (entity.originalUrl.endsWith(".m3u8")) {
downloadM3U8File(entity)
} else {
downloadSingleVideo(entity)
}
}
/**
* 下載但文件入口
*/
@JvmStatic
private fun downloadSingleVideo(entity: VideoDownloadEntity) {
if (entity.status == DELETE) {//刪除狀態(tài)的忽略
Log.d(TAG, "downloadSingleVideo---DELETE")
return
}
if (useProgress < MAX_PROGRESS) {//還有可用的線程數(shù)
SingleVideoDownloader.fileDownloader(entity)//進(jìn)入下載
Log.d(TAG, "-----useProgress===>$useProgress")
} else {//沒有可用線程的時(shí)候就添加到等待隊(duì)列
SingleVideoDownloader.initConfig(entity)//初始化一下下載任務(wù)
//不是下載中的內(nèi)容,且沒有在等待
if (!downloadingList.contains(entity.originalUrl) && !waitDownloadList.contains(entity.originalUrl)) {
downloadList.add(entity)
waitDownloadList.add(entity.originalUrl)
Log.d(TAG, "addDownloadList---${entity.originalUrl}")
entity.status = PREPARE
downloadCallback.postValue(entity)
} else {
if (entity.status == NO_START || entity.status == ERROR || entity.status == PAUSE) {
//如果要下載的內(nèi)容是等待中的,但是狀態(tài)還沒有修正過來糠睡,則修正狀態(tài)
entity.status = PREPARE
downloadCallback.postValue(entity)
}
Log.d(TAG, "下載中或等待中的文件")
}
}
}
@JvmStatic
private fun downloadM3U8File(entity: VideoDownloadEntity) {
if (entity.status == DELETE) {//刪除狀態(tài)的忽略
Log.d(TAG, "downloadM3U8File---DELETE")
return
}
Log.d(TAG, "$wait--downloadM3U8File--${entity.originalUrl}")
thread {
if (wait) {//如果有在獲取真實(shí)ts的內(nèi)容則添加到等待隊(duì)列
Log.d(TAG, "addWaiting")
waitList.add(entity)
return@thread
}
wait = true
val file = M3U8ConfigDownloader.start(entity)//準(zhǔn)備下載列表
if (useProgress < MAX_PROGRESS) {//還有可用的線程數(shù)
if (file != null) {//需要下載
var times = 50
Log.d(TAG, "file.exists()==>${file.exists()}")
while (!file.exists() && times > 0) {//如果文件還不存在則等待100ms
Log.d(TAG, "waiting...")
Thread.sleep(100)
times--
}
if (file.exists()) {//如果文件存在了則開始下載
M3U8Downloader.bunchDownload(getDownloadPath(entity.originalUrl))
}
Log.d(TAG, "${file.exists()}-----useProgress===>$useProgress")
} else {
Log.d(TAG, "file===null")
}
} else {//沒有可用線程的時(shí)候就添加到等待隊(duì)列
//不是下載中的內(nèi)容,且沒有在等待
if (!downloadingList.contains(entity.originalUrl) &&
!waitDownloadList.contains(entity.originalUrl)
) {//添加到任務(wù)隊(duì)列
downloadList.add(entity)
waitDownloadList.add(entity.originalUrl)
Log.d(TAG, "addDownloadList---${entity.originalUrl}")
entity.status = PREPARE
downloadCallback.postValue(entity)
} else {
Log.d(TAG, "下載中或等待中的文件")
if (entity.status == NO_START || entity.status == ERROR || entity.status == PAUSE) {
//如果要下載的內(nèi)容是等待中的挽鞠,但是狀態(tài)還沒有修正過來,則修正狀態(tài)
entity.status = PREPARE
downloadCallback.postValue(entity)
}
}
}
wait = false
if (waitList.isNotEmpty()) {
//有等待獲取真實(shí)ts流的則繼續(xù)回調(diào)
Log.d(TAG, "removeWaiting")
downloadM3U8File(waitList.removeAt(0))
}
}
}
}
使用測試
編寫完下載庫,下面就進(jìn)行測試了
下載列表的item
具體代碼如下:
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:paddingStart="15dp"
android:paddingTop="8dp"
android:paddingEnd="15dp"
android:paddingBottom="8dp">
<TextView
android:id="@+id/download"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:background="@drawable/shape_download_prepare"
android:paddingStart="15dp"
android:paddingTop="5dp"
android:paddingEnd="15dp"
android:paddingBottom="5dp"
android:text="@string/btn_download"
android:textColor="@color/blue"
android:textSize="12sp"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintTop_toTopOf="parent" />
<TextView
android:id="@+id/title"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:layout_marginStart="15dp"
android:ellipsize="end"
android:maxLines="1"
android:textSize="18sp"
app:layout_constraintEnd_toStartOf="@id/download"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent"
tools:text="@string/app_name" />
<TextView
android:id="@+id/current_size"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginTop="5dp"
android:textSize="12sp"
app:layout_constraintStart_toStartOf="@id/title"
app:layout_constraintTop_toBottomOf="@id/title"
tools:text="201.37MB" />
<TextView
android:id="@+id/speed"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginStart="15dp"
android:textSize="12sp"
app:layout_constraintBottom_toBottomOf="@id/current_size"
app:layout_constraintStart_toEndOf="@id/current_size"
tools:text="90.5%|251.37kB/s" />
<TextView
android:id="@+id/url"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:ellipsize="end"
android:maxLines="2"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="@id/title"
app:layout_constraintTop_toBottomOf="@id/speed"
tools:text="https://qq.com-ok-qq.com/20191015/24619_fc6ad1d6/index.m3u8" />
</androidx.constraintlayout.widget.ConstraintLayout>
Adapter的編寫
class VideoDownloadAdapter(private val list: MutableList<VideoDownloadEntity>) :
RecyclerView.Adapter<VideoDownloadAdapter.ViewHolder>() {
override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): ViewHolder {
return ViewHolder(
LayoutInflater.from(parent.context).inflate(
R.layout.item_download_list, parent, false
)
)
}
override fun getItemCount() = list.size
/**
* 避免出現(xiàn)整個(gè)item閃爍
*/
override fun onBindViewHolder(holder: ViewHolder, position: Int, payloads: MutableList<Any>) {
if (payloads.isNullOrEmpty()) {
super.onBindViewHolder(holder, position, payloads)
} else {
holder.updateProgress(list[position])
}
}
override fun onBindViewHolder(holder: ViewHolder, position: Int) {
holder.setData(list[position])
}
class ViewHolder(private val view: View) : RecyclerView.ViewHolder(view) {
private val title = view.findViewById<TextView>(R.id.title)
private val currentSize = view.findViewById<TextView>(R.id.current_size)
private val speed = view.findViewById<TextView>(R.id.speed)
private val url = view.findViewById<TextView>(R.id.url)
private val download = view.findViewById<TextView>(R.id.download)
/**
* 設(shè)置數(shù)據(jù)
*/
@SuppressLint("SetTextI18n")
fun setData(data: VideoDownloadEntity?) {
if (data == null) {
return
}
val context = view.context
url.text = data.originalUrl
val name = if (data.name.isNotEmpty()) {
if (data.subName.isNotEmpty()) {
"${data.name}(${data.subName})"
} else {
data.name
}
} else {
if (data.subName.isNotEmpty()) {
"${context.getString(R.string.unknow_movie)}(${data.subName})"
} else {
context.getString(R.string.unknow_movie)
}
}
title.text = name
updateProgress(data)
}
/**
* 進(jìn)度更新
*/
@SuppressLint("SetTextI18n")
fun updateProgress(data: VideoDownloadEntity) {
if (data.originalUrl.endsWith(".m3u8") || data.status == COMPLETE) {
currentSize.text =
getSizeUnit(data.currentSize.toDouble())
} else {
currentSize.text =
"${getSizeUnit(data.currentSize.toDouble())}/${getSizeUnit(
data.fileSize.toDouble()
)}"
}
speed.text =
"${DecimalFormat("#.##%").format(data.currentProgress)}|${data.currentSpeed}"
val context = view.context
//狀態(tài)邏輯處理
when (data.status) {
NO_START -> {
download.setTextColor(ContextCompat.getColor(context, R.color.blue))
download.background =
ContextCompat.getDrawable(context, R.drawable.shape_download_prepare)
download.setText(R.string.btn_download)
download.isVisible = true
speed.isVisible = false
currentSize.isVisible = false
currentSize.setText(R.string.wait_download)
download.setOnClickListener {
if (data.startDownload != null) {
data.startDownload!!.invoke()
} else {
FileDownloader.downloadVideo(data)
}
}
}
DOWNLOADING -> {
currentSize.isVisible = true
speed.isVisible = true
speed.setTextColor(ContextCompat.getColor(speed.context, R.color.blue))
download.isVisible = true
download.setText(R.string.pause)
download.setOnClickListener {
data.downloadContext?.stop()
data.downloadTask?.cancel()
}
download.setTextColor(ContextCompat.getColor(context, R.color.white))
download.background =
ContextCompat.getDrawable(context, R.drawable.shape_blue_btn)
}
PAUSE -> {
currentSize.isVisible = true
download.setTextColor(ContextCompat.getColor(context, R.color.white))
download.background =
ContextCompat.getDrawable(context, R.drawable.shape_blue_btn)
download.isVisible = true
download.setText(R.string.go_on)
download.setOnClickListener {
if (data.startDownload != null) {
data.startDownload!!.invoke()
} else {
FileDownloader.downloadVideo(data)
}
}
speed.isVisible = true
speed.setText(R.string.already_paused)
speed.setTextColor(ContextCompat.getColor(speed.context, R.color.red))
}
COMPLETE -> {
currentSize.isVisible = true
download.isVisible = false
speed.isVisible = false
}
PREPARE -> {
currentSize.isVisible = true
download.setText(R.string.prepareing)
currentSize.setText(R.string.wait_download)
download.isVisible = true
download.setOnClickListener {
if (data.startDownload != null) {
data.startDownload!!.invoke()
} else {
FileDownloader.downloadVideo(data)
}
}
download.setTextColor(ContextCompat.getColor(context, R.color.blue))
download.background =
ContextCompat.getDrawable(context, R.drawable.shape_download_prepare)
speed.isVisible = false
}
ERROR -> {
currentSize.isVisible = false
speed.isVisible = false
download.isVisible = true
download.setText(R.string.retry)
download.setOnClickListener {
if (data.startDownload != null) {
data.startDownload!!.invoke()
} else {
FileDownloader.downloadVideo(data)
}
}
download.setTextColor(ContextCompat.getColor(context, R.color.white))
download.background =
ContextCompat.getDrawable(context, R.drawable.shape_blue_btn)
}
}
}
}
}
由于是下載列表信认,如果頻繁刷新是會(huì)導(dǎo)致整個(gè)item不斷閃爍的串稀,所以在下載庫那邊也有處理了1秒鐘才發(fā)出一次進(jìn)度更新,而在接收的時(shí)候一定要注意狮杨,需要重寫onBindViewHolder(holder: ViewHolder, position: Int, payloads: MutableList<Any>)
這個(gè)函數(shù)母截,通知adapter更新的時(shí)候應(yīng)該調(diào)用notifyItemChanged(int position, @Nullable Object payload)
這樣可以避免整個(gè)item閃爍,實(shí)現(xiàn)只更新局部控件的效果
Activity的實(shí)現(xiàn)
@RuntimePermissions
class MainActivity : AppCompatActivity() {
private lateinit var adapter: VideoDownloadAdapter
private val videoList = arrayListOf<VideoDownloadEntity>()
private val tempList = arrayListOf<String>()
private val gson = GsonBuilder().create()
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
initListView()
initListWithPermissionCheck()
//接收進(jìn)度通知
FileDownloader.downloadCallback.observe(this, Observer {
onProgress(it)
})
//新建下載
add.setOnClickListener {
newDownload()
}
}
private fun initListView() {
adapter = VideoDownloadAdapter(videoList)
list.adapter = adapter
}
@NeedsPermission(
Manifest.permission.READ_EXTERNAL_STORAGE,
Manifest.permission.WRITE_EXTERNAL_STORAGE
)
fun initList() {
thread {//在線程中處理橄教,防止ANR
FileDownloader.getBaseDownloadPath().listFiles().forEach {
val file = File(it, "video.config")
if (file.exists()) {
val text = file.readText()
if (text.isNotEmpty()) {
val data = gson.fromJson<VideoDownloadEntity>(
text,
VideoDownloadEntity::class.java
)
if (data != null) {
if (data.status == DELETE) {
it.deleteRecursively()
} else if (!tempList.contains(data.originalUrl)) {
videoList.add(data)
tempList.add(data.originalUrl)
}
}
}
}
}
runOnUiThread {
//主線程通知刷新布局
adapter.notifyDataSetChanged()
}
videoList.sort()
//依次添加下載隊(duì)列
videoList.filter { it.status == DOWNLOADING }.forEach {
FileDownloader.downloadVideo(it)
}
videoList.filter { it.status == PREPARE }.forEach {
FileDownloader.downloadVideo(it)
}
videoList.filter { it.status == NO_START }.forEach {
FileDownloader.downloadVideo(it)
}
}
}
@OnPermissionDenied(
Manifest.permission.READ_EXTERNAL_STORAGE,
Manifest.permission.WRITE_EXTERNAL_STORAGE
)
fun onDenied() {
toast(R.string.need_permission_tips)
}
private fun toast(@StringRes msg: Int) {
Toast.makeText(this, msg, Toast.LENGTH_LONG).show()
}
override fun onRequestPermissionsResult(
requestCode: Int, permissions: Array<out String>, grantResults: IntArray
) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults)
onRequestPermissionsResult(requestCode, grantResults)
}
private fun onProgress(entity: VideoDownloadEntity) {
for ((index, item) in videoList.withIndex()) {
if (item.originalUrl == entity.originalUrl) {
videoList[index].status = entity.status
videoList[index].currentSize = entity.currentSize
videoList[index].currentSpeed = entity.currentSpeed
videoList[index].currentProgress = entity.currentProgress
videoList[index].fileSize = entity.fileSize
videoList[index].tsSize = entity.tsSize
videoList[index].downloadContext = entity.downloadContext
videoList[index].downloadTask = entity.downloadTask
videoList[index].startDownload = entity.startDownload
adapter.notifyItemChanged(index, 0)
break
}
}
}
private fun newDownload() {
val editText = EditText(this)
editText.setHint(R.string.please_input_download_address)
val downloadDialog = AlertDialog.Builder(this)
.setView(editText)
.setTitle(R.string.new_download)
.setPositiveButton(R.string.ok) { dialog, _ ->
if (editText.text.isNullOrEmpty()) {
toast(R.string.please_input_download_address)
return@setPositiveButton
}
val url = editText.text.toString()
if (tempList.contains(url)) {
toast(R.string.already_download)
dialog.dismiss()
return@setPositiveButton
}
val name = if (url.contains("?")) {
url.substring(url.lastIndexOf("/") + 1, url.indexOf("?"))
} else {
url.substring(url.lastIndexOf("/") + 1)
}
val entity = VideoDownloadEntity(url, name)
entity.toFile()
videoList.add(0, entity)
adapter.notifyItemInserted(0)
FileDownloader.downloadVideo(entity)
}
.setNegativeButton(R.string.cancle) { dialog, _ ->
dialog.dismiss()
}.create()
downloadDialog.show()
}
}