說明：使用axios方式上傳僻澎，文件不能過大，因為過多的連續(xù)Ajax請求會使后臺崩潰十饥，接口報錯窟勃；所以使用分段上傳的方式，減輕服務(wù)器的壓力逗堵。其實就是將 文件變小秉氧，也就是通過 文件資源分塊 后再上傳。
問題 1：誰負責(zé)資源分塊蜒秤？誰負責(zé)資源整合汁咏？
前端負責(zé)分塊，服務(wù)端負責(zé)整合.
問題 2：前端怎么對資源進行分塊作媚？
首先是選擇上傳的文件資源梆暖，接著就可以得到對應(yīng)的文件對象 File，而 File.prototype.slice 方法可以實現(xiàn)資源的分塊掂骏，當(dāng)然也有人說是 Blob.prototype.slice 方法，因為 Blob.prototype.slice === File.prototype.slice.
問題 3：服務(wù)端怎么知道什么時候要整合資源厚掷？如何保證資源整合的有序性弟灼？
由于前端會將資源分塊级解，然后單獨發(fā)送請求，也就是說田绑，原來 1 個文件對應(yīng) 1 個上傳請求勤哗，現(xiàn)在可能會變成 1 個文件對應(yīng) n 個上傳請求，所以前端可以基于 Promise.all 將這多個接口整合掩驱，上傳完成在發(fā)送一個合并的請求芒划，通知服務(wù)端進行合并。
在發(fā)送請求資源時欧穴，前端會定好每個文件對應(yīng)的序號民逼，并將當(dāng)前分塊、序號以及文件 hash 等信息一起發(fā)送給服務(wù)端涮帘，服務(wù)端在進行合并時拼苍，通過序號進行依次合并即可。
此示例是純前端代碼调缨，不涉及后端疮鲫。
錄屏.gif
第一步：使用input或者antd_upload獲取文件
image.png
第二步：調(diào)接口獲取文件段數(shù)，分段列表和分段尺寸弦叶；使用slice方法俊犯，分段讀取文件為blob
let dataMsg = await createMultipart({
fileSize: file.size, // 傳參數(shù)
filename: file.name
}).then(
(rem) => {
return rem.data;
},
(err) => {
return upFailed(file, onUpload); // 如果接口報錯，使用upFailed方法處理
}
);
let urlList = dataMsg?.parts || []; // 分段列表
let DEFAULT_SIZE = dataMsg?.partSize; // 分段尺寸
for (let i = 0; i < urlList.length; i++) {
let url = urlList[i]['url'];
let fname = encodeURIComponent(file.name);
let start = i * DEFAULT_SIZE;
let stepFile;
if (i === urlList.length - 1) {
// 使用slice方法伤哺，分段讀取文件為blob
stepFile = file.slice(start, -1); // 如果是最后一段燕侠，直接截取剩下的所有內(nèi)容
} else {
stepFile = file.slice(start, start + DEFAULT_SIZE); // 分割文件
}
urlList[i]['stepFile'] = stepFile;
urlList[i]['fname'] = fname;
urlList[i]['uid'] = file.uid;
}
urlList已準備好
image.png
數(shù)據(jù)說明： {
fname: '使用encodeURIComponent 編碼過的文件名',
partNumber: '段數(shù)序號,合并時候使用',
stepFile: '截取的文件'
uid: 'antd組件生成的文件唯一值',
url: '上傳該段文件的路徑'
};
第三步：循環(huán)urlList，上傳每一段文件
準備工作：單個文件上傳方法
const detalItem = ({ url, stepFile, fname, partNumber }) => {
return new Promise((resolve, reject) => {
fileAxios({
url,
method: 'PUT',
data: stepFile,
headers: {
'Content-Type': '',
'Content-disposition': `filename*=utf-8\'zh_cn\'${fname}`
}
})
.then((res) => {
let str = res.headers.etag.split('"').join('');
resolve({ eTag: str, partNumber });
})
.catch((err) => {
reject({ eTag: '', partNumber });
});
});
};
準備工作：并發(fā)上傳默责，控制每次上傳的接口數(shù)量贬循，防止上傳接口數(shù)量過多，瀏覽器崩潰桃序。
參數(shù)說明：
poolLimit（數(shù)字類型）：表示限制的并發(fā)數(shù)杖虾；
array（數(shù)組類型）：表示任務(wù)數(shù)組；
iteratorFn（函數(shù)類型）：表示迭代函數(shù)媒熊，用于實現(xiàn)對每個任務(wù)項進行處理奇适，該函數(shù)會返回一個 Promise 對象或異步函數(shù)；
onUpload: 進度條
async function asyncPool(poolLimit, array, iteratorFn, onUpload) {
const ret = []; // 存儲所有的異步任務(wù)
const executing = []; // 存儲正在執(zhí)行的異步任務(wù)
for (const item of array) {
// 結(jié)束運行
if (endExecution.end && endExecution.uid === item?.uid) {
return;
}
--------重點開始---------------
// 調(diào)用iteratorFn函數(shù)創(chuàng)建異步任務(wù)
const p = Promise.resolve().then(() => iteratorFn(item, array));
ret.push(p); // 保存新的異步任務(wù)
// 當(dāng)poolLimit值小于或等于總?cè)蝿?wù)個數(shù)時芦鳍，進行并發(fā)控制
if (poolLimit <= array.length) {
// 當(dāng)任務(wù)完成后嚷往，從正在執(zhí)行的任務(wù)數(shù)組中移除已完成的任務(wù)
// e 是個promise 。其后續(xù)的then接受的回調(diào)是 “自殺”柠衅，給executing 這個數(shù)組騰出空位
const e = p.then(() => executing.splice(executing.indexOf(e), 1));
executing.push(e); // 保存正在執(zhí)行的異步任務(wù)
if (executing.length >= poolLimit) {
await Promise.race(executing); // 等待較快的任務(wù)執(zhí)行完成
}
}
--------重點結(jié)束---------------
// 進度條
onUpload &&
onUpload({
loaded: ret.length < array.length ? ret.length : ret.length - 1, // 等到接口合并完成皮仁，再返回100%
total: array.length,
uid: item['uid'],
endAction: endAction // 如果用戶刪除文件，調(diào)用此函數(shù)，結(jié)束文件上傳
}); // 進度條
}
return Promise.all(ret); // 集合多個返回結(jié)果
}
整合方法贷祈，開始上傳
let etags = [];
try {
etags = await asyncPool(5, urlList, detalItem, onUpload); // 重點
} catch {
// 上傳失敗
etags = [];
endExecution.end = true;
endExecution.uid = file?.uid;
file['url'] = '';
file['link'] = '';
file['attachmentID'] = '';
return upFailed(file, onUpload);
}
請求中趋急，保證5條并發(fā)數(shù)，如果5條中有請求結(jié)束了势誊，自動補上
image.png
創(chuàng)建請求呜达，請求全部發(fā)出，結(jié)束后合并文件
image.png
文件上傳完的結(jié)果粟耻，etags查近。
eTag是每段文件的唯一值，
partNumber： 文件順序挤忙。后端根據(jù)這個數(shù)據(jù)表來合并文件霜威，避免順序亂了。
image.png
第四步： 通知后端饭玲，合并文件
let params = {
attachmentID: dataMsg?.attachmentID,
uploadID: dataMsg?.uploadID
};
if (endExecution.end && endExecution.uid === file?.uid) {
console.log('刪除文件侥祭，結(jié)束上傳，調(diào)用結(jié)束上傳接口茄厘，后端清除已經(jīng)上傳的數(shù)據(jù)');
cancelMultipart(params);
file['url'] = '';
file['link'] = '';
file['attachmentID'] = '';
return { file: file, upResult: '' };
}
--------重點開始---------------
// 調(diào)接口 傳參數(shù)
result = await completeMultipart({
...params,
etags
});
if (etags) {
onUpload &&
onUpload({
loaded: 100,
total: 100,
uid: file['uid'],
endAction: endAction
}); // 進度條
}
let presignedURL = result?.data?.presignedURL;
// console.log('result', result, 'presignedURL', presignedURL);
file['url'] = presignedURL;
file['link'] = presignedURL;
file['attachmentID'] = dataMsg?.attachmentID;
file['status'] = 'done';
--------重點結(jié)束---------------
return { file: file, upResult: '' };
附加功能：
1 返回進度條onUpload矮冬，原理： 當(dāng)前發(fā)出去的請求數(shù)，除以總條數(shù)
2 結(jié)束請求endAction次哈，應(yīng)用場景胎署，文件正在上傳中，刪除文件窑滞，結(jié)束接口調(diào)用
全部代碼：
React上傳組件：
import React, { Component } from 'react';
import { Upload, Progress, Tooltip, Modal } from 'antd';
const { Dragger } = Upload;
export default class List extends Component {
constructor(props) {
super(props);
this.state = {
fileList: [
// {
// uid: '-1',
// name: 'image.png',
// status: 'done',
// url:
// 'https://zos.alipayobjects.com/rmsportal/jkjgkEfvpUPVyRjUImniVslZfWPnJuuZ.png'
// }
],
visible: false,
upLoading: false
};
this.formRef = React.createRef();
this.littleRef = React.createRef();
this.departmentRef = React.createRef();
this.smalledepartmentRef = React.createRef();
}
// 進度條
handleProgress = (progressEvent) => {
const num = (progressEvent.loaded / progressEvent.total) * 100;
let percent = num >= 100 ? 100 : num.toFixed(2) * 1;
const { fileList } = this.state;
this[`${progressEvent?.uid}_up`] = progressEvent;
// if (progressEvent.loaded > 5) {
// progressEvent.endAction();
// }
this.setState({
fileList: fileList.map((p) => {
if (p?.uid === progressEvent?.uid) {
p['percent'] = percent;
}
return p;
})
});
};
// 刪除文件
onRemove = (file) => {
if (!file?.status) {
// 刪除正在上傳的文件琼牧，結(jié)束調(diào)用
this[`${file?.uid}_up`] &&
this[`${file?.uid}_up`]?.endAction &&
this[`${file?.uid}_up`]?.endAction(file?.uid);
}
const { fileList } = this.state;
let newList = fileList.filter((p) => p.uid !== file.uid);
let loading = false;
for (let v of newList) {
if (!v?.status) {
loading = true;
break;
}
}
this.setState({
fileList: newList,
upLoading: loading
});
};
beforeUpload = (file, fileLists) => {
console.log('打印file：', file);
let repeat = [...this.state.fileList, ...fileLists];
let obj = {};
let noRepeat = repeat.reduce((pur, item) => {
if (!obj[item?.uid]) {
obj[item?.uid] = true;
pur.push(item);
}
return pur;
}, []);
this.setState({ fileList: noRepeat, upLoading: true });
commonUpload({ file, onUpload: this.handleProgress })
.then((rem) => {
const { fileList } = this.state;
var data = {};
for (var key in rem.file) {
data[key] = rem.file[key];
}
let newFilelist = fileList
.map((p) => {
if (p) {
if (p?.uid === data?.['uid']) {
p = { ...p, ...data };
}
return p;
}
})
.filter((p) => p?.status !== 'error');
if (isNotEmpty(rem.file)) {
this.setState({
fileList: newFilelist
});
}
})
.finally(() => {
const { fileList } = this.state;
// 批量上傳完成，關(guān)閉loading
let flag = true;
for (let item of fileList) {
if (!item?.status) {
flag = false;
break;
}
}
flag && this.setState({ upLoading: false });
// console.log('this.state.fqwFile', JSON.parse(this.state.fqwFile));
});
// 阻止默認上傳
return false;
};
render(){
<Dragger
fileList={fileList}
className="drag-uploader"
onPreview={this.handlePreview} // 點擊文件鏈接或預(yù)覽圖標時的回調(diào)
onRemove={this.onRemove}
multiple={true} // 支持多個文件一起上傳
// onChange={this.onfileChange}
itemRender={(originNode, file, currFileList) => (
<UploadListItem
originNode={originNode}
file={file}
currFileList={currFileList}
fileList={fileList}
/>
)}
beforeUpload={this.beforeUpload}
showUploadList={{
showPreviewIcon: false,
downloadIcon: true
}}
>
{fileList.length >= 15 ? null : UploadButton}
</Dragger>
}
進度條uploadListItem.jsx文件
/*
* @desc 文件上傳哀卫，自定義上傳列表項, 帶進度條
* @author fqw
*/
import React, { Component } from 'react';
import { Progress, Tooltip } from 'antd';
import Cns from 'classnames';
import './index.scss';
const UploadListItem = ({ originNode, file, current, fileList }) => {
const errorNode = <Tooltip title={file['response']}>{originNode.props.children}</Tooltip>;
let have = file.percent < 100;
return (
<div
className={Cns('ant-upload-draggable-list-item', have && 'progressIng')}
style={{ cursor: 'move' }}
key={file.percent}
>
{file.status === 'error' ? errorNode : originNode}
{have && <Progress style={{ width: '100px' }} percent={file.percent} />}
</div>
);
};
export default UploadListItem;
fileAxios.js文件
import { message } from 'antd';
import axios from 'axios';
import { cancelMultipart } from './common';
import {
getUserPresignedurl,
submitFileMsg,
createMultipart,
completeMultipart
} from 'services/common';
// 結(jié)束運行
let endExecution = {
end: false,
uid: ''
};
let endAction = (uid) => {
endExecution.end = true;
endExecution.uid = uid;
};
let upFailed = (file, onUpload) => {
endExecution.end = true;
endExecution.uid = file?.uid;
file['status'] = 'error';
file['response'] = '上傳失敗巨坊，請重試';
message.warning({
content: `文件 ${file.name} 上傳失敗，請重試`,
duration: 5
});
onUpload &&
onUpload({
loaded: 1, // 結(jié)束進度條此改，不顯示
total: 1,
uid: file['uid'],
endAction: endAction
}); // 進度條
return { file, upResult: false };
};
// 普通上傳
const uploadFile = async (file, onUpload) => {
// 獲取上傳接口的路徑
let urlRest = await getUserPresignedurl({ filename: file.name, fileSize: file.size }).then(
(rem) => {
if (rem.status === 200) {
// file['uid'] = rem.data['attachmentID'];
file = Object.assign(file, rem.data);
return rem.data;
}
}
);
// 獲取文件類型
let fileType = file.name.split('.').slice(-1)[0];
let typesObj = {
jpg: 'image/jpeg',
jpe: 'image/jpeg',
jpeg: 'image/jpeg',
png: 'image/png',
gif: 'image/gif',
bmp: 'application/x-bmp',
wbmp: 'image/vnd.wap.wbmp',
ico: 'image/x-icon',
pdf: 'application/pdf',
ppt: 'application/x-ppt',
doc: 'application/msword',
xls: 'application/vnd.ms-excel'
};
let url = urlRest.presignedURL;
const fileAxios = axios.create();
let fname = encodeURIComponent(file.name);
let upBool = false;
upBool = await fileAxios({
url,
method: 'PUT',
data: file,
headers: {
'Content-Type': typesObj[fileType] || '',
'Content-disposition': `filename*=utf-8\'zh_cn\'${fname}`
},
onUploadProgress: (arg) => {
arg.uid = file.uid;
onUpload(arg);
}
})
.then((res) => {
return res.status === 200;
})
.catch((err) => {
return false;
});
// 上傳失敗趾撵，結(jié)束運行
if (!upBool) {
return upFailed(file, onUpload);
}
// 獲取文件下載或預(yù)覽鏈接
let upResult = await submitFileMsg({
filename: file.name,
fileSize: file.size,
attachmentID: urlRest.attachmentID
}).then(
(rem) => {
file['url'] = rem.data['link'];
file['status'] = 'done';
file['attachmentID'] = urlRest.attachmentID;
file = Object.assign(file, rem.data);
return true;
},
(err) => {
return false;
}
);
let copy = JSON.parse(JSON.stringify(file));
copy['name'] = file.name;
return { file: copy, upResult };
};
// poolLimit（數(shù)字類型）：表示限制的并發(fā)數(shù)；
// array（數(shù)組類型）：表示任務(wù)數(shù)組共啃；
// iteratorFn（函數(shù)類型）：表示迭代函數(shù)占调，用于實現(xiàn)對每個任務(wù)項進行處理，該函數(shù)會返回一個 Promise 對象或異步函數(shù)
// onUpload: 進度條
async function asyncPool(poolLimit, array, iteratorFn, onUpload) {
const ret = []; // 存儲所有的異步任務(wù)
const executing = []; // 存儲正在執(zhí)行的異步任務(wù)
for (const item of array) {
// 結(jié)束運行
if (endExecution.end && endExecution.uid === item?.uid) {
console.log('結(jié)束上傳0');
return;
}
// 調(diào)用iteratorFn函數(shù)創(chuàng)建異步任務(wù)
const p = Promise.resolve().then(() => iteratorFn(item, array));
ret.push(p); // 保存新的異步任務(wù)
// 當(dāng)poolLimit值小于或等于總?cè)蝿?wù)個數(shù)時移剪，進行并發(fā)控制
if (poolLimit <= array.length) {
// 當(dāng)任務(wù)完成后究珊，從正在執(zhí)行的任務(wù)數(shù)組中移除已完成的任務(wù)
const e = p.then(() => executing.splice(executing.indexOf(e), 1));
executing.push(e); // 保存正在執(zhí)行的異步任務(wù)
if (executing.length >= poolLimit) {
await Promise.race(executing); // 等待較快的任務(wù)執(zhí)行完成
}
}
onUpload &&
onUpload({
loaded: ret.length < array.length ? ret.length : ret.length - 1, // 等到接口合并完成，再返回100%
total: array.length,
uid: item['uid'],
endAction: endAction
}); // 進度條
}
return Promise.all(ret);
}
// 分段上傳
const multiPartUpload = async (file, onUpload = null) => {
// 獲取段數(shù)
let dataMsg = await createMultipart({
fileSize: file.size, // 傳參數(shù)
filename: file.name
}).then(
(rem) => {
return rem.data;
},
(err) => {
return upFailed(file, onUpload); // 如果接口報錯纵苛，使用upFailed方法處理
}
);
let urlList = dataMsg?.parts || []; // 分段列表
let DEFAULT_SIZE = dataMsg?.partSize; // 分段尺寸
for (let i = 0; i < urlList.length; i++) {
let url = urlList[i]['url'];
let fname = encodeURIComponent(file.name);
let start = i * DEFAULT_SIZE;
let stepFile;
if (i === urlList.length - 1) {
stepFile = file.slice(start, -1); // 如果是最后一段的話剿涮，直接截取剩下的所有內(nèi)容
} else {
stepFile = file.slice(start, start + DEFAULT_SIZE); // 分割文件
}
urlList[i]['stepFile'] = stepFile;
urlList[i]['fname'] = fname;
urlList[i]['uid'] = file.uid;
}
const fileAxios = axios.create();
const detalItem = ({ url, stepFile, fname, partNumber }) => {
return new Promise((resolve, reject) => {
fileAxios({
url,
method: 'PUT',
data: stepFile,
headers: {
'Content-Type': '',
'Content-disposition': `filename*=utf-8\'zh_cn\'${fname}`
}
})
.then((res) => {
let str = res.headers.etag.split('"').join('');
resolve({ eTag: str, partNumber });
})
.catch((err) => {
reject({ eTag: '', partNumber });
});
});
};
let etags = [];
try {
etags = await asyncPool(5, urlList, detalItem, onUpload);
} catch {
// 上傳失敗
etags = [];
endExecution.end = true;
endExecution.uid = file?.uid;
file['url'] = '';
file['link'] = '';
file['attachmentID'] = '';
return upFailed(file, onUpload);
}
let params = {
attachmentID: dataMsg?.attachmentID,
uploadID: dataMsg?.uploadID
};
if (endExecution.end && endExecution.uid === file?.uid) {
cancelMultipart(params);
file['url'] = '';
file['link'] = '';
file['attachmentID'] = '';
return { file: file, upResult: '' };
}
let result = null;
// console.log('etags', etags);
// 上傳完合并文件
try {
result = await completeMultipart({
...params,
etags
});
if (etags) {
onUpload &&
onUpload({
loaded: 100,
total: 100,
uid: file['uid'],
endAction: endAction
}); // 進度條
}
let presignedURL = result?.data?.presignedURL;
// console.log('result', result, 'presignedURL', presignedURL);
file['url'] = presignedURL;
file['link'] = presignedURL;
file['attachmentID'] = dataMsg?.attachmentID;
file['status'] = 'done';
} catch {
file['url'] = '';
file['link'] = '';
file['attachmentID'] = '';
return upFailed(file, onUpload);
}
let copy = JSON.parse(JSON.stringify(file));
copy['name'] = file.name;
return { file: copy, upResult: '' };
};
export const commonUpload = ({ file, onUpload }) => {
let fileSize = 100; // 100M
if (file.size / 1024 / 1024 > fileSize) {
// 當(dāng)文件大于100M采用分段上傳;
return multiPartUpload(file, onUpload);
} else {
return uploadFile(file, onUpload);
}
};
