At the heart of MSE is the MediaSource object. This object is created by the application and attached to the media element. Its purpose is to provide the media data for playback as requested by the media element.

The MediaSource object maintains a collection of SourceBuffers. These are the interface through which the application appends media data to the source and methods are provided to insert, remove and manage media data. They are essentially an abstraction of a timeline – media data can appended to the buffer based on media playback timestamp, or it can be appended sequentially, ignoring timestamps. The latter mode enables unrelated media to be spliced together, which allows uses such as advert insertion or even video editing in the browser.

The application handles the requesting of media data from the server and appends the response to the SourceBuffer. Decoupling the fetching of media data from playback allows the media data to be sourced using novel transport mechanisms or from different locations.

SourceBuffers can contain audio, video or timed text and an instance is created for each stream that needs to be presented. Typically there might be one video stream, one audio stream and perhaps a subtitle stream. Since each media type is handled separately, access services such as audio description or subtitling can be selected simply by requesting a different stream.

Finally, the specification also includes extensions to the HTMLVideoElement allowing measurement of video decode and rendering performance which could be used to help decide the most appropriate video stream to present if a number of options are available.

An additional benefit of not hardcoding features into the browser is that any functionality upgrades such as improved adaptive algorithms or defect fixes are simply a case of updating the Javascript application, which is freshly fetched each time the page is loaded, rather than requiring every user to upgrade their browser. Software updates to the browser itself might be fairly easy on a PC but happen infrequently on a smart TV or set top box.

MSE的核心是MediaSource對(duì)象见坑。該對(duì)象由應(yīng)用程序創(chuàng)建并附加到媒體元素冕象。其目的是提供媒體元素請(qǐng)求的媒體數(shù)據(jù)進(jìn)行回放。

MediaSource對(duì)象維護(hù)SourceBuffers的集合。這些是應(yīng)用程序通過(guò)其將媒體數(shù)據(jù)附加到源的接口淮阐，并且提供用于插入辣垒，移除和管理媒體數(shù)據(jù)的方法。它們本質(zhì)上是一個(gè)時(shí)間軸的抽象 - 媒體數(shù)據(jù)可以基于媒體播放時(shí)間戳附加到緩沖區(qū)傀蚌，也可以按順序附加善炫，忽略時(shí)間戳。后一種模式可以將不相關(guān)的媒體拼接在一起，從而允許在瀏覽器中使用諸如廣告插入或者甚至視頻編輯。

應(yīng)用程序處理從服務(wù)器請(qǐng)求媒體數(shù)據(jù)并將響應(yīng)附加到SourceBuffer撒妈。將媒體數(shù)據(jù)從回放中取出允許使用新穎的傳輸機(jī)制或從不同的位置獲取媒體數(shù)據(jù)。

SourceBuffers可以包含音頻棋蚌，視頻或定時(shí)文本，并為每個(gè)需要呈現(xiàn)的流創(chuàng)建一個(gè)實(shí)例湿弦。通呈叱洌可能有一個(gè)視頻流，一個(gè)音頻流和一個(gè)字幕流庸队。由于每種媒體類型是分開處理的纽谒，因此可以簡(jiǎn)單地通過(guò)請(qǐng)求不同的流來(lái)選擇諸如音頻描述或字幕的訪問服務(wù)央勒。

最后崔步，該規(guī)范還包括對(duì)HTMLVideoElement的擴(kuò)展瑞你，允許測(cè)量視頻解碼和渲染性能，如果有多個(gè)選項(xiàng)可用砌创，可用于幫助確定最合適的視頻流虏缸。

瀏覽器沒有對(duì)特征進(jìn)行硬編碼的另外一個(gè)好處是，任何功能升級(jí)（例如改進(jìn)的自適應(yīng)算法或缺陷修復(fù)）都只是更新JavaScript應(yīng)用程序的一種情況嫩实，每次頁(yè)面加載時(shí)都會(huì)提取新的Javascript應(yīng)用程序瓤湘，而不是要求每個(gè)用戶升級(jí)瀏覽器憔晒。瀏覽器本身的軟件更新在PC上可能相當(dāng)容易，但在智能電視或機(jī)頂盒上很少發(fā)生犁柜。