上一篇: Android WebRTC完整入門教程03: 信令
多人視頻有三種理論方案, 如下圖所示, 從左到右分別是Mesh,SFU,MCU.
Mesh 網(wǎng)格, 每個(gè)人都跟其他人單獨(dú)建立連接. 4個(gè)人的情況下, 每個(gè)人建立3個(gè)連接, 也就是3個(gè)上傳流和3個(gè)下載流. 此方案對(duì)客戶端網(wǎng)絡(luò)和計(jì)算能力要求最高, 對(duì)服務(wù)端沒(méi)有特別要求.
SFU(Selective Forwarding Unit) 可選擇轉(zhuǎn)發(fā)單元, 有一個(gè)中心單元, 負(fù)責(zé)轉(zhuǎn)發(fā)流. 每個(gè)人只跟中心單元建立一個(gè)連接, 上傳自己的流, 并下載別人的流. 4個(gè)人的情況下, 每個(gè)人建立一個(gè)連接, 包括1個(gè)上傳流和3個(gè)下載流. 此方案對(duì)客戶端要求較高, 對(duì)服務(wù)端要求較高.
MCU(Multipoint Control Unit) 多端控制單元, 有一個(gè)中心單元, 負(fù)責(zé)混流處理和轉(zhuǎn)發(fā)流. 每個(gè)人只跟中心單元建立一個(gè)連接, 上傳自己的流, 并下載混流. 4個(gè)人的情況下, 每個(gè)人建立一個(gè)連接, 包括1個(gè)上傳流和1個(gè)下載流. 此方案對(duì)客戶端沒(méi)有特別要求, 對(duì)服務(wù)端要求最高.
Mesh實(shí)現(xiàn)
先從理論上分析一下, 客戶端A與B之間建立連接完全是通過(guò)PeerConnection對(duì)象, 那么只要客戶端A有多個(gè)PeerConnection對(duì)象, 它就可以同時(shí)跟B,C,D...連接.
雖然PeerConnection有多個(gè), 但是客戶端A跟信令服務(wù)器仍然是一個(gè)socket連接, 這樣A向服務(wù)器發(fā)送信令時(shí)就要指定發(fā)送給誰(shuí), 收到信令時(shí)要判斷來(lái)自誰(shuí), 服務(wù)端收到信令時(shí)要判斷發(fā)給誰(shuí). 這就需要在所有信令中添加兩個(gè)字段 from 和 to, 代表信令發(fā)送方和接收方. 每個(gè)socket連接都有唯一socketId, 可以用socketId來(lái)標(biāo)識(shí)一個(gè)客戶端. 每個(gè)客戶端用一個(gè)HashMap<String, PeerConnection>(key是socketId)來(lái)保存自己的連接.
撥號(hào)方案: 客戶端A加入房間, 如果房間內(nèi)還有其他客戶端B和C. 服務(wù)端向B和C發(fā)送A的socketId, B和C收到后各自給A發(fā)送Offer建立連接, A分別回復(fù)Answer被動(dòng)建立多個(gè)連接. 這樣保證每個(gè)客戶端的邏輯是一樣的, 如果它新加入房間, 那么只需要等待其他人的Offer; 如果它已在房間中, 那么等待別人加入時(shí)向別人發(fā)送Offer.
信令服務(wù)端
在上一篇基礎(chǔ)上做如下修改,
- 轉(zhuǎn)發(fā)message時(shí)根據(jù)其中的to, 來(lái)選擇發(fā)送目標(biāo)
- 某人加入房間時(shí), 向其他人發(fā)送此人的socketId
- 去掉房間內(nèi)最多兩個(gè)人的限制
socket.on('message', function(message) {
// for a real app, would be room-only (not broadcast)
// socket.broadcast.emit('message', message);
var to = message['to'];
log('from:' + socket.id + " to:" + to, message);
io.sockets.sockets[to].emit('message', message);
});
socket.on('create or join', function(room) {
log('Received request to create or join room ' + room);
var clientsInRoom = io.sockets.adapter.rooms[room];
var numClients = clientsInRoom ? Object.keys(clientsInRoom.sockets).length : 0;
log('Room ' + room + ' now has ' + numClients + ' client(s)');
if (numClients === 0) {
socket.join(room);
log('Client ID ' + socket.id + ' created room ' + room);
socket.emit('created', room, socket.id);
} else {
log('Client ID ' + socket.id + ' joined room ' + room);
io.sockets.in(room).emit('join', room, socket.id);
socket.join(room);
socket.emit('joined', room, socket.id);
io.sockets.in(room).emit('ready');
}
});
MainActivity.java
在上一篇的基礎(chǔ)上, 添加HashMap<String, PeerConnection> peerConnectionMap(key是socketId)管理所有的PeerConnection連接, 收到信令時(shí)判斷來(lái)源的socketId, 發(fā)送時(shí)加上自己和對(duì)方的socketId.
public class MainActivity extends AppCompatActivity implements SignalingClient.Callback {
EglBase.Context eglBaseContext;
PeerConnectionFactory peerConnectionFactory;
SurfaceViewRenderer localView;
MediaStream mediaStream;
List<PeerConnection.IceServer> iceServers;
HashMap<String, PeerConnection> peerConnectionMap;
SurfaceViewRenderer[] remoteViews;
int remoteViewsIndex = 0;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
peerConnectionMap = new HashMap<>();
iceServers = new ArrayList<>();
iceServers.add(PeerConnection.IceServer.builder("stun:stun.l.google.com:19302").createIceServer());
eglBaseContext = EglBase.create().getEglBaseContext();
// create PeerConnectionFactory
PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions
.builder(this)
.createInitializationOptions());
PeerConnectionFactory.Options options = new PeerConnectionFactory.Options();
DefaultVideoEncoderFactory defaultVideoEncoderFactory =
new DefaultVideoEncoderFactory(eglBaseContext, true, true);
DefaultVideoDecoderFactory defaultVideoDecoderFactory =
new DefaultVideoDecoderFactory(eglBaseContext);
peerConnectionFactory = PeerConnectionFactory.builder()
.setOptions(options)
.setVideoEncoderFactory(defaultVideoEncoderFactory)
.setVideoDecoderFactory(defaultVideoDecoderFactory)
.createPeerConnectionFactory();
SurfaceTextureHelper surfaceTextureHelper = SurfaceTextureHelper.create("CaptureThread", eglBaseContext);
// create VideoCapturer
VideoCapturer videoCapturer = createCameraCapturer(true);
VideoSource videoSource = peerConnectionFactory.createVideoSource(videoCapturer.isScreencast());
videoCapturer.initialize(surfaceTextureHelper, getApplicationContext(), videoSource.getCapturerObserver());
videoCapturer.startCapture(480, 640, 30);
localView = findViewById(R.id.localView);
localView.setMirror(true);
localView.init(eglBaseContext, null);
// create VideoTrack
VideoTrack videoTrack = peerConnectionFactory.createVideoTrack("100", videoSource);
// // display in localView
videoTrack.addSink(localView);
remoteViews = new SurfaceViewRenderer[]{
findViewById(R.id.remoteView),
findViewById(R.id.remoteView2),
findViewById(R.id.remoteView3),
};
for(SurfaceViewRenderer remoteView : remoteViews) {
remoteView.setMirror(false);
remoteView.init(eglBaseContext, null);
}
mediaStream = peerConnectionFactory.createLocalMediaStream("mediaStream");
mediaStream.addTrack(videoTrack);
SignalingClient.get().init(this);
}
private synchronized PeerConnection getOrCreatePeerConnection(String socketId) {
PeerConnection peerConnection = peerConnectionMap.get(socketId);
if(peerConnection != null) {
return peerConnection;
}
peerConnection = peerConnectionFactory.createPeerConnection(iceServers, new PeerConnectionAdapter("PC:" + socketId) {
@Override
public void onIceCandidate(IceCandidate iceCandidate) {
super.onIceCandidate(iceCandidate);
SignalingClient.get().sendIceCandidate(iceCandidate, socketId);
}
@Override
public void onAddStream(MediaStream mediaStream) {
super.onAddStream(mediaStream);
VideoTrack remoteVideoTrack = mediaStream.videoTracks.get(0);
runOnUiThread(() -> {
remoteVideoTrack.addSink(remoteViews[remoteViewsIndex++]);
});
}
});
peerConnection.addStream(mediaStream);
peerConnectionMap.put(socketId, peerConnection);
return peerConnection;
}
@Override
public void onCreateRoom() {
}
@Override
public void onPeerJoined(String socketId) {
PeerConnection peerConnection = getOrCreatePeerConnection(socketId);
peerConnection.createOffer(new SdpAdapter("createOfferSdp:" + socketId) {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
super.onCreateSuccess(sessionDescription);
peerConnection.setLocalDescription(new SdpAdapter("setLocalSdp:" + socketId), sessionDescription);
SignalingClient.get().sendSessionDescription(sessionDescription, socketId);
}
}, new MediaConstraints());
}
@Override
public void onSelfJoined() {
}
@Override
public void onPeerLeave(String msg) {
}
@Override
public void onOfferReceived(JSONObject data) {
runOnUiThread(() -> {
final String socketId = data.optString("from");
PeerConnection peerConnection = getOrCreatePeerConnection(socketId);
peerConnection.setRemoteDescription(new SdpAdapter("setRemoteSdp:" + socketId),
new SessionDescription(SessionDescription.Type.OFFER, data.optString("sdp")));
peerConnection.createAnswer(new SdpAdapter("localAnswerSdp") {
@Override
public void onCreateSuccess(SessionDescription sdp) {
super.onCreateSuccess(sdp);
peerConnectionMap.get(socketId).setLocalDescription(new SdpAdapter("setLocalSdp:" + socketId), sdp);
SignalingClient.get().sendSessionDescription(sdp, socketId);
}
}, new MediaConstraints());
});
}
@Override
public void onAnswerReceived(JSONObject data) {
String socketId = data.optString("from");
PeerConnection peerConnection = getOrCreatePeerConnection(socketId);
peerConnection.setRemoteDescription(new SdpAdapter("setRemoteSdp:" + socketId),
new SessionDescription(SessionDescription.Type.ANSWER, data.optString("sdp")));
}
@Override
public void onIceCandidateReceived(JSONObject data) {
String socketId = data.optString("from");
PeerConnection peerConnection = getOrCreatePeerConnection(socketId);
peerConnection.addIceCandidate(new IceCandidate(
data.optString("id"),
data.optInt("label"),
data.optString("candidate")
));
}
@Override
protected void onDestroy() {
super.onDestroy();
SignalingClient.get().destroy();
}
private VideoCapturer createCameraCapturer(boolean isFront) {
Camera1Enumerator enumerator = new Camera1Enumerator(false);
final String[] deviceNames = enumerator.getDeviceNames();
// First, try to find front facing camera
for (String deviceName : deviceNames) {
if (isFront ? enumerator.isFrontFacing(deviceName) : enumerator.isBackFacing(deviceName)) {
VideoCapturer videoCapturer = enumerator.createCapturer(deviceName, null);
if (videoCapturer != null) {
return videoCapturer;
}
}
}
return null;
}
}
多人視頻
啟動(dòng)node.js服務(wù)器, 在多個(gè)安卓手機(jī)上安裝客戶端, 先后啟動(dòng), 隨后就能在一個(gè)客戶端上看到其他所有人的畫面. (這里布局文件只放了4個(gè)SurfaceViewRenderer, 因此支持2,3,4個(gè)手機(jī)同時(shí)連接).
本項(xiàng)目GitHub地址/step4multipeers
本項(xiàng)目GitHub地址/step4web
