上一篇: Android WebRTC完整入門教程01: 使用相機
在上一篇中完成了WebRTC最基本的使用--相機的使用. 這一篇將介紹WebRTC中最核心的概念PeerConnection?, 給同一手機中的前后攝像頭建立虛擬的連接, 相互傳輸畫面.
PeerConnection
PeerConnection也就是Peer-to-Peer connection(P2P), 就是兩個"人"的連接. 雙方分別創(chuàng)建PeerConnection對象, 然后向?qū)Ψ桨l(fā)送自己的網(wǎng)絡狀況ICE和多媒體編碼格式SDP(因為這時候連接還沒建立, 所以發(fā)送內(nèi)容是通過服務器完成的). 當雙方網(wǎng)絡和編碼格式協(xié)商好后, 連接就建立好了, 這時從PeerConnection中能獲取到對方的MediaStream數(shù)據(jù)流, 也就能播放對方的音視頻了.
ICE
Interactive Connectivity Establishment, 交互式連接建立. 其實是一個整合STUN和TURN的框架, 給它提供STUN和TURN服務器地址, 它會自動選擇優(yōu)先級高的進行NAT穿透.
SDP
Session Description Protocol: 會話描述協(xié)議. 發(fā)送方的叫Offer, 接受方的叫Answer, 除了名字外沒有區(qū)別. 就是一些文本描述本地的音視頻編碼和網(wǎng)絡地址等.
主要流程
A(local)和B(remote)代表兩個人, 初始化PeerConnectionFactory并分別創(chuàng)建PeerConnection?, 并向PeerConnection?添加本地媒體流.
- A創(chuàng)建Offer
- A保存Offer(set local description)
- A發(fā)送Offer給B
- B保存Offer(set remote description)
- B創(chuàng)建Answer
- B保存Answer(set local description)
- B發(fā)送Answer給A
- A保存Answer(set remote description)
- A發(fā)送Ice Candidates給B
- B發(fā)送Ice Candidates給A
-
A,B收到對方的媒體流并播放
建立PeerConnection.jpeg
如上圖所示, 總共11步, 雖然步驟不少, 但其實并不復雜, 雙方基本是對稱的. 主要代碼如下.
準備步驟
主要是初始化PeerConnectionFactory和使用相機, 在上一篇已介紹過.
public class MainActivity extends AppCompatActivity {
PeerConnectionFactory peerConnectionFactory;
PeerConnection peerConnectionLocal;
PeerConnection peerConnectionRemote;
SurfaceViewRenderer localView;
SurfaceViewRenderer remoteView;
MediaStream mediaStreamLocal;
MediaStream mediaStreamRemote;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
EglBase.Context eglBaseContext = EglBase.create().getEglBaseContext();
// create PeerConnectionFactory
PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions
.builder(this)
.createInitializationOptions());
PeerConnectionFactory.Options options = new PeerConnectionFactory.Options();
DefaultVideoEncoderFactory defaultVideoEncoderFactory =
new DefaultVideoEncoderFactory(eglBaseContext, true, true);
DefaultVideoDecoderFactory defaultVideoDecoderFactory =
new DefaultVideoDecoderFactory(eglBaseContext);
peerConnectionFactory = PeerConnectionFactory.builder()
.setOptions(options)
.setVideoEncoderFactory(defaultVideoEncoderFactory)
.setVideoDecoderFactory(defaultVideoDecoderFactory)
.createPeerConnectionFactory();
SurfaceTextureHelper surfaceTextureHelper = SurfaceTextureHelper.create("CaptureThread", eglBaseContext);
// create VideoCapturer
VideoCapturer videoCapturer = createCameraCapturer(true);
VideoSource videoSource = peerConnectionFactory.createVideoSource(videoCapturer.isScreencast());
videoCapturer.initialize(surfaceTextureHelper, getApplicationContext(), videoSource.getCapturerObserver());
videoCapturer.startCapture(480, 640, 30);
localView = findViewById(R.id.localView);
localView.setMirror(true);
localView.init(eglBaseContext, null);
// create VideoTrack
VideoTrack videoTrack = peerConnectionFactory.createVideoTrack("100", videoSource);
// // display in localView
// videoTrack.addSink(localView);
SurfaceTextureHelper remoteSurfaceTextureHelper = SurfaceTextureHelper.create("RemoteCaptureThread", eglBaseContext);
// create VideoCapturer
VideoCapturer remoteVideoCapturer = createCameraCapturer(false);
VideoSource remoteVideoSource = peerConnectionFactory.createVideoSource(remoteVideoCapturer.isScreencast());
remoteVideoCapturer.initialize(remoteSurfaceTextureHelper, getApplicationContext(), remoteVideoSource.getCapturerObserver());
remoteVideoCapturer.startCapture(480, 640, 30);
remoteView = findViewById(R.id.remoteView);
remoteView.setMirror(false);
remoteView.init(eglBaseContext, null);
// create VideoTrack
VideoTrack remoteVideoTrack = peerConnectionFactory.createVideoTrack("102", remoteVideoSource);
// // display in remoteView
// remoteVideoTrack.addSink(remoteView);
mediaStreamLocal = peerConnectionFactory.createLocalMediaStream("mediaStreamLocal");
mediaStreamLocal.addTrack(videoTrack);
mediaStreamRemote = peerConnectionFactory.createLocalMediaStream("mediaStreamRemote");
mediaStreamRemote.addTrack(remoteVideoTrack);
call(mediaStreamLocal, mediaStreamRemote);
}
}
使用相機
對createCameraCapturer()方法略作修改, 傳入boolean參數(shù)就能分別獲取前后攝像頭.
private VideoCapturer createCameraCapturer(boolean isFront) {
Camera1Enumerator enumerator = new Camera1Enumerator(false);
final String[] deviceNames = enumerator.getDeviceNames();
// First, try to find front facing camera
for (String deviceName : deviceNames) {
if (isFront ? enumerator.isFrontFacing(deviceName) : enumerator.isBackFacing(deviceName)) {
VideoCapturer videoCapturer = enumerator.createCapturer(deviceName, null);
if (videoCapturer != null) {
return videoCapturer;
}
}
}
return null;
}
撥打
建立連接的兩人肯定有一個是撥打方, 另一個是接受方. 撥打方創(chuàng)建Offer發(fā)給接受方, 接收方收到后回復Answer.
private void call(MediaStream localMediaStream, MediaStream remoteMediaStream) {
List<PeerConnection.IceServer> iceServers = new ArrayList<>();
peerConnectionLocal = peerConnectionFactory.createPeerConnection(iceServers, new PeerConnectionAdapter("localconnection") {
@Override
public void onIceCandidate(IceCandidate iceCandidate) {
super.onIceCandidate(iceCandidate);
peerConnectionRemote.addIceCandidate(iceCandidate);
}
@Override
public void onAddStream(MediaStream mediaStream) {
super.onAddStream(mediaStream);
VideoTrack remoteVideoTrack = mediaStream.videoTracks.get(0);
runOnUiThread(() -> {
remoteVideoTrack.addSink(localView);
});
}
});
peerConnectionRemote = peerConnectionFactory.createPeerConnection(iceServers, new PeerConnectionAdapter("remoteconnection") {
@Override
public void onIceCandidate(IceCandidate iceCandidate) {
super.onIceCandidate(iceCandidate);
peerConnectionLocal.addIceCandidate(iceCandidate);
}
@Override
public void onAddStream(MediaStream mediaStream) {
super.onAddStream(mediaStream);
VideoTrack localVideoTrack = mediaStream.videoTracks.get(0);
runOnUiThread(() -> {
localVideoTrack.addSink(remoteView);
});
}
});
peerConnectionLocal.addStream(localMediaStream);
peerConnectionLocal.createOffer(new SdpAdapter("local offer sdp") {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
super.onCreateSuccess(sessionDescription);
// todo crashed here
peerConnectionLocal.setLocalDescription(new SdpAdapter("local set local"), sessionDescription);
peerConnectionRemote.addStream(remoteMediaStream);
peerConnectionRemote.setRemoteDescription(new SdpAdapter("remote set remote"), sessionDescription);
peerConnectionRemote.createAnswer(new SdpAdapter("remote answer sdp") {
@Override
public void onCreateSuccess(SessionDescription sdp) {
super.onCreateSuccess(sdp);
peerConnectionRemote.setLocalDescription(new SdpAdapter("remote set local"), sdp);
peerConnectionLocal.setRemoteDescription(new SdpAdapter("local set remote"), sdp);
}
}, new MediaConstraints());
}
}, new MediaConstraints());
}
}
注意: 雖然這里沒有真正使用到網(wǎng)絡, 但是要添加網(wǎng)絡權限
<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
網(wǎng)上大部分本地回環(huán)(Loopback)的Demo都只用到一個攝像頭, 這里使用到同一個手機的前后攝像頭, 把它們當做兩個客戶端, 建立模擬連接, 發(fā)送媒體數(shù)據(jù). 這跟實際WebRTC工作流程非常接近了, 只有一點差別--這里的數(shù)據(jù)傳輸是內(nèi)存共享, 而實際是通過網(wǎng)絡發(fā)送.
