說在前面的話:視頻實時羣聊天有三種架構:html
Mesh架構:終端之間互相鏈接,沒有中心服務器,產生的問題,每一個終端都要鏈接n-1個終端,每一個終端的編碼和網絡壓力都很大。羣聊人數N不可能太大。java
Router架構:終端之間引入中心服務器,學名MCU(Multi Point Control Unit),每一個終端的視頻流都發布到MCU服務器上,而後服務器負責編碼發佈多視頻流的工做,減輕客戶端的壓力。node
Mix架構:在Router架構基礎上,多個視頻流在服務器端被合爲一個視頻流,減輕網絡壓力。android
下面講咱們的選擇,在MCU方面有licode、kurento等解決方案。kurento在視頻羣聊領域有專門的kurento Room解決方案,官方還提供一個kurento room server的樣例實現。ios
首先能夠考慮不是一個Kurento Room Demo做爲搭建方案原型的MCU組件。nginx
Room Demo的部署可見:http://doc-kurento-room.readthedocs.io/en/stable/demo_deployment.htmlgit
其中碰到一些Maven編譯問題:程序員
- Unable to initialise extensions Component descriptor role: 'com.jcraft.jsch.UIKeyboardInteractive', implementation: 'org.apache.maven.wagon.providers.ssh.jsch.interactive.PrompterUIKeyboardInteractive', role hint: 'default' has a hint, but there are other implementations that don't
Maven的安裝版本須要時3.0以上github
還有碰到找不到bower命令行問題。bower是Node.js下面的一個包管理工具,安裝node.js之後用npm安裝便可web
最後按照部署指南網頁中的命令啓動服務器便可。
Demo服務器有兩部分,一部分是Demo Web服務器,二是把官方的kurento room server也集成到了這個demo中。不用再架設獨立的kurento room server
說說Android段的實施:再說一個公司:http://www.nubomedia.eu/,這家公司提供實時媒體通訊開源雲服務,核心組件多是kurento media server,它的官網和kurento官網用一個模板,about裏面顯示兩家組織有聯繫,kurento官方提供的Java Client由於底層API緣由在Android上不願用,這個nubomedia組織提供了一個kurento android client的實現,同時還提供了一個kurento room client的實現以及room使用案例:https://github.com/nubomedia-vtt/nubo-test,這家公司對其開發的開源方案管理很是及時,早晨提個接口的issue,下午已經commit了代碼修改。
這個案例雖然支持room溝通,但視頻溝通是基於room發佈訂閱機制作的雙人聊天。略改一下代碼應該就能夠實現多人聊天不過這家組織提供的兩個client實現和官方的接口高度類似。主要改的是PeerVideoActivity這個類,下面我share一個基本走通多端通訊的這個類的代碼,供你們參考:
- package fi.vtt.nubotest;
- import android.app.ListActivity;
- import android.content.SharedPreferences;
- import android.graphics.PixelFormat;
- import android.opengl.GLSurfaceView;
- import android.os.Bundle;
- import android.os.Handler;
- import android.util.Log;
- import android.view.Menu;
- import android.view.MenuItem;
- import android.view.View;
- import android.view.WindowManager;
- import android.widget.TextView;
- import android.widget.Toast;
- import org.webrtc.IceCandidate;
- import org.webrtc.MediaStream;
- import org.webrtc.PeerConnection;
- import org.webrtc.RendererCommon;
- import org.webrtc.SessionDescription;
- import org.webrtc.VideoRenderer;
- import org.webrtc.VideoRendererGui;
- import java.util.Map;
- import fi.vtt.nubomedia.kurentoroomclientandroid.RoomError;
- import fi.vtt.nubomedia.kurentoroomclientandroid.RoomListener;
- import fi.vtt.nubomedia.kurentoroomclientandroid.RoomNotification;
- import fi.vtt.nubomedia.kurentoroomclientandroid.RoomResponse;
- import fi.vtt.nubomedia.webrtcpeerandroid.NBMMediaConfiguration;
- import fi.vtt.nubomedia.webrtcpeerandroid.NBMPeerConnection;
- import fi.vtt.nubomedia.webrtcpeerandroid.NBMWebRTCPeer;
- import fi.vtt.nubotest.util.Constants;
- /**
- * Activity for receiving the video stream of a peer
- * (based on PeerVideoActivity of Pubnub's video chat tutorial example.
- */
- public class PeerVideoActivity extends ListActivity implements NBMWebRTCPeer.Observer, RoomListener {
- private static final String TAG = "PeerVideoActivity";
- private NBMMediaConfiguration peerConnectionParameters;
- private NBMWebRTCPeer nbmWebRTCPeer;
- private SessionDescription localSdp;
- private SessionDescription remoteSdp;
- private String PaticipantID;
- private VideoRenderer.Callbacks localRender;
- private VideoRenderer.Callbacks remoteRender;
- private GLSurfaceView videoView;
- private SharedPreferences mSharedPreferences;
- private int publishVideoRequestId;
- private int sendIceCandidateRequestId;
- private TextView mCallStatus;
- private String username, calluser;
- private boolean backPressed = false;
- private Thread backPressedThread = null;
- private static final int LOCAL_X_CONNECTED = 72;
- private static final int LOCAL_Y_CONNECTED = 72;
- private static final int LOCAL_WIDTH_CONNECTED = 25;
- private static final int LOCAL_HEIGHT_CONNECTED = 25;
- // Remote video screen position
- private static final int REMOTE_X = 0;
- private static int REMOTE_Y = 0;
- private static final int REMOTE_WIDTH = 25;
- private static final int REMOTE_HEIGHT = 25;
- private Handler mHandler;
- private CallState callState;
- private enum CallState{
- IDLE, PUBLISHING, PUBLISHED, WAITING_REMOTE_USER, RECEIVING_REMOTE_USER,PATICIPANT_JOINED,RECEIVING_PATICIPANT,
- }
- @Override
- public void onCreate(Bundle savedInstanceState) {
- super.onCreate(savedInstanceState);
- callState = CallState.IDLE;
- setContentView(R.layout.activity_video_chat);
- getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
- mHandler = new Handler();
- Bundle extras = getIntent().getExtras();
- if (extras == null || !extras.containsKey(Constants.USER_NAME)) {
- ;
- Toast.makeText(this, "Need to pass username to PeerVideoActivity in intent extras (Constants.USER_NAME).",
- Toast.LENGTH_SHORT).show();
- finish();
- return;
- }
- this.username = extras.getString(Constants.USER_NAME, "");
- Log.i(TAG, "username: " + username);
- if (extras.containsKey(Constants.CALL_USER)) {
- this.calluser = extras.getString(Constants.CALL_USER, "");
- Log.i(TAG, "callUser: " + calluser);
- }
- this.mCallStatus = (TextView) findViewById(R.id.call_status);
- TextView prompt = (TextView) findViewById(R.id.receive_prompt);
- prompt.setText("Receive from " + calluser);
- this.videoView = (GLSurfaceView) findViewById(R.id.gl_surface);
- // Set up the List View for chatting
- RendererCommon.ScalingType scalingType = RendererCommon.ScalingType.SCALE_ASPECT_FILL;
- VideoRendererGui.setView(videoView, null);
- localRender = VideoRendererGui.create( LOCAL_X_CONNECTED, LOCAL_Y_CONNECTED,
- LOCAL_WIDTH_CONNECTED, LOCAL_HEIGHT_CONNECTED,
- scalingType, true);
- NBMMediaConfiguration.NBMVideoFormat receiverVideoFormat = new NBMMediaConfiguration.NBMVideoFormat(352, 288, PixelFormat.RGB_888, 20);
- peerConnectionParameters = new NBMMediaConfiguration( NBMMediaConfiguration.NBMRendererType.OPENGLES,
- NBMMediaConfiguration.NBMAudioCodec.OPUS, 0,
- NBMMediaConfiguration.NBMVideoCodec.VP8, 0,
- receiverVideoFormat,
- NBMMediaConfiguration.NBMCameraPosition.FRONT);
- nbmWebRTCPeer = new NBMWebRTCPeer(peerConnectionParameters, this, localRender, this);
- nbmWebRTCPeer.initialize();
- Log.i(TAG, "PeerVideoActivity initialized");
- mHandler.postDelayed(publishDelayed, 4000);
- MainActivity.getKurentoRoomAPIInstance().addObserver(this);
- callState = CallState.PUBLISHING;
- mCallStatus.setText("Publishing...");
- }
- private Runnable publishDelayed = new Runnable() {
- @Override
- public void run() {
- nbmWebRTCPeer.generateOffer("derp", true);
- }
- };
- @Override
- public boolean onCreateOptionsMenu(Menu menu) {
- // Inflate the menu; this adds items to the action bar if it is present.
- getMenuInflater().inflate(R.menu.menu_video_chat, menu);
- return true;
- }
- @Override
- public boolean onOptionsItemSelected(MenuItem item) {
- // Handle action bar item clicks here. The action bar will
- // automatically handle clicks on the Home/Up button, so long
- // as you specify a parent activity in AndroidManifest.xml.
- int id = item.getItemId();
- //noinspection SimplifiableIfStatement
- if (id == R.id.action_settings) {
- return true;
- }
- return super.onOptionsItemSelected(item);
- }
- @Override
- protected void onStart() {
- super.onStart();
- }
- @Override
- protected void onPause() {
- nbmWebRTCPeer.stopLocalMedia();
- super.onPause();
- }
- @Override
- protected void onResume() {
- super.onResume();
- nbmWebRTCPeer.startLocalMedia();
- }
- @Override
- protected void onStop() {
- endCall();
- super.onStop();
- }
- @Override
- protected void onDestroy() {
- super.onDestroy();
- }
- @Override
- public void onBackPressed() {
- // If back button has not been pressed in a while then trigger thread and toast notification
- if (!this.backPressed){
- this.backPressed = true;
- Toast.makeText(this,"Press back again to end.",Toast.LENGTH_SHORT).show();
- this.backPressedThread = new Thread(new Runnable() {
- @Override
- public void run() {
- try {
- Thread.sleep(5000);
- backPressed = false;
- } catch (InterruptedException e){ Log.d("VCA-oBP","Successfully interrupted"); }
- }
- });
- this.backPressedThread.start();
- }
- // If button pressed the second time then call super back pressed
- // (eventually calls onDestroy)
- else {
- if (this.backPressedThread != null)
- this.backPressedThread.interrupt();
- super.onBackPressed();
- }
- }
- public void hangup(View view) {
- finish();
- }
- public void receiveFromRemote(View view){
- Log.e(TAG,"--->receiveFromRemote");
- if (callState == CallState.PUBLISHED){
- callState = CallState.WAITING_REMOTE_USER;
- nbmWebRTCPeer.generateOffer("remote", false);
- runOnUiThread(new Runnable() {
- @Override
- public void run() {
- mCallStatus.setText("Waiting remote stream...");
- }
- });
- }
- }
- /**
- * Terminates the current call and ends activity
- */
- private void endCall() {
- callState = CallState.IDLE;
- try
- {
- if (nbmWebRTCPeer != null) {
- nbmWebRTCPeer.close();
- nbmWebRTCPeer = null;
- }
- }
- catch (Exception e){e.printStackTrace();}
- }
- @Override
- public void onLocalSdpOfferGenerated(final SessionDescription sessionDescription, NBMPeerConnection nbmPeerConnection) {
- Log.e(TAG,"--->onLocalSdpOfferGenerated");
- if (callState == CallState.PUBLISHING || callState == CallState.PUBLISHED) {
- localSdp = sessionDescription;
- Log.e(TAG,"--->onLocalSdpOfferGenerated:publish");
- runOnUiThread(new Runnable() {
- @Override
- public void run() {
- if (MainActivity.getKurentoRoomAPIInstance() != null) {
- Log.d(TAG, "Sending " + sessionDescription.type);
- publishVideoRequestId = ++Constants.id;
- // String sender = calluser + "_webcam";
- // MainActivity.getKurentoRoomAPIInstance().sendReceiveVideoFrom(sender, localSdp.description, publishVideoRequestId);
- MainActivity.getKurentoRoomAPIInstance().sendPublishVideo(localSdp.description, false, publishVideoRequestId);
- }
- }
- });
- } else { // Asking for remote user video
- Log.e(TAG,"--->onLocalSdpOfferGenerated:remote");
- remoteSdp = sessionDescription;
- // nbmWebRTCPeer.selectCameraPosition(NBMMediaConfiguration.NBMCameraPosition.BACK);
- runOnUiThread(new Runnable() {
- @Override
- public void run() {
- if (MainActivity.getKurentoRoomAPIInstance() != null) {
- Log.e(TAG, "Sending--> " +calluser+ sessionDescription.type);
- publishVideoRequestId = ++Constants.id;
- String sender = calluser + "_webcam";
- MainActivity.getKurentoRoomAPIInstance().sendReceiveVideoFrom(sender, remoteSdp.description, publishVideoRequestId);
- }
- }
- });
- }
- }
- @Override
- public void onLocalSdpAnswerGenerated(SessionDescription sessionDescription, NBMPeerConnection nbmPeerConnection) {
- }
- @Override
- public void onIceCandidate(IceCandidate iceCandidate, NBMPeerConnection nbmPeerConnection) {
- Log.e(TAG,"--->onIceCandidate");
- sendIceCandidateRequestId = ++Constants.id;
- if (callState == CallState.PUBLISHING || callState == CallState.PUBLISHED){
- Log.e(TAG,"--->onIceCandidate:publish");
- MainActivity.getKurentoRoomAPIInstance().sendOnIceCandidate(this.username, iceCandidate.sdp,
- iceCandidate.sdpMid, Integer.toString(iceCandidate.sdpMLineIndex), sendIceCandidateRequestId);
- } else{
- Log.e(TAG,"--->onIceCandidate:"+this.calluser);
- MainActivity.getKurentoRoomAPIInstance().sendOnIceCandidate(this.calluser, iceCandidate.sdp,
- iceCandidate.sdpMid, Integer.toString(iceCandidate.sdpMLineIndex), sendIceCandidateRequestId);
- }
- }
- @Override
- public void onIceStatusChanged(PeerConnection.IceConnectionState iceConnectionState, NBMPeerConnection nbmPeerConnection) {
- Log.i(TAG, "onIceStatusChanged");
- }
- @Override
- public void onRemoteStreamAdded(MediaStream mediaStream, NBMPeerConnection nbmPeerConnection) {
- if (callState == CallState.PUBLISHING || callState == CallState.PUBLISHED) {
- Log.e(TAG, "-->onRemoteStreamAdded-->no");
- return;
- }
- Log.e(TAG, "-->onRemoteStreamAdded");
- RendererCommon.ScalingType scalingType = RendererCommon.ScalingType.SCALE_ASPECT_FILL;
- remoteRender = VideoRendererGui.create( REMOTE_X, REMOTE_Y,
- REMOTE_WIDTH, REMOTE_HEIGHT,
- scalingType, false);
- REMOTE_Y = REMOTE_Y+25;
- nbmWebRTCPeer.attachRendererToRemoteStream(remoteRender, mediaStream);
- runOnUiThread(new Runnable() {
- @Override
- public void run() {
- mCallStatus.setText("");
- }
- });
- }
- @Override
- public void onRemoteStreamRemoved(MediaStream mediaStream, NBMPeerConnection nbmPeerConnection) {
- Log.i(TAG, "onRemoteStreamRemoved");
- }
- @Override
- public void onPeerConnectionError(String s) {
- Log.e(TAG, "onPeerConnectionError:" + s);
- }
- @Override
- public void onRoomResponse(RoomResponse response) {
- Log.e(TAG, "-->OnRoomResponse:" + response);
- if (Integer.valueOf(response.getId()) == publishVideoRequestId){
- SessionDescription sd = new SessionDescription(SessionDescription.Type.ANSWER,
- response.getValue("sdpAnswer").get(0));
- if (callState == CallState.PUBLISHING){
- callState = CallState.PUBLISHED;
- nbmWebRTCPeer.processAnswer(sd, "derp");
- } else if (callState == CallState.WAITING_REMOTE_USER){
- callState = CallState.RECEIVING_REMOTE_USER;
- nbmWebRTCPeer.processAnswer(sd, "remote");
- } else if (callState == CallState.PATICIPANT_JOINED){
- callState = CallState.RECEIVING_PATICIPANT;
- nbmWebRTCPeer.processAnswer(sd, this.PaticipantID);
- //NOP
- }
- }
- }
- @Override
- public void onRoomError(RoomError error) {
- Log.e(TAG, "OnRoomError:" + error);
- }
- @Override
- public void onRoomNotification(RoomNotification notification) {
- Log.e(TAG, "OnRoomNotification--> (state=" + callState.toString() + "):" + notification);
- if(notification.getMethod().equals("iceCandidate")) {
- Map<String, Object> map = notification.getParams();
- String sdpMid = map.get("sdpMid").toString();
- int sdpMLineIndex = Integer.valueOf(map.get("sdpMLineIndex").toString());
- String sdp = map.get("candidate").toString();
- IceCandidate ic = new IceCandidate(sdpMid, sdpMLineIndex, sdp);
- Log.e(TAG, "callState-->" + callState);
- if (callState == CallState.PUBLISHING || callState == CallState.PUBLISHED) {
- nbmWebRTCPeer.addRemoteIceCandidate(ic, "derp");
- }else if(callState==CallState.PATICIPANT_JOINED || callState== CallState.RECEIVING_PATICIPANT){
- nbmWebRTCPeer.addRemoteIceCandidate(ic,this.PaticipantID);
- }else {
- nbmWebRTCPeer.addRemoteIceCandidate(ic, "remote");
- }
- }
- if(notification.getMethod().equals("participantPublished"))
- {
- Map<String, Object> map = notification.getParams();
- final String user = map.get("id").toString();
- this.calluser = user;
- this.PaticipantID = "pt_"+this.calluser;
- PeerVideoActivity.this.runOnUiThread(new Runnable() {
- @Override
- public void run() {
- callState = CallState.PATICIPANT_JOINED;
- nbmWebRTCPeer.generateOffer(PaticipantID, false);
- }
- });
- }
- }
- @Override
- public void onRoomConnected() {
- }
- @Override
- public void onRoomDisconnected() {
- }
- }
再就是android room demo中的MainActivity的添加cert的代碼要去掉註釋,讓這段代碼生效,就能夠連通服務器了。
在iOS的實施方面,上面這家公司也提供了一個工具包:https://github.com/nubomediaTI/Kurento-iOS ,工具包裏面也有demo
Web方面,最上面官方的哪一個demo就足夠參考了
後記:很榮幸這篇博客得到了不少CSDN程序員的關注和詢問,這隻能證實我很榮幸有機會在去年的那個時間點(16年7月)在你們以前處理了一個後續你們都很關注的技術問題,而處理這個問題主要用到的服務器端room server項目和android端nubo test項目,官方在後續好像都作了必定的升級,反而是我本身搞完這個以後,由於產品設計的緣由,後來再沒有深刻地去生產實施這個東西,甚至開發筆記本關於這個項目的源碼項目好像都已經刪除了,對於你們提出的問題,早期的我還能答一答,後面的我估計大家用到的源碼和我用到的源碼估計都不是一個版本了,再就是裏面的代碼細節也基本忘得差很少,在這兒我建議後續開發這個功能能夠去深刻閱讀分析Kurento官方(https://github.com/Kurento)和歐洲媒體服務雲服務商nubomedia官方(https://github.com/nubomedia-vtt)的代碼示例和文檔。我面給出的代碼樣例是基於nubomedia一對一視聊樣例改的,官方原始代碼樣例在這段時間內都有了變動。在掌握大的基本WebRTC通訊的原理的前提下,我以爲改新的代碼估計也不會太難。