讓android支持RTSP及live555分析
#DATE 2012/05/09
#2012/08/27由cnblogs遷入android
如何讓Android支持C++異常機制
Android不支持C++異常機制,若是須要用到的話,則須要在編譯的時候加入比較完整的C++庫.
Android支持的C++庫能夠在Android NDK中找到(解壓後找到libsupc++.a放到代碼環境中便可):
http://www.crystax.net/en/android/ndk/7
編譯時加上參數: c++
-fexceptions -lstdc++
還須要將libsupc++.a連接上git
移植live555到Android的例子
https://github.com/boltonli/ohbee/tree/master/android/streamer/jnigithub
RTSP協議
參考: rfc2326, rfc3550, rfc3984shell
RTP Header結構[#0]ubuntu
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC |M| PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | contributing source (CSRC) identifiers | | .... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
H.264視頻格式
參考: rfc3984, 『H.264中的NAL技術』, 『H.264 NAL層解析』session
ACC音頻格式
參考: ISO_IEC_13818-7.pdf架構
live555架構分析
0 總述
0.1 這裏主要以H264+ACC爲基礎做介紹
0.2 live555中的demo說明，RTSP服務端爲live555MediaServer，openRTSP爲調試用客戶端。
0.3 能夠在live555中實現一個trace_bin的函數跟蹤流媒體數據的處理過程。
void trace_bin(const unsigned char *bytes_ptr, int bytes_num)
{
#define LOG_LINE_BYTES 16
int i, j;
for (i = 0; i <= bytes_num / LOG_LINE_BYTES; i++) {
for (j = 0;
j < ( (i < (bytes_num / LOG_LINE_BYTES))
? LOG_LINE_BYTES
: (bytes_num % LOG_LINE_BYTES) );
j++)
{
if (0 == j) printf("%04d ", i * LOG_LINE_BYTES);
if (LOG_LINE_BYTES/2 == j) printf(" ");
printf(" %02x", bytes_ptr[i * LOG_LINE_BYTES + j]);
}
printf("\n");
}
}
1 宏觀流程
1.1 對每一個播放請求創建一個session，並對應音視頻創建subsession,subsession則是具體處理流媒體的單位。
------------------------------------------------------[#1]-- session <---> client requst | subsession <---> audio/video ------------------------------------------------------------
1.2 數據處理流程:
------------------------------------------------------[#2]--
source --> filter(source) ... --> sink
| | |
+-------+-------------------+ |
| v
v subsession.createNewRTPSink()
subsession.createNewStreamSource()
------------------------------------------------------------
1.3 BasicTaskScheduler::SingleStep()
BasicTaskScheduler是live555的任務處理器，他的主要工做都是在SingleStep()中完成的.
在SingleStep()中主要完成下面三種工做:
void BasicTaskScheduler::SingleStep(unsigned maxDelayTime) {
//1. 處理io任務
...
int selectResult = select(fMaxNumSockets, &readSet, &writeSet,
&exceptionSet, &tv_timeToDelay);
...
while ((handler = iter.next()) != NULL) {
...
(*handler->handlerProc)(handler->clientData, resultConditionSet);
break;
}
...
//2. handle any newly-triggered event
...
//3. handle any delayed event
fDelayQueue.handleAlarm();
}
RTSP請求、連接創建、開始播放處理主要是在1中完成的，而視頻播放主要是在3中完成。
以ACC播放爲例：
void ADTSAudioFileSource::doGetNextFrame() {
// 讀取數據並作一些簡單處理
...
int numBytesRead = fread(fTo, 1, numBytesToRead, fFid);
...
// 將FramedSource::afterGetting加入fDelayQueue中
// FramedSource::afterGetting會處理讀取到數據，並又會調用
// ADTSAudioFileSource::doGetNextFrame()，這樣實現循環讀取文件。
nextTask() = envir().taskScheduler().scheduleDelayedTask(0,
(TaskFunc*)FramedSource::afterGetting, this);
}
1.4 DelayQueue
fDelayQueue是一個須要處理的任務的隊列，每次SingleStep()只會執行第一個任務head()，這裏的任務對應DelayQueue的元素，DelayQueue的各個元素都會有本身的DelayTime，用來表示延時多久後執行。而隊列中的元素即是按照DelayTime有小到大排列的，元素中fDeltaTimeRemaining記錄的是該元素相對於它以前元素的延時。參照函數DelayQueue::addEntry()即可看出是如何入隊列的。
例如([]中的數字即是相對延時(fDeltaTimeRemaining))：
[0]->[1]->[3]->[2]->...->[1]->NULL ^ | head()
在處理DelayQueue時每每都要先作一次計時同步操做synchronize()，由於DelayQueue中元素的延時都是相對的，因此通常只要處理首元素便可，不過若是同步以後延時有小於0的，便都會改成DELAY_ZERO(即表示須要當即執行的)。
執行任務：
void DelayQueue::handleAlarm() {
...
toRemove->handleTimeout();
}
void AlarmHandler::handleTimeout() {
(*fProc)(fClientData);
DelayQueueEntry::handleTimeout();
}
任務在處理完成後便會被刪除。
2 類關係
* live555的流程分析主要就放在這個章節中，若是有須要參考函數關係或者對象關係的請參考3, 4兩個章節。
2.1 涉及到的主要類的關係圖:
------------------------------------------------------[#3]--
Medium
+ServerMediaSubsession
| +OnDemandServerMediaSubsession
| +FileServerMediaSubsession
| +H264VideoFileServerMediaSubsession //h264
| +ADTSAudioFileServerMediaSubsession //aac
|
+MediaSink
| +RTPSink
| +MultiFramedRTPSink
| +VideoRTPSink
| | +H264VideoRTPSink //h264
| +MPEG4GenericRTPSink //aac
|
+MediaSource
+FramedSource //+doGetNextFrame(); +fAfterGettingFunc;
+FramedFilter
| +H264FUAFragmenter //h264
| +MPEGVideoStreamFramer
| +H264VideoStreamFramer //h264
+FramedFileSource
+ByteStreamFileSource //h264
+ADTSAudioFileSource //acc
StreamParser
+MPEGVideoStreamParser
+H264VideoStreamParser //h264
------------------------------------------------------------
咱們看下FramedFilter和FramedFileSource相對於FramedSource增長了哪些成員:
FramedFilter {
FramedSource* fInputSource;
}
FramedFileSource {
FILE* fFid;
}
從二者的命名和增長的成員能夠看出各自的做用。FramedFilter即是對應着[#2]中的filter，而FramedFileSource則是以本地文件爲輸入的source。
2.2 如何實現帶有filter流程:
這便用到了FramedFilter中的fInputSource成員，以H264爲例，
H264VideoStreamFramer.fInputSource = ByteStreamFileSource; H264FUAFragmenter.fInputSource = H264VideoStreamFramer;
將上游source賦值到下游filter的fInputSource便可，對於H264即可以獲得下面的一個處理流程:
ByteStreamFileSource -> H264VideoStreamFramer -> H264FUAFragmenter -> H264VideoRTPSink
在H264VideoStreamFramer的父類MPEGVideoStreamFramer中也有新增成員，
MPEGVideoStreamFramer {
MPEGVideoStreamParser* fParser;
}
MPEGVideoStreamFramer.fParser = H264VideoStreamParser;
H264VideoStreamParser是用來filter過程當中處理視頻數據的。
在MultiFramedRTPSink::buildAndSendPacket()中添加RTP頭[#0]。
3 函數關係
3.1 H264函數調用關係
------------------------------------------------------[#4]--
RTSPServer::RTSPClientSession::handleCmd_SETUP()
OnDemandServerMediaSubsession::getStreamParameters(
streamToken: new StreamState(
fMediaSource: H264VideoFileServerMediaSubsession::createNewStreamSource() )
)
**********
RTSPServer::RTSPClientSession::handleCmd_DESCRIBE()
ServerMediaSession::generateSDPDescription()
OnDemandServerMediaSubsession::sdpLines()
H264VideoFileServerMediaSubsession::createNewStreamSource()
H264VideoStreamFramer::createNew( fInputSource: ByteStreamFileSource::createNew(),
fParser: new H264VideoStreamParser(
fInputSource: H264VideoStreamFramer.fInputSource) )
**********
RTSPServer::RTSPClientSession::handleCmd_PLAY()
H264VideoFileServerMediaSubsession::startStream() [OnDemandServerMediaSubsession::startStream()]
StreamState::startPlaying()
H264VideoRTPSink::startPlaying() [MediaSink::startPlaying(fMediaSource)]//got in handleCmd_SETUP()
H264VideoRTPSink::continuePlaying()
fSource, fOurFragmenter: H264FUAFragmenter(fInputSource: fMediaSource)
MultiFramedRTPSink::continuePlaying()
MultiFramedRTPSink::buildAndSendPacket()
MultiFramedRTPSink::packFrame()
H264FUAFragmenter::getNextFrame() [FramedSource::getNextFrame()]
H264FUAFragmenter::doGetNextFrame() {1}
1)=No NALU=
H264VideoStreamFramer::getNextFrame() [FramedSource::getNextFrame()]
MPEGVideoStreamFramer::doGetNextFrame()
H264VideoStreamParser::registerReadInterest()
MPEGVideoStreamFramer::continueReadProcessing()
H264VideoStreamParser::parse()
H264VideoStreamFramer::afterGetting() [FramedSource::afterGetting()]
H264FUAFragmenter::afterGettingFrame()
H264FUAFragmenter::afterGettingFrame1()
goto {1} //Now we have got NALU
2)=Has NALU=
FramedSource::afterGetting()
MultiFramedRTPSink::afterGettingFrame()
MultiFramedRTPSink::afterGettingFrame1()
MultiFramedRTPSink::sendPacketIfNecessary()
------------------------------------------------------------
4 對象關係
4.1 H264對象關係圖
------------------------------------------------------[#5]--
ServerMediaSession{#1}.fSubsessionsTail = H264VideoFileServerMediaSubsession{2}.fParentSession = {1}
fStreamStates[] {
.subsession = {2}
.streamToken = StreamState {
.fMaster = {2}
.fRTPSink = H264VideoRTPSink{5}.fSource/fOurFragmenter
= H264FUAFragmenter{4} {
.fInputSource = H264VideoStreamFramer{3}
.fAfterGettingFunc = MultiFramedRTPSink::afterGettingFrame()
.fAfterGettingClientData = {5}
.fOnCloseFunc = MultiFramedRTPSink::ourHandleClosure()
.fOnCloseClientData = {5}
}
.fMediaSource = {3} {
.fParser = H264VideoStreamParser {
.fInputSource = ByteStreamFileSource{6}
.fTo = [{5}.]fOutBuf->curPtr()
}
.fInputSource = {6}
.fAfterGettingFunc = H264FUAFragmenter::afterGettingFrame()
.fAfterGettingClientData = {4}
.fOnCloseFunc = FramedSource::handleClosure()
.fOnCloseClientData = {4}
}
}
}
------------------------------------------------------------
4.2 AAC對象關係圖
------------------------------------------------------[#6]--
ServerMediaSession{1}.fSubsessionsTail = ADTSAudioFileServerMediaSubsession{2}.fParentSession = {1}
fStreamStates[] {
.subsession = {2}
.streamToken = StreamState {
.fMaster = {2}
.fRTPSink = MPEG4GenericRTPSink {
.fOutBuf = OutPacketBuffer
.fSource = ADTSAudioFileSource {3}
.fRTPInterface = RTPInterface.fGS = Groupsock
}
.fMediaSource = {3}
}
}
------------------------------------------------------------
5 RTSP
5.1 RTSP命令和處理函數的對應關係:
RTSP命令 live555中處理函數 --------------------------------------------- OPTIONS <---> handleCmd_OPTIONS DESCRIBE <---> handleCmd_DESCRIBE SETUP <---> handleCmd_SETUP PLAY <---> handleCmd_PLAY PAUSE <---> handleCmd_PAUSE TEARDOWN <---> handleCmd_TEARDOWN GET_PARAMETER <---> handleCmd_GET_PARAMETER SET_PARAMETER <---> handleCmd_SET_PARAMETER
5.2 RTSP播放交互示例(openRTSP)
-------------------------------------------------------------------------------- ubuntu$ ./openRTSP rtsp://192.168.43.1/grandma.264 Opening connection to 192.168.43.1, port 554... ...remote connection opened Sending request: OPTIONS rtsp://192.168.43.1/grandma.264 RTSP/1.0 CSeq: 2 User-Agent: ./openRTSP (LIVE555 Streaming Media v2012.02.29) Received 152 new bytes of response data. Received a complete OPTIONS response: RTSP/1.0 200 OK CSeq: 2 Date: Tue, Jan 25 2011 21:02:53 GMT Public: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETER -------------------------------------------------------------------------------- Sending request: DESCRIBE rtsp://192.168.43.1/grandma.264 RTSP/1.0 CSeq: 3 User-Agent: ./openRTSP (LIVE555 Streaming Media v2012.02.29) Accept: application/sdp Received 682 new bytes of response data. Received a complete DESCRIBE response: RTSP/1.0 200 OK CSeq: 3 Date: Tue, Jan 25 2011 21:02:53 GMT Content-Base: rtsp://192.168.43.1/grandma.264/ Content-Type: application/sdp Content-Length: 517 v=0 o=- 1295989373493698 1 IN IP4 0.0.0.0 s=H.264 Video, streamed by the LIVE555 Media Server i=grandma.264 t=0 0 a=tool:LIVE555 Streaming Media v2012.02.04 a=type:broadcast a=control:* a=range:npt=0- a=x-qt-text-nam:H.264 Video, streamed by the LIVE555 Media Server a=x-qt-text-inf:grandma.264 m=video 0 RTP/AVP 96 c=IN IP4 0.0.0.0 b=AS:500 a=rtpmap:96 H264/90000 a=fmtp:96 packetization-mode=1;profile-level-id=4D4033; sprop-parameter-sets=Z01AM5p0FidCAAADAAIAAAMAZR4wZUA=,aO48gA== a=control:track1 Opened URL "rtsp://192.168.43.1/grandma.264", returning a SDP description: v=0 o=- 1295989373493698 1 IN IP4 0.0.0.0 s=H.264 Video, streamed by the LIVE555 Media Server i=grandma.264 t=0 0 a=tool:LIVE555 Streaming Media v2012.02.04 a=type:broadcast a=control:* a=range:npt=0- a=x-qt-text-nam:H.264 Video, streamed by the LIVE555 Media Server a=x-qt-text-inf:grandma.264 m=video 0 RTP/AVP 96 c=IN IP4 0.0.0.0 b=AS:500 a=rtpmap:96 H264/90000 a=fmtp:96 packetization-mode=1;profile-level-id=4D4033; sprop-parameter-sets=Z01AM5p0FidCAAADAAIAAAMAZR4wZUA=,aO48gA== a=control:track1 Created receiver for "video/H264" subsession (client ports 56488-56489) -------------------------------------------------------------------------------- Sending request: SETUP rtsp://192.168.43.1/grandma.264/track1 RTSP/1.0 CSeq: 4 User-Agent: ./openRTSP (LIVE555 Streaming Media v2012.02.29) Transport: RTP/AVP;unicast;client_port=56488-56489 Received 205 new bytes of response data. Received a complete SETUP response: RTSP/1.0 200 OK CSeq: 4 Date: Tue, Jan 25 2011 21:02:53 GMT Transport: RTP/AVP;unicast;destination=192.168.43.244;source=192.168.43.1; client_port=56488-56489;server_port=6970-6971 Session: 7626020D Setup "video/H264" subsession (client ports 56488-56489) Created output file: "video-H264-1" -------------------------------------------------------------------------------- Sending request: PLAY rtsp://192.168.43.1/grandma.264/ RTSP/1.0 CSeq: 5 User-Agent: ./openRTSP (LIVE555 Streaming Media v2012.02.29) Session: 7626020D Range: npt=0.000- Received 186 new bytes of response data. Received a complete PLAY response: RTSP/1.0 200 OK CSeq: 5 Date: Tue, Jan 25 2011 21:02:53 GMT Range: npt=0.000- Session: 7626020D RTP-Info: url=rtsp://192.168.43.1/grandma.264/track1;seq=26490;rtptime=1809652062 Started playing session Receiving streamed data (signal with "kill -HUP 6297" or "kill -USR1 6297" to terminate)... Received RTCP "BYE" on "video/H264" subsession (after 35 seconds) -------------------------------------------------------------------------------- Sending request: TEARDOWN rtsp://192.168.43.1/grandma.264/ RTSP/1.0 CSeq: 6 User-Agent: ./openRTSP (LIVE555 Streaming Media v2012.02.29) Session: 7626020D Received 65 new bytes of response data. Received a complete TEARDOWN response: RTSP/1.0 200 OK CSeq: 6 Date: Tue, Jan 25 2011 21:03:28 GMT --------------------------------------------------------------------------------*** END ***
相關文章
- 1. LIVE555再學習 -- live555實現RTSP直播服務器 分析
- 2. 優化擴展live555支持MP2T（mpeg2ts）RTSP拉流附源代碼
- 3. Live555-SingleStep分析及驗證
- 4. [live555]rtsp直播基於live555的實現
- 5. 讓Android studio支持java8
- 6. Android：讓WebView支持元素
- 7. 讓Android WiFi支持中文
- 8. 讓Android的WebView支持WebSocket
- 9. 提供HTML5播放RTSP流 提供微信播放RTSP流 HTML5支持rtsp web播放rtsp，微信支持rtsp
- 10. live555 設置RTP over RTSP(TCP)
- 更多相關文章...
- • SVN分支 - SVN 教程
- • Git 分支管理 - Git 教程
- • Flink 數據傳輸及反壓詳解
- • Git五分鐘教程
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. LIVE555再學習 -- live555實現RTSP直播服務器 分析
- 2. 優化擴展live555支持MP2T（mpeg2ts）RTSP拉流附源代碼
- 3. Live555-SingleStep分析及驗證
- 4. [live555]rtsp直播基於live555的實現
- 5. 讓Android studio支持java8
- 6. Android：讓WebView支持元素
- 7. 讓Android WiFi支持中文
- 8. 讓Android的WebView支持WebSocket
- 9. 提供HTML5播放RTSP流 提供微信播放RTSP流 HTML5支持rtsp web播放rtsp，微信支持rtsp
- 10. live555 設置RTP over RTSP(TCP)