GB28181的PS流徹底分析(封裝 / 分包發送 / 接收組包 / 解析)

1、PS流傳輸格式預覽html

一、視頻關鍵幀的封裝 RTP + PS header + PS system header + PS system Map + PES header +h264 data
二、視頻非關鍵幀的封裝 RTP +PS header + PES header + h264 data
三、音頻幀的封裝: RTP + PES header + G711tcp

2、服務端PS流接收組包和解析ide

以海康DS-IPC-B12H2-I爲例優化

1. rtp over udpspa

80 60 00 00 00 00 00 00 00 00 04 00 00 00 01 ba
44 f0 4f 69 64 01 02 5f 03 fe ff ff 00 01 11 0c
00 00 01 bb 00 12 81 2f 81 04 e1 7f e0 e0 80 c0
c0 08 bd e0 80 bf e0 80 00 00 01 bc 00 5e f8 ff
00 24 40 0e 48 4b 01 00 14 14 40 16 6b bf 00 ff
ff ff 41 12 48 4b 00 01 02 03 04 05 06 07 08 09
0a 0b 0c 0d 0e 0f 00 30 1b e0 00 1c 42 0e 07 10
10 ea 05 00 02 d0 11 30 00 00 1c 21 2a 0a 7f ff
00 00 07 08 1f fe a0 5a 90 c0 00 0c 43 0a 01 40
fe 00 7d 03 03 e8 03 ff f6 53 94 03 00 00 01 e0
00 1e 8c 80 08 21 3c 13 da 59 ff ff fc 00 00 00
01 67 4d 00 1f 96 35 40 a0 0b 74 dc 04 04 04 08
00 00 01 e0 00 0e 8c 00 03 ff ff fc 00 00 00 01
68 ee 3c 80 00 00 01 e0 00 0e 8c 00 02 ff fc 00
00 00 01 06 e5 01 d5 80 00 00 01 e0 c2 06 8c 00
05 ff ff ff ff f8 00 00 00 01 65 b8 00 00 0a 77
80 00 00 4f e7 e5 34 0f f3 41 4b b9 58 a9 4e 4c
f4 ea 04 0e 32 a5 f9 51 df cc 4c b8 99 f2 cf 16
3e 32 19 ed 86 df 05 6b fc 21 5e 0f 87 90 20 c3
16 02 03 73 0f a3 d2 9b 52 1b b1 a7 7c b4 61 6d
d9 aa f4 5d 34 f6 49 d4 f6 72 af b6 c7 11 c0 ff
3d 1b fd e3 5d 41 db 32 3a c7 9f f4 f2 c0 99 e6 ...3d

數據解析，這是個I幀code

rtp header 80 60 00 00 00 00 00 00 00 00 04 00orm

——12byte固定長度視頻

ps header 00 00 01 ba ...htm

——00 00 01 ba 44 f0 4f 69 64 01 02 5f 03 fe ff ff 00 01 11 0c 前14byte是固定的，第14byte 0xfe & 0x07 = 0x0e 也就是後面拓展6byte

ps system header 00 00 01 bb ...

——隨後的 00 12是長度，也就是ps system header長度=4+18 byte

ps system map 00 00 01 bc ...

——一樣隨後的 00 5e是長度，也就是ps system map的長度=4+94 byte

pes header 00 00 01 e0/c0 ...

——e0是視頻，c0是音頻，一樣隨後的00 1e是長度，也就是pes header的長度 4+30 byte，剩下的就是ps payload數據

ps payload SPS 00 00 00 01 67 ...

ps payload PPS 00 00 00 01 68 ...

ps payload I 00 00 00 01 65 ...

ps payload P 00 00 00 01 61/41 ...

海康的攝像頭ps payload的起始標識是00 00 00 01 61，有些廠家的是00 00 00 01 41，經過第五字節 & 0x1F = 1，就是正確的ps payload起始標識

另外：

0x000001BD 私有數據，同0x000001E0 ，直接跳過
具體的PS格式能夠參考網上的其餘資料，另外，若是數據中包含0x000001，按h264協議會進行轉義，即變成0x00000301，涉及到3個轉義 0x000001 -> 0x00000301 0x000002 -> 0x00000302 0x000003 -> 0x00000303

接收，解析流程 udp--->rtp--->ps--->h264

while ((pack = sess.GetNextPacket()) != NULL) { loaddata = pack->GetPayloadData(); len = pack->GetPayloadLength(); /* payload type: ps */
    if(pack->GetPayloadType() == 96) { /*the last packet*/
        if(pack->HasMarker()) { if(pos + len < PS_BUFFER_SIZE){ memcpy(&buff[pos],loaddata,len); printf("!!! GetPayload len = %ld !!!! \n ",pos+len); size_t r = ps_demuxer_input(ps, buff, pos+len); pos = 0; lasttime = nowtime; } } else { if(pos + len < PS_BUFFER_SIZE){ memcpy(&buff[pos],loaddata,len); pos = pos + len; } } } else { printf("!!! GetPayloadType = %d !!!! \n ",pack->GetPayloadType()); } sess.DeletePacket(pack); }

將解析出來的h264視頻和g711音頻填充到待處理的列隊

vedio

/* SPS frame 00 00 00 01 67 PPS frame 00 00 00 01 68 I frame 00 00 00 01 65 P slice 00 00 00 01 41/61 */
if(FindStartCode(p)) { data_t d; if(rb_numitems(task->buffer) < DATA_ITEM_NMAX){ if(task->pos < ITEM_BUFFER_SIZE) { memcpy(d.buf,task->buf, task->pos); d.size = task->pos; d.type = H264; rb_put(task->buffer, &d); fflush(stdout); } } task->pos = 0; if((task->pos + bytes) < TASK_BUFFER_SIZE){ memcpy(&(task->buf[task->pos]),p,bytes); task->pos = task->pos + bytes; } } else { if((task->pos + bytes) < TASK_BUFFER_SIZE){ memcpy(&(task->buf[task->pos]),p,bytes); task->pos = task->pos + bytes; } }

audio

data_t d; if (rb_numitems(task->buffer) < DATA_ITEM_NMAX){ if(bytes < ITEM_BUFFER_SIZE){ memcpy(d.buf, p, bytes); d.size = bytes; d.type = G711A; rb_put(task->buffer, &d); fflush(stdout); } }

h264 nalu數據幀解析能夠參考live555的處理

H264VideoRTPSink.cpp

Boolean H264VideoRTPSource ::processSpecialHeader(BufferedPacket* packet, unsigned& resultSpecialHeaderSize) { unsigned char* headerStart = packet->data(); unsigned packetSize = packet->dataSize(); unsigned numBytesToSkip; // Check the 'nal_unit_type' for special 'aggregation' or 'fragmentation' packets:
  if (packetSize < 1) return False; fCurPacketNALUnitType = (headerStart[0]&0x1F); switch (fCurPacketNALUnitType) { case 24: { // STAP-A
    numBytesToSkip = 1; // discard the type byte
    break; } case 25: case 26: case 27: { // STAP-B, MTAP16, or MTAP24
    numBytesToSkip = 3; // discard the type byte, and the initial DON
    break; } case 28: case 29: { // // FU-A or FU-B // For these NALUs, the first two bytes are the FU indicator and the FU header. // If the start bit is set, we reconstruct the original NAL header into byte 1:
    if (packetSize < 2) return False; unsigned char startBit = headerStart[1]&0x80; unsigned char endBit = headerStart[1]&0x40; if (startBit) { fCurrentPacketBeginsFrame = True; headerStart[1] = (headerStart[0]&0xE0)|(headerStart[1]&0x1F); numBytesToSkip = 1; } else { // The start bit is not set, so we skip both the FU indicator and header:
      fCurrentPacketBeginsFrame = False; numBytesToSkip = 2; } fCurrentPacketCompletesFrame = (endBit != 0); break; } default: { // This packet contains one complete NAL unit:
    fCurrentPacketBeginsFrame = fCurrentPacketCompletesFrame = True; numBytesToSkip = 0; break; } } resultSpecialHeaderSize = numBytesToSkip; return True; }

H264VideoRTPSink.cpp

H264VideoRTPSink* H264VideoRTPSink ::createNew(UsageEnvironment& env, Groupsock* RTPgs, unsigned char rtpPayloadFormat, char const* sPropParameterSetsStr) { u_int8_t* sps = NULL; unsigned spsSize = 0; u_int8_t* pps = NULL; unsigned ppsSize = 0; unsigned numSPropRecords; SPropRecord* sPropRecords = parseSPropParameterSets(sPropParameterSetsStr, numSPropRecords); for (unsigned i = 0; i < numSPropRecords; ++i) { if (sPropRecords[i].sPropLength == 0) continue; // bad data
    u_int8_t nal_unit_type = (sPropRecords[i].sPropBytes[0])&0x1F; if (nal_unit_type == 7/*SPS*/) { sps = sPropRecords[i].sPropBytes; spsSize = sPropRecords[i].sPropLength; } else if (nal_unit_type == 8/*PPS*/) { pps = sPropRecords[i].sPropBytes; ppsSize = sPropRecords[i].sPropLength; } } H264VideoRTPSink* result = new H264VideoRTPSink(env, RTPgs, rtpPayloadFormat, sps, spsSize, pps, ppsSize); delete[] sPropRecords; return result; }

或者h264bitstream也有h264 nalu的解析

https://www.cnblogs.com/dong1/p/10149980.html

幀屬性分離

SPS: 0x67 header & 0x1F = 7

PPS: 0x68 header & 0x1F = 8

SEI: 0x66 header & 0x1F = 6

I Frame: 0x65 header & 0x1F = 5

P Frame: 0x41 header & 0x1F = 1

h264 nalu的解析偏題了，順帶提了一下，這裏主要解析ps流。

2. rtp over tcp

(len)05 84 (rtp header)80 60 6d ee 00 b5 62 60 00 00 a5 f6 (ps header)00 00
01 ba 44 76 55 85 74 01 02 5f 03 fe ff ff 00 00
86 24 (pes header)00 00 01 e0 21 ba 8c 80 0a 21 1d 95 61 5d
ff ff ff ff f8 (ps payload P)00 00 00 01 61 e0 40 00 59 13 ff
01 23 44 a1 02 38 33 0f 99 df 89 95 01 9e 6d 31
00 2a 8f 05 a5 fb 96 67 38 b8 7f c5 73 bb 25 b6
96 3d 0c 15 0e a4 ed 95 30 6b 43 35 51 9a 04 a1
89 26 6a 6a fc 64 c6 44 37 2a 32 6d 16 12 41 83
53 42 d7 66 e3 51 6b 8e bc 8f 40 73 2a 22 9d a0
d7 b9 c1 ed f8 a5 14 91 2d 8e 90 07 0e b4 2e 4a
0e cb 03 4b 73 f4 1a 49 0a d3 1f bb 72 c5 28 13
b7 9b 35 a0 18 3a c0 91 73 99 1d 4c dd 3b fd eb
ce 8e 73 79 34 8a 05 6d 98 d6 a9 20 d3 43 44 d9
b3 cd be 5b f6 74 86 f4 67 26 2f a1 be fb 5c 2c
aa 81 4d 51 85 06 c7 65 82 52 47 05 5b ae 76 93

數據解析，這是個P幀

len 05 84

rtp header 80 60 6d ee 00 b5 62 60 00 00 a5 f6

ps header 00 00 01 ba ...

pes header 00 00 01 e0 ...

ps payload P 00 00 00 01 61/41 ...

RTP over TCP模式比RTP over UDP模式多了長度字段，可是經過長度信息組包，組成完整的TCP包，完整的TCP包去掉長度信息就是RTP包。而後因爲tcp底層會做拆包和粘包的優化處理，因此應用層須要特殊處理，最好用jrtplib的tcp模式，jrtplib庫已經處理好了拆包和粘包。

接收，解析流程 tcp--->rtp--->ps--->h264

提供一個示例代碼

RTP over TCP/UDP example in jrtplib

https://www.cnblogs.com/dong1/p/12179996.html

3、設備端PS流封裝和發送

既然從設備接收到的數據已經分析得比較完全，那麼如何反過來封包，分片，發送呢？

最麻煩得ps封裝能夠照抄 ffmpeg-4.1/doc/examples/muxing.c

能夠直接將aac+h264(也支持其餘一些音頻視頻格式)封裝成PS流，PS流封裝麻煩，封好以後，切片，發送不麻煩。

封好的ps buf，按FU-A的格式，每隔1400 byte切一片，每片往前添加個12 byte的rtp header，最後一個切片置位Marker，往外發就行。

rtp over tcp模式下，rtp heade前還得加2 byte長度，因此rtp over tcp頭是14 byte, rtp over udp頭是12byte.

各類分片細節看rfc3984 https://datatracker.ietf.org/doc/rfc3984/