Internet Engineering Task Force J. Arbeiter, Ed. Internet-Draft Harris Corporation Intended status: Standards Track J. Downs, Ed. Expires: June 12, 2010 PAR Government Systems Corp. December 9, 2009 RTP Payload Format for SMPTE 336M Encoded Data draft-arbeiter-rtp-klv-01.txt Abstract This document describes a mechanism for carriage of KLV (Key-Length- Value) Encoded Data, as defined by the Society of Motion Picture and Television Engineers (SMPTE) in SMPTE 336M. One payload format is described for transmitting KLV Encoded Data on a separate RTP session dedicated for the transmission of KLV Encoded Data. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on June 12, 2010. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. Arbeiter & Downs Expires June 12, 2010 [Page 1] Internet-Draft RTP Format for SMPTE 336M Data December 2009 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 2. Usage of RTP . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1. Payload Format for Transmission of KLV Encoded Data . . . 4 2.2. The KLVunit . . . . . . . . . . . . . . . . . . . . . . . 5 2.3. KLVunit Mapping to RTP Packet Payload . . . . . . . . . . 5 2.4. Synchronization of KLV Encoded Data with Other Media . . . 5 2.5. RTP Packet Header . . . . . . . . . . . . . . . . . . . . 6 3. Loss of Data . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1. Damaged KLVunits . . . . . . . . . . . . . . . . . . . . . 7 3.2. Treatment of Damaged KLVunits . . . . . . . . . . . . . . 7 4. Recommended Procedure . . . . . . . . . . . . . . . . . . . . 8 4.1. Recommended Basic Procedure . . . . . . . . . . . . . . . 8 4.2. Detection of Lost KLV Packets . . . . . . . . . . . . . . 8 4.3. Compensation for Packets Out of Order . . . . . . . . . . 8 5. Security Considerations . . . . . . . . . . . . . . . . . . . 9 5.1. Confidentiality . . . . . . . . . . . . . . . . . . . . . 9 5.2. Integrity . . . . . . . . . . . . . . . . . . . . . . . . 9 5.3. Source Authentication . . . . . . . . . . . . . . . . . . 9 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 6.1. Registration of MIME media type application/smpte336m . . 10 7. SDP Mapping . . . . . . . . . . . . . . . . . . . . . . . . . 10 8. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 8.1. RTP Packetization example for the application/smpte336m format . . . . . . . . . . . . . . . 11 8.2. SDP example . . . . . . . . . . . . . . . . . . . . . . . 11 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 9.1. Normative References . . . . . . . . . . . . . . . . . . . 11 9.2. Informative References . . . . . . . . . . . . . . . . . . 12 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 Arbeiter & Downs Expires June 12, 2010 [Page 2] Internet-Draft RTP Format for SMPTE 336M Data December 2009 1. Introduction [SMPTE336M], Data Encoding Protocol Using Key-Length-Value, defines a byte-level data encoding protocol for representing data items and data groups. This encoding protocol definition is independent of the application or transportation method used. SMPTE 336M data encoding can be applied to a wide variety of binary data. This encoding has been used to provide diverse and rich metadata sets that describe or enhance associated video presentations. Use of SMPTE 336M encoded metadata in conjunction with video has enabled improvements in multimedia presentations, content management and distribution, archival and retrieval, and production workflow. The SMPTE 336M standard defines a Key-Length-Value (KLV) triplet as a data interchange protocol for data items or data groups where the Key identifies the data, the Length specifies the length of the data and the Value is the data itself. The KLV protocol provides a common interchange point for all compliant applications irrespective of the method of implementation or transport. The standard also provides methods for combining associated KLV triplets in data sets where the set of KLV triplets is itself coded with KLV data coding protocol. Such sets can be coded in either full form (Universal Sets) or in one of four increasingly bit-efficient forms (Global Sets, Local Sets, Variable Length Packs and Defined Length Packs). The standard provides a definition of each of these data constructs. The standard also describes implications of KLV coding including the use of a SMPTE Universal Label as a value within a KLV coding triplet or whose meaning is entirely conveyed by the SMPTE UL itself. The two kinds of usage for such standalone SMPTE Universal Labels are a) as a value in a K L V construct and b) as a Key that has no Length and no Value. This standard defines where SMPTE ULs may be used for each kind of construct. The standard also defines the use of KLV coding to provide a means to carry information that is registered with a non-SMPTE external agency. The encoding byte range (length of the payload) may accommodate unusually large volumes of data. Consequently, a specific application of KLV encoding may require only a limited operating data range and those details shall be defined in a relevant application document. Arbeiter & Downs Expires June 12, 2010 [Page 3] Internet-Draft RTP Format for SMPTE 336M Data December 2009 KLV Encoded Data is often used to express metadata that accompanies media streams. Users expect that the metadata will be delivered with no, or a low level, of lost information. Therefore, a mechanism based on RTP is specified here. It enables metadata arrival in correct order, and with detection and indication of loss. By using RTP for metadata transmission in multimedia application, uniform handling of metadata and other media can be achieved in, for example, acquisition systems, firewalls, and network translation devices. This, in turn, facilitates the design for either combined or separate media delivery as a function of application needs and channel capacity. 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2. Usage of RTP The payload format for real-time SMPTE 336M Encoded Data transmission with RTP [RFC3550] described in this memo is intended for general binary data constructs, assembled in accordance with [SMPTE336M], and uses the application/smpte336m MIME type. 2.1. Payload Format for Transmission of KLV Encoded Data A KLV Encoded Data RTP payload format consists of one, and only one, block of KLV Encoded Data, referred to as a "KLVunit" (see Section 2.2). The standard 12-byte RTP header is assumed. The structure of the RTP packet is shown below in Table 1. +-------------------+----------------+---------------------+ | Field | Length | Semantics | +-------------------+----------------+---------------------+ | RTP packet header | 96 (see below) | Standard RTP header | | Payload | Remainder | KLVunit | +-------------------+----------------+---------------------+ Table 1: Structure of RTP Packet There are no additional headers specific to this payload format. The fields in the RTP header are set as defined in Section 2.3, carried in network byte order (see [RFC0791]). The RTP packet header may be Arbeiter & Downs Expires June 12, 2010 [Page 4] Internet-Draft RTP Format for SMPTE 336M Data December 2009 longer than 96 bits if CSRC fields are used (as defined in [RFC3880]). 2.2. The KLVunit A KLVunit is a collection of all KLV items that are to be presented at a specific time. A KLVunit is comprised of one or more self- contained KLV items, where a self-contained KLV item consists of a universal key, length, and value triplet. Compound items (sets, packs) are allowed as per [SMPTE336M], but the contents of a compound item MUST NOT be split across two KLVunits. Multiple KLV items in a KLVunit occur one after another with no padding or stuffing between items. 2.3. KLVunit Mapping to RTP Packet Payload An RTP packet payload SHALL contain one, and only one, KLVunit or a segment thereof. KLVunits small enough to fit into a single RTP packet (RTP packet size is up to implementation but should consider underlying transport/network factors such as MTU limitations) are placed directly into the payload of the RTP packet, with the first byte of the KLVunit (which is the first byte of a KLV universal key) being the first byte of the RTP packet payload. KLVunits too large to fit into a single RTP packet payload may span multiple RTP packet payloads. All RTP packets comprising the parts of a KLVunit MUST have the same timestamp. KLVunits are bounded with changes in RTP packet timestamps. The marker (M) bit in the RTP packet headers marks the last RTP packet comprising a KLVunit. 2.4. Synchronization of KLV Encoded Data with Other Media Usually, each medium in a session utilizes a separate RTP stream. As such, if synchronization of the KLV Encoded Data and other media packets is important, the streams MUST be associated when the sessions are established and the streams MUST share the same reference clock (refer to the description of the timestamp field as it relates to synchronization in Section 5.1 of [RFC3550]). One method of associating RTP streams is to use the CNAME field found in RTCP SDES packets. The choice of a specific method for stream association is dependent on the particular application and is outside the scope of this document. Arbeiter & Downs Expires June 12, 2010 [Page 5] Internet-Draft RTP Format for SMPTE 336M Data December 2009 2.5. RTP Packet Header Each RTP packet starts with a fixed RTP header. The following fields of the RTP fixed header are specified for SMPTE 336M KLV Encoded Data streams: +-----------+-------------------------------------------------------+ | Field | Usage | +-----------+-------------------------------------------------------+ | Payload | The assignment of an RTP payload type is specific to | | Type (PT) | the RTP profile under which the payload format is | | | used. For profiles that use dynamic payload type | | | number assignment, this payload format can be | | | identified by the MIME type "application/smpte336m". | | Sequence | The definition of sequence numbers is available in | | number | [RFC3550]. When transmitting KLV metadata using the | | | payload format for application/smpt336m, it is used | | | for detection of packet loss and out-of-order | | | packets, and reordering of KLVunits and marking | | | missing KLVunits. | | Timestamp | The RTP Timestamp encodes the approximate instant of | | | the start of the KLV encoded data in the packet. A | | | clock frequency equal to that of the video time base | | | MUST be used. Because packets do not represent any | | | constant duration, the timestamp cannot be used to | | | directly infer packet loss. | | M-bit | The M-bit MUST be included. The marker bit shall be | | | set to '1' for any RTP packet which contains the | | | final byte of a KLVunit. All other RTP packets shall | | | have the marker bit set to '0'. This allows | | | receivers to pass a KLVunit for parsing/decoding | | | immediately upon receipt of the last RTP packet | | | comprising the KLVunit. Without this, a receiver | | | would need to wait for the next RTP packet with a | | | different timestamp to arrive, thus signaling the end | | | of one KLVunit and the start of another. | | V | Version number (currently = 2) | | P | Padding bit. If padding is added the P bit is set. | | X | The X bit is set to denote that extension headers are | | | present after the fixed RTP header. X=0 in this | | | spec. | | CC | The number of contributing sources (CSRCs) if any. | +-----------+-------------------------------------------------------+ Arbeiter & Downs Expires June 12, 2010 [Page 6] Internet-Draft RTP Format for SMPTE 336M Data December 2009 3. Loss of Data RTP is generally deployed in network environments where packet loss may occur. Receivers MUST be prepared to accept gaps in KLV data caused by RTP packet loss. Packet loss can cause the loss of whole KLVunits or portions thereof. 3.1. Damaged KLVunits A damaged KLVunit is any KLVunit that was carried in one or more RTP packets that have been lost. Damaged KLVunits can be recognized and flagged by looking at the sequence numbers, marker bits, and timestamps of the RTP headers. Certain patterns of lost packets in a sequence can, to a receiver, are indistinguishable between whole lost KLVunits and single damaged KLVunits. One such sequence is given as an example below. In cases such as this, where it is ambiguous if all packets carrying a KLVunit have been properly received, receivers SHOULD consider the questionable KLVunit as damaged. +---------+------+ +----------------+ +---------+------+ | RTP Hdr | Data | | | | RTP Hdr | Data | +---------+------+ +----------------+ +---------+------+ | ts = 30 | | | | | ts = 45 | | ... | M = 1 | KLV | | | | M = 1 | KLV | | seq = 5 | | | | | seq = 7 | | +---------+------+ +----------------+ +---------+------+ Last RTP pkt Lost RTP Packet RTP pkt for time 45 For time 30 (seq = 6) Lost packet may have included data for KLVunit of time 45, or may have been a whole KLVunit with 30 < ts < 45. 3.2. Treatment of Damaged KLVunits KLV data streams are built in such a way that it is possible to partially recover from errors or missing data in a stream. Exact specifics of how damaged KLVunits are handled are left to each implementation, as different implementations may have differing capabilities and robustness in their downstream KLV payload processing. Because some implementations may be particularly limited in their capacity to handle damaged KLVunits, receivers MAY drop damaged KLVunits entirely. Arbeiter & Downs Expires June 12, 2010 [Page 7] Internet-Draft RTP Format for SMPTE 336M Data December 2009 4. Recommended Procedure This section contains RECOMMENDED procedures for usage of the payload format. Based on the information in the received packets, the receiver can: o reorder KLV data received out of order o mark where KLV data is missing/damaged because of packet loss 4.1. Recommended Basic Procedure Packets are transmitted when there is valid KLV data to transmit. When new KLV data is available for transmission, it is RECOMMENDED to send it as soon as possible. 4.2. Detection of Lost KLV Packets Packet loss for KLVunit packets is detected by observing gaps in the sequence numbers of RTP packets received by the receiver. The highest RTP sequence number received may also be compared with that in RTCP reports, as an additional check for loss of the last packet before an idle period. Lost data SHOULD be dealt with as described in Section 3. The sequence number in the RTP packet header is used for detection of loss and, therefore, is not suitable for applications where it needs to be alternating with other payloads in the same RTP stream. It would be complicated and unreliable to try to detect loss of data at the edges of the shifts between KLV data and other stream contents. Therefore, application/smpte336m is RECOMMENDED to be the only payload type in the RTP stream. 4.3. Compensation for Packets Out of Order For protection against packets arriving out of order, the following procedure MAY be implemented in the receiver. If analysis of a received packet reveals a gap in the sequence, the received packet SHOULD be kept in a buffer to allow time for the missing packet(s) to arrive. If a packet with a KLVunit belonging to the gap arrives before the waiting time expires, this KLVunit is inserted into the gap and then consecutive KLVunits from the leading edge of the gap may be consumed. Any KLVunit that does not arrive before the time limit expires should be treated as lost. Arbeiter & Downs Expires June 12, 2010 [Page 8] Internet-Draft RTP Format for SMPTE 336M Data December 2009 5. Security Considerations All of the security considerations from Section 14 of [RFC3550] apply. 5.1. Confidentiality Because the intention of the described payload format is to carry KLV metadata that adds value to a motion imagery sequence, security measures in the form of encryption are of importance. The amount of data in a metadata session can vary greatly, but in general is less than the amount of motion imagery data with which it is associated. Therefore, any encryption method selected must be done with the final application in mind, as it will affect the performance. Secure Real- time Transport Protocol (SRTP) [RFC3711] may provide a suitable method for ensuring confidentiality. 5.2. Integrity It will be desirable to protect the KLV data content of an RTP stream against manipulation. SRTP [RFC3711] provides methods for providing integrity that MAY be applied. 5.3. Source Authentication There are several methods of making sure the source of the metadata is the intended one. Metadata streams are usually used in a multimedia control environment. Security measures for authentication are available and SHOULD be applied in the registration and session establishment procedures, so that the identity of the sender of the metadata stream is reliably associated with the person or device setting up the session. Once established, SRTP [RFC3711] mechanisms MAY be applied to ascertain that the source is maintained the same during the session. 6. IANA Considerations The MIME type "application/smpte336m" has been assigned by the IANA in the MIME Media Types registry to describe a stream of KLV data conforming to [SMPTE336M]. The media subtype "smpte336m" under media type "application" has been registered by the IANA in the RTP Payload Format media types registry to indicate RTP payload data in a format described by this RFC. Arbeiter & Downs Expires June 12, 2010 [Page 9] Internet-Draft RTP Format for SMPTE 336M Data December 2009 6.1. Registration of MIME media type application/smpte336m MIME media type name: application MIME subtype name: smpte336m Required parameters: None Optional parameters: None Security considerations: None Interoperability considerations: None 7. SDP Mapping The information carried in the application/smpte336m MIME media type specification has a specific mapping to fields in the Session Description Protocol (SDP) [RFC4566], which is commonly used to describe RTP sessions. When SDP is used to specify sessions employing data of type application/smpte336m, the mapping is as follows: o The "media" field in the SDP media description line ("m=") is set to the MIME type ("application"). o The "encoding name" field of the SDP "a=rtpmap" line is set to smpte336m to indicate the MIME media subtype of smpte336m. o The RTP clock rate in "a=rtpmap" SHOULD be a sub-multiple of the video clock rate for application/smpte336m. 8. Examples Arbeiter & Downs Expires June 12, 2010 [Page 10] Internet-Draft RTP Format for SMPTE 336M Data December 2009 8.1. RTP Packetization example for the application/smpte336m format Below is an example of an application/smpte336m RTP packet. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC=0 |M| KLV PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp (1000Hz) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | contributing source (CSRC) identifiers | | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | One KLVUnit of KLV encoded data | + +---------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 8.2. SDP example Below is an example SDP, which describes RTP KLV transport on port 11000 with dynamic payload type = smpte336m PT (number in the range of [96,127]); subtype = smpte336m, and clock rate = 1000: m=application 11000 RTP/AVP 100 a=rtpmap:100 smpte336m/1000 9. References 9.1. Normative References [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [RFC3880] Lennox, J., Wu, X., and H. Schulzrinne, "Call Processing Arbeiter & Downs Expires June 12, 2010 [Page 11] Internet-Draft RTP Format for SMPTE 336M Data December 2009 Language (CPL): A Language for User Control of Internet Telephony Services", RFC 3880, October 2004. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. [SMPTE336M] SMPTE, "SMPTE336M-2007: Data Encoding Protocol Using Key- Length-Value", 2007, . 9.2. Informative References [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, March 2004. Authors' Addresses J. Arbeiter (editor) Harris Corporation US Phone: Email: jarbeite@harris.com J. Downs (editor) PAR Government Systems Corp. US Phone: Email: jeff_downs@partech.com Arbeiter & Downs Expires June 12, 2010 [Page 12]