RFC 9584: RTP Payload Format for Essential Video Coding (EVC)

1.1.1. Coding-Tool Features (Informative)

Coding blocks and transform structure

EVC uses a traditional block-based coding structure, which divides the encoded image into blocks of up to 64x64 luma samples for the Baseline profile and 128x128 luma samples for the Main profile that can be recursively divided into smaller blocks. The Baseline profiles utilize HEVC-like quad-tree-blocks partitioning that allows a block to be divided horizontally and vertically into four smaller square blocks. The Main profile adds two advanced coding structure tools: 1) Binary Ternary Tree (BTT) partitioning that allows non-square coding units and 2) Split Unit Coding Order segmentation that changes the processing order of the blocks from traditional left-to-right and top-to-bottom scanning order processing to an alternative right-to-left and bottom-to-top scanning order. In the Main profile, the picture can be divided into slices and tiles, which can be independently encoded and/or decoded in parallel.¶

EVC also uses a traditional video codecs prediction model assuming two general types of predictions: Intra (spatial) and Inter (temporal) predictions. A residue block is calculated by subtracting predicted data from the original (encoded) one. The Baseline profile allows only discrete cosine transform (DCT-2) and scalar quantization to transform and quantize residue data, wherein the Main profile additionally has options to use discrete sine transform (DST-7) and another type of discrete cosine transform (DCT-8). In addition, for the Main profile, Improved Quantization and Transform (IQT) uses a different mapping or clipping function for quantization. An inverse zig-zag scanning order is used for coefficient coding. Advanced Coefficient Coding (ADCC) in the Main profile can code coefficient values more efficiently, for example, indicated by the last non-zero coefficient. The Baseline profile uses a straightforward RLE-based approach to encode the quantized coefficients.¶

Entropy coding

EVC uses a similar binary arithmetic coding mechanism as HEVC CABAC (context adaptive binary arithmetic coding) and VVC. The mechanism includes a binarization step and a probability update defined by a lookup table. In the Main profile, the derivation process of syntax elements based on adjacent blocks makes the context modeling and initialization process more efficient.¶

In-loop filtering

The Baseline profile of EVC uses the deblocking filter defined in H.263 Annex J [VIDEO-CODING]. In the Main profile, an Advanced Deblocking Filter (ADDB) can be used as an alternative, which can further reduce undesirable compression artifacts. The Main profile also defines two additional in-loop filters that can be used to improve the quality of decoded pictures before output and/or for Inter prediction. A Hadamard Transform Domain Filter (HTDF) is applied to the luma samples before deblocking, and a lookup table is used to determine four adjacent samples for filtering. An adaptive Loop Filter (ALF) allows signals of up to 25 different filters to be sent for the luma components; the best filter can be selected through the classification process for each 4x4 block. Similarly to VVC, the filter parameters of ALF are signaled in the Adaptation Parameter Set (APS).¶

Inter prediction

The basis of EVC's Inter prediction is motion compensation using interpolation filters with a quarter sample resolution. In the Baseline profile, a motion vector is transmitted using one of three spatially neighboring motion vectors and a temporally collocated motion vector as a predictor. A motion vector difference may be signaled relative to the selected predictor, but there is a case where no motion vector difference is signaled, and there is no remaining data in the block. This mode is called a "skip" mode. The Main profile includes six additional tools to provide improved Inter prediction. With Advanced Motion Vectors Prediction (ADMVP), adjacent blocks can be conceptually merged to indicate that they use the same motion, but more advanced schemes can also be used to create predictions from the basic model list of candidate predictors. The Merge with Motion Vector Difference (MMVD) tool uses a process similar to the concept of merging neighboring blocks but also allows the use of expressions that include a starting point, motion amplitude, and direction of motion to send a motion vector signal. Using Advanced Motion Vector Prediction (AMVP), candidate motion vector predictions for the block can be derived from its neighboring blocks in the same picture and collocated blocks in the reference picture. The Adaptive Motion Vector Resolution (AMVR) tool provides a way to reduce the accuracy of a motion vector from a quarter sample to half sample, full sample, double sample, or quad sample, which provides an efficiency advantage, such as when sending large motion vector differences. The Main profile also includes the Decoder-side Motion Vector Refinement (DMVR), which uses a bilateral template matching process to refine the motion vectors without additional signaling.¶

Intra prediction and intra coding

Intra prediction in EVC is performed on adjacent samples of coding units in a partitioned structure. For the Baseline profile, when all coding units are square, there are five different prediction modes: DC (mean value of the neighborhood), horizontal, vertical, and two different diagonal directions. In the Main profile, intra prediction can be applied to any rectangular coding unit, and 28 additional direction modes are available in the Enhanced Intra Prediction Directions (EIPDs). In the Main profile, an encoder can also use Intra Block Copy (IBC), where previously decoded sample blocks of the same picture are used as a predictor. A displacement vector in integer sample precision is signaled to indicate where the prediction block in the current picture is used for this mode.¶

Reference frames management

In EVC, decoded pictures can be stored in a decoded picture buffer (DPB) for predicting pictures that follow them in the decoding order. In the Baseline profile, the management of the DPB (i.e., the process of adding and deleting reference pictures) is controlled by a straightforward AVC-like sliding window approach with very few parameters from the sequence parameter set (SPS). For the Main profile, DPB management can be handled much more flexibly using explicitly signaled Reference Picture Lists (RPLs) in the SPS or slice level.¶

1.1.2. Systems and Transport Interfaces

EVC inherits the basic systems and transport interface designs from AVC and HEVC. These include the NAL-unit-based syntax, hierarchical syntax and data unit structure, and Supplemental Enhancement Information (SEI) message mechanism. The hierarchical syntax and data unit structure consists of a sequence-level parameter set (i.e., SPS), two picture-level parameter sets (i.e., PPS and APS, each of which can apply to one or more pictures), slice-level header parameters, and lower-level parameters.¶

A number of key components that influenced the NAL design of EVC as well as this document are described below:¶

Sequence parameter set

The Sequence Parameter Set (SPS) contains syntax elements pertaining to a Coded Video Sequence (CVS), which is a group of pictures, starting with a random access point picture and followed by zero or more pictures that may depend on each other and the random access point picture. In MPEG-2, the equivalent of a CVS is a Group of Pictures (GOP), which generally starts with an I frame and is followed by P and B frames. While more complex in its options of random access points, EVC retains this basic concept. In many TV-like applications, a CVS contains a few hundred milliseconds to a few seconds of video. In video conferencing (without switching Multipoint Control Units (MCUs) involved), a CVS can be as long in duration as the whole session.¶

Picture and adaptation parameter set

The Picture Parameter Set (PPS) and the Adaptation Parameter Set (APS) carry information pertaining to a single picture. The PPS contains information that is likely to stay constant from picture to picture, at least for pictures of a certain type; whereas the APS contains information, such as adaptive loop filter coefficients, that are likely to change from picture to picture.¶

Profile, level, and toolsets

Profiles and levels follow the same design considerations known from AVC, HEVC, and video codecs as old as MPEG-1 Video. The profile defines a set of tools (not to be confused with the "toolset" discussed below) that a decoder compliant with this profile has to support. In EVC, profiles are defined in Annex A of [EVC]. Formally, they are defined as a set of constraints that a bitstream needs to conform to. In EVC, the Baseline profile is much more severely constrained than the Main profile, reducing implementation complexity. Levels relate to bitstream complexity in dimensions such as maximum sample decoding rate, maximum picture size, and similar parameters directly related to computational complexity and/or memory demands.¶

Profiles and levels are signaled in the highest parameter set available, the SPS.¶

EVC contains another mechanism related to the use of coding tools, known as the toolset syntax elements. These syntax elements, toolset_idc_h and toolset_idc_l (located in the SPS), are bitmasks that allow encoders to indicate which coding tools they are using within the menu of profiles offered by the profile that is also signaled. No decoder conformance point is associated with the toolset, but a bitstream that was using a coding tool that is indicated as not being used in the toolset syntax element would be non-compliant. While MPEG specifically rules out the use of the toolset syntax element as a conformance point, walled garden implementations could do so without incurring the interoperability problems MPEG fears and create bitstreams and decoders that do not support one or more given tools. That, in turn, may be useful to mitigate certain intellectual property-related risks.¶

Bitstream and elementary stream

Above the Coded Video Sequence (CVS), EVC defines a video bitstream that can be used as an elementary stream in the MPEG systems context. For this document, the video bitstream syntax level is not relevant.¶

Random access support

EVC supports random access mechanisms based on IDR and clean random access (CRA) access units.¶

Temporal scalability support

EVC supports temporal scalability through the generalized reference picture selection approach known since AVC/SVC. Up to six temporal layers are supported. The temporal layer is signaled in the NAL unit header (which co-serves as the payload header in this document), in the nuh_temporal_id field.¶

Reference picture management

EVC's reference picture management is POC-based, similar to HEVC. In the Main profile, substantially all reference picture list manipulations available in HEVC are specified, including explicit transmissions or updates of reference picture lists. Although for reference pictures management purposes, EVC uses a modern VVC-like RPL approach, which is conceptually simpler than the HEVC one. In the Baseline profile, reference picture management is more restricted, allowing for a comparatively simple group of picture structures only.¶

SEI Message

EVC inherits many of HEVC's SEI messages, occasionally with syntax and/or semantics changes, making them applicable to EVC. In addition, some of the codec-agnostic SEI messages of the VSEI specification [VSEI] are also mapped.¶

1.1.3. Parallel Processing Support (Informative)

EVC's Baseline profile includes no tools specifically addressing parallel-processing support. The Main profile includes independently decodable slices for parallel processing. The slices are defined as any rectangular region within a picture. They can be encoded to have coding dependencies with other slices from the previous picture but not with other slices in the same picture. No specific support for parallel processing is specified in this RTP payload format.¶

1.1.4. NAL Unit Header

EVC maintains the NAL unit concept of [VVC] with different parameter options. EVC also uses a two-byte NAL unit header, as shown in Figure 1. The payload of a NAL unit refers to the NAL unit excluding the NAL unit header.¶

                     0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
                    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                    |F|   Type    | TID | Reserve |E|
                    +-------------+-----------------+

The semantics of the fields in the NAL unit header are as specified in EVC and described briefly below for convenience. In addition to the name and size of each field, the corresponding syntax element name in EVC is also provided.¶

F:

1 bit¶

forbidden_zero_bit:

Required to be zero in EVC. Note that the inclusion of this bit in the NAL unit header was included to enable transport of EVC video over MPEG-2 transport systems (avoidance of start code emulations) [MPEG2S]. In this document, the value 1 may be used to indicate a syntax violation, e.g., for a NAL unit resulting from aggregating a number of fragmented units of a NAL unit but missing the last fragment, as described in Section 4.3.3.¶

Type:

6 bits¶

nal_unit_type_plus1:

This field allows the NAL Unit Type to be computed. The NAL Unit Type (NalUnitType) is equal to the value found in this field, minus 1; in other words:¶

NalUnitType = nal_unit_type_plus1 - 1.¶

The NAL unit type is detailed in Table 4 of [EVC]. If the value of NalUnitType is less than or equal to 23, the NAL unit is a VCL NAL unit. Otherwise, the NAL unit is a non-VCL NAL unit. For a reference of all currently defined NAL unit types and their semantics, please refer to Section 7.4.2.2 of [EVC]. Note that nal_unit_type_plus1 MUST NOT be zero.¶

TID:

3 bits¶

nuh_temporal_id:

This field specifies the temporal identifier of the NAL unit. The value of TemporalId is equal to TID. TemporalId shall be equal to 0 if it is an IDR NAL unit type (NAL unit type 1).¶

Reserve:

5 bits¶

nuh_reserved_zero_5bits:

This field shall be equal to the version of the EVC standard. Values of nuh_reserved_zero_5bits greater than 0 are reserved for future use by ISO/IEC. Decoders conforming to a profile specified in Annex A of [EVC] shall ignore (i.e., remove from the bitstream and discard) all NAL units with values of nuh_reserved_zero_5bits greater than 0.¶

E:

1 bit¶

nuh_extension_flag:

This field shall be equal to the version of the EVC standard. The value of nuh_extension_flag equal to 1 is reserved for future use by ISO/IEC. Decoders conforming to a profile specified in Annex A of [EVC] shall ignore (i.e., remove from the bitstream and discard) all NAL units with values of nuh_extension_flag equal to 1.¶

3.1.1. Definitions from the EVC Standard

Access Unit (AU):

A set of NAL units that are associated with each other according to a specified classification rule, are consecutive in decoding order, and contain exactly one coded picture.¶

Adaptation Parameter Set (APS):

A syntax structure containing syntax elements that apply to zero or more slices as determined by zero or more syntax elements found in slice headers.¶

Bitstream:

A sequence of bits, in the form of a NAL unit stream or a byte stream, that forms the representation of coded pictures and associated data forming one or more CVSs.¶

Coded Picture:

A coded representation of a picture containing all CTUs of the picture.¶

Coded Video Sequence (CVS):

A sequence of access units that consists, in decoding order, of an IDR access unit, followed by zero or more access units that are not IDR access units, including all subsequent access units up to but not including any subsequent access unit that is an IDR access unit.¶

Coding Tree Block (CTB):

An NxN block of samples for some value of N such that the division of a component into CTBs is a partitioning.¶

Coding Tree Unit (CTU):

A CTB of luma samples, two corresponding CTBs of chroma samples of a picture that has three sample arrays, or a CTB of samples of a monochrome picture or a picture that is coded using three separate color planes and syntax structures used to code the samples.¶

Decoded Picture:

A decoded picture is derived by decoding a coded picture.¶

Decoded Picture Buffer (DPB):

A buffer holding decoded pictures for reference, output reordering, or output delay specified for the hypothetical reference decoder in Annex C of the [EVC] standard.¶

Dynamic Range Adjustment (DRA):

A mapping process that is applied to the decoded picture prior to cropping and output as part of the decoding process; it is controlled by parameters conveyed in an Adaptation Parameter Set (APS).¶

Hypothetical Reference Decoder (HRD):

A hypothetical decoder model that specifies constraints on the variability of conforming NAL unit streams or conforming byte streams that an encoding process may produce.¶

IDR Access Unit:

An access unit in which the coded picture is an IDR picture.¶

IDR Picture:

The coded picture for which each VCL NAL unit has NalUnitType equal to IDR_NUT.¶

Level:

A defined set of constraints on the values that may be taken by the syntax elements and variables of this document, or the value of a transform coefficient prior to scaling.¶

Network Abstraction Layer (NAL) Unit:

A syntax structure containing an indication of the type of data to follow and bytes containing that data in the form of an RBSP interspersed as necessary.¶

Network Abstraction Layer (NAL) Unit Stream:

A sequence of NAL units.¶

Non-IDR Picture:

A coded picture that is not an IDR picture.¶

Non-VCL NAL Unit:

A NAL unit that is not a VCL NAL unit.¶

Picture Parameter Set (PPS):

A syntax structure containing syntax elements that apply to zero or more entire coded pictures as determined by a syntax element found in each slice header.¶

Picture Order Count (POC):

A variable that is associated with each picture, uniquely identifies the associated picture among all pictures in the CVS, and (when the associated picture is to be output from the DPB) indicates the position of the associated picture in output order relative to the output order positions of the other pictures in the same CVS that are to be output from the DPB.¶

Raw Byte Sequence Payload (RBSP):

A syntax structure containing an integer number of bytes that is encapsulated in a NAL unit and that is either empty or has the form of a string of data bits containing syntax elements followed by an RBSP stop bit and zero or more subsequent bits equal to 0.¶

Sequence Parameter Set (SPS):

A syntax structure containing syntax elements that apply to zero or more entire CVSs as determined by the content of a syntax element found in the PPS referred to by a syntax element found in each slice header.¶

Slice:

An integer number of tiles of a picture in the tile scan of the picture, exclusively contained in a single NAL unit.¶

Tile:

A rectangular region of CTUs within a particular tile column and a particular tile row in a picture.¶

Tile Column:

A rectangular region of CTUs having a height equal to the height of the picture and width specified by syntax elements in the PPS.¶

Tile Row:

A rectangular region of CTUs having a height specified by syntax elements in the PPS and a width equal to the width of the picture.¶

Tile Scan:

A specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile, whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture.¶

Video Coding Layer (VCL) NAL Unit:

A collective term for coded slice NAL units and the subset of NAL units that have reserved values of NalUnitType that are classified as VCL NAL units in this document.¶

3.1.2. Definitions Specific to This Document

Media-Aware Network Element (MANE):

A network element, such as a middlebox, selective forwarding unit, or application-layer gateway, that is capable of parsing certain aspects of the RTP payload headers or the RTP payload and reacting to their contents.¶

NAL unit decoding order:

A NAL unit order that conforms to the constraints on NAL unit order given in Section 7.4.2.3 of [EVC] and follows the order of NAL units in the bitstream.¶

NALU-time:

The value that the RTP timestamp would have if the NAL unit would be transported in its own RTP packet.¶

NAL unit output order:

A NAL unit order in which NAL units of different access units are in the output order of the decoded pictures corresponding to the access units, as specified in [EVC], and in which NAL units within an access unit are in their decoding order.¶

RTP stream:

See [RFC7656]. Within the scope of this document, one RTP stream is utilized to transport an EVC bitstream, which may contain one or more temporal sub-layers.¶

Transmission order:

The order of packets in ascending RTP sequence number order (in modulo arithmetic). Within an Aggregation Packet (AP), the NAL unit transmission order is the same as the order of appearance of NAL units in the packet.¶

4.3.1. Single NAL Unit Packets

A single NAL unit packet contains exactly one NAL unit and consists of a payload header as defined in Table 4 of [EVC] (denoted as PayloadHdr), followed by a conditional 16-bit DONL field (in network byte order), and the NAL unit payload data (the NAL unit excluding its NAL unit header) of the contained NAL unit, as shown in Figure 3.¶

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           PayloadHdr          |      DONL (conditional)       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  |                  NAL unit payload data                        |
  |                                                               |
  |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               :...OPTIONAL RTP padding        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The DONL field, when present, specifies the value of the 16 least significant bits of the decoding order number of the contained NAL unit. If sprop-max-don-diff (defined in Section 7.2) is greater than 0, the DONL field MUST be present, and the variable DON for the contained NAL unit is derived as equal to the value of the DONL field. Otherwise (where sprop-max-don-diff is equal to 0), the DONL field MUST NOT be present.¶

4.3.2. Aggregation Packets (APs)

Aggregation Packets (APs) enable the reduction of packetization overhead for small NAL units, such as most of the non-VCL NAL units, which are often only a few octets in size.¶

An AP aggregates NAL units of one access unit, and it MUST NOT contain NAL units from more than one AU. Each NAL unit to be carried in an AP is encapsulated in an aggregation unit. NAL units aggregated in one AP are included in NAL-unit-decoding order.¶

An AP consists of a payload header, as defined in Table 4 of [EVC] (denoted here as PayloadHdr with Type=56), followed by two or more aggregation units, as shown in Figure 4.¶

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |    PayloadHdr (Type=56)       |                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
 |                                                               |
 |             two or more aggregation units                     |
 |                                                               |
 |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                               :...OPTIONAL RTP padding        |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The fields in the payload header of an AP are set as follows. The F bit MUST be equal to 0 if the F bit of each aggregated NAL unit is equal to zero; otherwise, it MUST be equal to 1. The Type field MUST be equal to 56.¶

The value of TID MUST be the smallest value of TID of all the aggregated NAL units. The value of Reserve and E MUST be equal to 0 for this specification.¶

An AP MUST carry at least two aggregation units and can carry as many aggregation units as necessary; however, the total amount of data in an AP obviously MUST fit into an IP packet, and the size SHOULD be chosen so that the resulting IP packet is smaller than the path MTU size so to avoid IP layer fragmentation. An AP MUST NOT contain FUs specified in Section 4.3.3. APs MUST NOT be nested; i.e., an AP cannot contain another AP.¶

The first aggregation unit in an AP consists of a conditional 16-bit DONL field (in network byte order) followed by a 16-bit unsigned size information (in network byte order) that indicates the size of the NAL unit in bytes (excluding these two octets but including the NAL unit header), followed by the NAL unit itself, including its NAL unit header, as shown in Figure 5.¶

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |               :       DONL (conditional)      |   NALU size   |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   NALU size   |                                               |
 +-+-+-+-+-+-+-+-+         NAL unit                              |
 |                                                               |
 |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                               :
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The DONL field, when present, specifies the value of the 16 least significant bits of the decoding order number of the aggregated NAL unit.¶

If sprop-max-don-diff is greater than 0, the DONL field MUST be present in an aggregation unit that is the first aggregation unit in an AP. The variable DON for the aggregated NAL unit is derived as equal to the value of the DONL field, and the variable Decoding Order Number (DON) for an aggregation unit that is not the first aggregation unit in an AP-aggregated NAL unit is derived as equal to the DON of the preceding aggregated NAL unit in the same AP plus 1 modulo 65536. Otherwise (where sprop-max-don-diff is equal to 0), the DONL field MUST NOT be present in an aggregation unit that is the first aggregation unit in an AP.¶

An aggregation unit that is not the first aggregation unit in an AP will be followed immediately by a 16-bit unsigned size information (in network byte order) that indicates the size of the NAL unit in bytes (excluding these two octets but including the NAL unit header), followed by the NAL unit itself, including its NAL unit header, as shown in Figure 6.¶

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |               :       NALU size               |   NAL unit    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
  |                                                               |
  |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               :
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 7 presents an example of an AP that contains two aggregation units, labeled "NALU 1" and "NALU 2", without the DONL field being present.¶

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                          RTP Header                           |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   PayloadHdr (Type=56)        |         NALU 1 Size           |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |          NALU 1 HDR           |                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+         NALU 1 Data           |
 |                   . . .                                       |
 |                                                               |
 +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |  . . .        | NALU 2 Size                   | NALU 2 HDR    |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | NALU 2 HDR    |                                               |
 +-+-+-+-+-+-+-+-+              NALU 2 Data                      |
 |                   . . .                                       |
 |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                               :...OPTIONAL RTP padding        |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 8 presents an example of an AP that contains two aggregation units, labeled "NALU 1" and "NALU 2", with the DONL field being present.¶

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                          RTP Header                           |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   PayloadHdr (Type=56)        |        NALU 1 DONL            |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |          NALU 1 Size          |            NALU 1 HDR         |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                                               |
 |                 NALU 1 Data   . . .                           |
 |                                                               |
 +        . . .                  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                               :          NALU 2 Size          |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |          NALU 2 HDR           |                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+          NALU 2 Data          |
 |                                                               |
 |        . . .                  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                               :...OPTIONAL RTP padding        |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

4.3.3. Fragmentation Units (FUs)

FUs are introduced to enable fragmenting a single NAL unit into multiple RTP packets, possibly without cooperation or knowledge of the EVC encoder. A fragment of a NAL unit consists of an integer number of consecutive octets of that NAL unit. Fragments of the same NAL unit MUST be sent in consecutive order with ascending RTP sequence numbers (with no other RTP packets within the same RTP stream being sent between the first and last fragment).¶

When a NAL unit is fragmented and conveyed within FUs, it is referred to as a fragmented NAL unit. APs MUST NOT be fragmented. FUs MUST NOT be nested; i.e., an FU must not contain a subset of another FU.¶

The RTP timestamp of an RTP packet carrying an FU is set to the NALU-time of the fragmented NAL unit.¶

An FU consists of a payload header as defined in Table 4 of [EVC] (denoted as PayloadHdr with Type=57), an FU header of one octet, a conditional 16-bit DONL field (in network byte order), and an FU payload, as shown in Figure 9.¶

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |    PayloadHdr (Type=57)       |   FU header   | DONL (cond)   |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
 | DONL (cond)   |                                               |
 |-+-+-+-+-+-+-+-+                                               |
 |                         FU payload                            |
 |                                                               |
 |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                               :...OPTIONAL RTP padding        |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The fields in the payload header are set as follows. The Type field MUST be equal to 57. The fields F, TID, Reserve, and E MUST be equal to the fields F, TID, Reserve, and E, respectively, of the fragmented NAL unit.¶

The FU header consists of an S bit, an E bit, and a 6-bit FuType field, as shown in Figure 10.¶

                           0 1 2 3 4 5 6 7
                          +-+-+-+-+-+-+-+-+
                          |S|E|  FuType   |
                          +---------------+

The semantics of the FU header fields are as follows:¶

S:

1 bit¶

When set to 1, the S bit indicates the start of a fragmented NAL unit, i.e., the first byte of the FU payload is also the first byte of the payload of the fragmented NAL unit. When the FU payload is not the start of the fragmented NAL unit payload, the S bit MUST be set to 0.¶

E:

1 bit¶

When set to 1, the E bit indicates the end of a fragmented NAL unit, i.e., the last byte of the payload is also the last byte of the fragmented NAL unit. When the FU payload is not the last fragment of a fragmented NAL unit, the E bit MUST be set to 0.¶

FuType:

6 bits¶

The field FuType MUST be equal to the field Type of the fragmented NAL unit.¶

The DONL field, when present, specifies the value of the 16 least significant bits of the decoding order number of the fragmented NAL unit.¶

If sprop-max-don-diff is greater than 0 and the S bit is equal to 1, the DONL field MUST be present in the FU, and the variable DON for the fragmented NAL unit is derived as equal to the value of the DONL field. Otherwise (where sprop-max-don-diff is equal to 0, or where the S bit is equal to 0), the DONL field MUST NOT be present in the FU.¶

A non-fragmented NAL unit MUST NOT be transmitted in one FU; i.e., the S-bit and E-bit MUST NOT both be set to 1 in the same FU header.¶

The FU payload consists of fragments of the payload of the fragmented NAL unit so that if the FU payloads of consecutive FUs, starting with an FU with the S bit equal to 1 and ending with an FU with the E bit equal to 1, are sequentially concatenated, the payload of the fragmented NAL unit can be reconstructed. The NAL unit header of the fragmented NAL unit is not included as such in the FU payload. Instead, the information of the NAL unit header of the fragmented NAL unit is conveyed in F, TID, Reserve, and E fields of the FU payload headers of the FUs and the FuType field of the FU header of the FUs. An FU payload MUST NOT be empty.¶

If an FU is lost, the receiver SHOULD discard all following fragmentation units in transmission order corresponding to the same fragmented NAL unit unless the decoder in the receiver is known to gracefully handle incomplete NAL units.¶

A receiver in an endpoint or a MANE MAY aggregate the first n-1 fragments of a NAL unit to an (incomplete) NAL unit, even if fragment n of that NAL unit is not received. In this case, the forbidden_zero_bit of the NAL unit MUST be set to 1 to indicate a syntax violation.¶

7.3.1. Mapping of Payload Type Parameters to SDP

The media type video/evc string is mapped to fields in the Session Description Protocol (SDP) [RFC8866] as follows:¶

A general usage of media representation in SDP is as follows:¶

m=video 49170 RTP/AVP 98
a=rtpmap:98 evc/90000
a=fmtp:98 profile-id=1;
  sprop-sps=<sequence parameter set data>;
  sprop-pps=<picture parameter set data>;

A SIP Offer/Answer exchange wherein both parties are expected to both send and receive could look like the following. Only the media codec-specific parts of the SDP are shown.¶

Offerer->Answerer:
      m=video 49170 RTP/AVP 98
      a=rtpmap:98 evc/90000
      a=fmtp:98 profile-id=1; level_id=90;

The above represents an offer for symmetric video communication using [EVC] and its payload specification at the main profile and level 3. Informally speaking, this offer tells the receiver of the offer that the sender is willing to receive up to xKpxx resolution at the maximum bitrates specified in [EVC]. At the same time, if this offer were accepted "as is", the offer can expect that the Answerer would be able to receive and properly decode EVC media up to and including level 3.¶

Answerer->Offerer:
      m=video 49170 RTP/AVP 98
      a=rtpmap:98 evc/90000
      a=fmtp:98 profile-id=1; level_id=60

With this answer to the offer above, the system receiving the offer advises the Offerer that it is incapable of handling evc at level 3 but is capable of decoding level 2. As EVC video codecs must support decoding at all levels below the maximum level they implement, the resulting user experience would likely be that both systems send video at level 2. However, nothing prevents an encoder from further downgrading its sending to, for example, level 1 if it were short of cycles or bandwidth or for other reasons.¶

7.3.2. Usage with SDP Offer/Answer Model

This section describes the negotiation of unicast messages using the Offer/Answer model described in [RFC3264] and its updates.¶

This section applies to all profiles defined in [EVC], specifically to Baseline, Main, and the associated still image profiles.¶

The following limitations and rules pertaining to the media configuration apply:¶

The parameters identifying a media format configuration for EVC are profile-id and level-id. Profile_id MUST be used symmetrically.¶

The Answerer MUST structure its answer according to one of the following two options:¶

To simplify handling and matching of these configurations, the same RTP payload type number used in the offer SHOULD also be used in the answer, as specified in [RFC3264].¶

The answer MUST NOT contain a payload type number used in the offer for the media subtype unless the configuration is the same as in the offer or the configuration in the answer only differs from that in the offer with a different value of level-id.¶

                                 sendonly --+
                              recvonly --+  |
                           sendrecv --+  |  |
                                      |  |  |
   profile-id                         C  C  P
   level-id                           D  D  P
   toolset-id                         C  C  P
   max-recv-level-id                  R  R  -
   sprop-max-don-diff                 P  -  P
   sprop-depack-buf-bytes             P  -  P
   depack-buf-cap                     R  R  -
   sprop-sei                          P  -  P
   sprop-sps                          P  -  P
   sprop-pps                          P  -  P


Legend:

 C: configuration for sending and receiving bitstreams
 D: changeable configuration; same as C, except possible to
    answer with a different but consistent value (see the semantics
    of the level-id parameter on these parameters being
    consistent -- basically, level down-grading is allowed)

 P: properties of the bitstream to be sent
 R: receiver capabilities
 -: not usable; when present MUST be ignored

7.3.3. Multicast

For bitstreams being delivered over multicast, the following rules apply:¶

7.3.4. Usage in Declarative Session Descriptions

When EVC over RTP is offered with SDP in a declarative style, as in the Real-Time Streaming Protocol (RTSP) [RFC7826] or Session Announcement Protocol (SAP) [RFC2974], the following considerations apply.¶

7.3.5. Considerations for Parameter Sets

When out-of-band transport of parameter sets is used, parameter sets MAY still be additionally transported in-band unless explicitly disallowed by an application, and some of these additional parameter sets may update some of the out-of-band transported parameter sets. An update of a parameter set refers to the sending of a parameter set of the same type using the same parameter set ID but with different values for at least one other parameter of the parameter set.¶