RFC 9772: Active Operations, Administration, and Maintenance (OAM) for Use in Generic Network Virtualization Encapsulation (Geneve)

2.1. Requirements for Active OAM Protocols in Geneve Networks

OAM protocols, whether part of fault management or performance monitoring, are intended to provide reliable information that can be used to detect a failure, identify the defect, and localize it, thus helping to identify and apply corrective actions to minimize the negative impact on service. Several OAM protocols are used to perform these functions; these protocols require demultiplexing at the receiving instance of Geneve. To improve the accuracy of the correlation between the condition experienced by the monitored Geneve tunnel and the state of the OAM protocol, the OAM encapsulation is required to comply with the following requirements:¶

Requirement 1:

Geneve OAM test packets MUST share the same fate as the data traffic of the monitored Geneve tunnel. Specifically, the OAM test packets MUST be in-band with the monitored traffic and follow the same overlay and transport paths as packets carrying data payloads in the forward direction, i.e., from the ingress toward the egress endpoint(s) of the OAM test.¶

An OAM protocol MAY be employed to monitor an entire Geneve tunnel. In this case, test packets could be in-band relative to a subset of tenant flows transported over the Geneve tunnel. If the goal is to monitor the conditions experienced by the flow of a particular tenant, the test packets MUST be in-band with that specific flow within the Geneve tunnel. Both scenarios are discussed in detail in Section 2.2.¶

Requirement 2:

The encapsulation of OAM control messages and data packets in the underlay network MUST be indistinguishable.¶

Requirement 3:

The presence of an OAM control message in a Geneve packet MUST be unambiguously identifiable to Geneve functionality, such as at endpoints of Geneve tunnels.¶

Requirement 4:

OAM test packets MUST NOT be forwarded to a tenant system.¶

A test packet generated by an active OAM protocol, whether for defect detection or performance measurement, MUST be in-band with the tunnel or data flow being monitored, as specified in Requirement 1. In environments where multiple paths through the domain are available, underlay transport nodes can be programmed to use characteristic information to balance the load across known paths. It is essential that test packets follow the same route -- that is, traverse the same set of nodes and links as a data packet of the monitored flow. Therefore, the following requirement supports OAM packet fate-sharing with the data flow.¶

Requirement 5:

There MUST be a way to encode entropy information into the underlay forwarding scheme so that OAM packets take the same data-flow paths as the transit traffic flows.¶

2.2. Defect Detection and Troubleshooting in Geneve Network with Active OAM

This section considers two scenarios where active OAM is used to detect and localize defects in a Geneve network. Figure 1 presents an example of a Geneve domain.¶

+--------+                                             +--------+
| Tenant +--+                                     +----| Tenant |
| VNI 28 |  |                                     |    | VNI 35 |
+--------+  |          ................           |    +--------+
            |  +----+  .              .  +----+   |
            |  | NVE|--.              .--| NVE|   |
            +--| A  |  .              .  | B  |---+
               +----+  .              .  +----+
               /       .              .
              /        .     Geneve   .
+--------+   /         .    Network   .
| Tenant +--+          .              .
| VNI 35 |             .              .
+--------+             ................
                              |
                            +----+
                            | NVE|
                            | C  |
                            +----+
                              |
                              |
                    =====================
                      |               |
                  +--------+      +--------+
                  | Tenant |      | Tenant |
                  | VNI 28 |      | VNI 35 |
                  +--------+      +--------+

In the first case, consider when a communication problem between Network Virtualization Edge (NVE) device A and NVE C exists. Upon investigation, the operator discovers that the forwarding in the IP underlay network is working accordingly. Still, the Geneve connection is unstable for all NVE A and NVE C tenants. Detection, troubleshooting, and localization of the problem can be done regardless of the VNI value.¶

In the second case, traffic on VNI 35 between NVE A and NVE B has no problems, as on VNI 28 between NVE A and NVE C. However, traffic on VNI 35 between NVE A and NVE C experiences problems, for example, excessive packet loss.¶

The first case can be detected and investigated using any VNI value, whether it connects tenant systems or not; however, to conform to Requirement 4, OAM test packets SHOULD be transmitted on a VNI that doesn't have any tenants. Such a Geneve tunnel is dedicated to carrying only control and management data between the tunnel endpoints, so it is referred to as a "Geneve control channel" and that VNI is referred to as the "Management VNI". A configured VNI MAY be used to identify the control channel, but it is RECOMMENDED that the default value 1 be used as the Management VNI. Encapsulation of test packets using the Management VNI is discussed in Section 2.3.¶

The control channel of a Geneve tunnel MUST NOT carry tenant data. As no tenants are connected using the control channel, a system that supports this specification MUST NOT forward a packet received over the control channel to any tenant. A packet received by the system that supports this specification over the control channel MUST be forwarded if and only if it is sent onto the control channel of the concatenated Geneve tunnel. Else, it MUST be terminated locally. The Management VNI SHOULD be terminated on the tenant-facing side of the Geneve encapsulation/decapsulation functionality, not the DC-network-facing side (per definitions in Section 4 of [RFC8014]), so that Geneve encap/decap functionality is included in its scope. This approach causes an active OAM packet, e.g., an ICMP echo request, to be decapsulated in the same fashion as any other received Geneve packet. In this example, the resulting ICMP packet is handed to NVE's local management functionality for the processing which generates an ICMP echo reply. The ICMP echo reply is encapsulated in Geneve (as specified in Section 2.3) for forwarding it back to the NVE that sent the echo request. One advantage of this approach is that a repeated ICMP echo request/reply test could detect an intermittent problem in Geneve encap/decap hardware, which would not be tested if the Management VNI were handled as a "special case" at the DC-network-facing interface.¶

The second case is when a test packet is transmitted using the VNI value associated with the monitored service flow. By doing that, the test packet experiences network treatment as the tenant's packets. An example of the realization of that scenario is discussed in [RFC9521].¶

2.3. Active OAM Encapsulation in Geneve

Active OAM over a Management VNI in the Geneve network uses an IP encapsulation. Protocols such as BFD [RFC5880] and STAMP [RFC8762] use UDP transport. The destination UDP port number in the inner UDP header (Figure 2) identifies the OAM protocol. This approach is well-known and has been used, for example, in MPLS networks [RFC8029]. To use IP encapsulation for an active OAM protocol, the Protocol Type field of the Geneve header MUST be set to 0x0800 (IPv4) or 0x86DD (IPv6). [RFC9521] describes the use of IP encapsulation for BFD.¶

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                        Outer IPvX Header                      ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                        Outer UDP Header                       ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                          Geneve Header                        ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                        Inner IPvX Header                      ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                        Inner UDP Header                       ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                        Active OAM Packet                      ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Inner IP header:

Destination IP:

The IP address MUST be set to the loopback address 127.0.0.1/32 for IPv4 version. For IPv6, the address MUST be selected from the Dummy IPv6 Prefix 100:0:0:1::/64 [RFC9780]. A source-only IPv6 address is used as the destination to generate an exception and a reply message to the request message received.¶

Source IP:

IP address of the NVE.¶

TTL or Hop Limit:

MUST be set to 255 per [RFC5082]. The receiver of an active OAM Geneve packet with IP/UDP encapsulation MUST drop packets whose TTL/Hop Limit is not 255.¶