- Home
- RFC 9754
RFC 9754: Extensions for Opening and Delegating Files in NFSv4.2
- T. Haynes,
- T. Myklebust
Abstract
The Network File System v4 (NFSv4) allows a client to both open a file and be granted a delegation of that file. This delegation provides the client the right to authoritatively cache metadata on the file locally. This document presents several extensions for both opening the file and delegating it to the client. This document extends NFSv4.2 (see RFC 7863).¶
Status of This Memo
This is an Internet Standards Track document.¶
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841.¶
Information about the current status of this document, any
errata, and how to provide feedback on it may be obtained at
https://
Copyright Notice
Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://
1. Introduction
In the Network File System version 4 (NFSv4), a client may be granted a delegation for a file (see Section 1.8.4 of [RFC8881]). This allows the client to act as the authority for the file's data and metadata. This document presents a number of extensions that enhance the functionality of opens and delegations. These allow the client to:¶
- detect an offline file, which may require significant effort to obtain;¶
- determine which extensions of OPEN flags are supported by the server;¶
- retrieve either the open or delegation stateid, but not both simultaneously, during the OPEN procedure; and¶
- cache both the access and modify timestamps, thereby reducing the frequency with which the client must query the server for this information.¶
Using the process detailed in [RFC8178], the revisions in this document become an extension of NFSv4.2 [RFC7862]. They are built on top of the External Data Representation (XDR) [RFC4506] generated from [RFC7863].¶
1.1. Definitions
This document uses the following terminology:¶
- offline file:
- A file that exists on a device that is not connected to the server. There is typically a cost associated with bringing the file to an online status. Historically, this would be a file on tape media, and the cost would have been finding and loading the tape. A more modern interpretation is that the file is in the cloud, and the cost is a monetary one in downloading the file.¶
- proxy:
- The proxying of attributes occurs when a client has the authority, as granted by the appropriate delegation, to represent the attributes normally maintained by the server. For read attributes, this occurs when the client has either a read or write delegation for the file. For write attributes, this occurs when the client has a write delegation for the file. The client having this authority is the "proxy" for those attributes.¶
Further, the definitions of the following terms are referenced as follows:¶
- CB_
GETATTR (Section 20.1 of [RFC8881])¶ - change (Section 5.8.1.4 of [RFC8881])¶
- CLOSE (Section 18.2 of [RFC8881])¶
- compound (Section 2.3 of [RFC8881])¶
- DELEGRETURN (Section 18.6 of [RFC8881])¶
- GETATTR (Section 18.7 of [RFC8881])¶
- LAYOUTGET (Section 18.43 of [RFC8881])¶
- LOCK (Section 18.10 of [RFC8881])¶
- NFS4ERR_
DELAY (Section 15.1.1.3 of [RFC8881])¶ - OPEN (Section 18.16 of [RFC8881])¶
- open_
delegation_ type4 (Section 18.16.1 of [RFC8881])¶ - READ (Section 18.22 of [RFC8881])¶
- READDIR (Section 18.23 of [RFC8881])¶
- SETATTR (Section 18.30 of [RFC8881])¶
- stateid (Section 8.2 of [RFC8881])¶
- time_
access (Section 5.8.2.37 of [RFC8881])¶ - time_
metadata (Section 5.8.2.42 of [RFC8881])¶ - time_
modify (Section 5.8.2.43 of [RFC8881])¶ - WRITE (Section 18.32 of [RFC8881])¶
1.2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
2. Offline Files
If a file is offline, then the server has immediate high
A compound with a GETATTR or READDIR can report the file's attributes
without bringing the file online. However, either an OPEN or a LAYOUTGET
might cause the file server to retrieve the archived data contents, bringing the
file online. For non-parallel NFS systems (see Section 12 of [RFC8881]), the OPEN operation
requires a filehandle to retrieve the data content. For parallel NFS (pNFS) systems, the
filehandle retrieved from an OPEN need not cause the data content to be
retrieved. However, when the LAYOUTGET operation is processed, a layout
If the client is not aware that the file is offline, it might
inadvertently open the file to determine what type of file it
is accessing. By interrogating the new attribute fattr4_
2.1. XDR for the Offline Attribute
<CODE BEGINS> /// /// typedef bool fattr4_offline; /// /// /// const FATTR4_OFFLINE = 83; /// <CODE ENDS>
3. Determining OPEN Feature Support
Section 4.4.2 of [RFC8178] allows for extending a particular minor version of the NFSv4 protocol without requiring the definition of a new minor version. The client can probe the capabilities of the server and, based on the result, determine if both it and the server support optional features not previously specified as part of the minor version.¶
The fattr4_
Two new flags are provided:¶
-
OPEN4_
SHARE_ ACCESS_ WANT_ OPEN_ XOR_ DELEGATION (see Section 4)¶ -
OPEN4_
SHARE_ ACCESS_ WANT_ DELEG_ TIMESTAMPS (see Section 5)¶
Subsequent extensions can use this framework when introducing new OPTIONAL functionality to OPEN by creating a new flag for each OPTIONAL parameter.¶
Since fattr4_
Some other concerns are how to process both currently REQUIRED flags and OPTIONAL flags that become REQUIRED in the future. The server MUST mark REQUIRED flags as being supported. Note that these flags MUST only change from OPTIONAL to REQUIRED when the NFSv4 minor version is incremented.¶
3.1. XDR for Open Arguments
<CODE BEGINS>
///
/// struct open_arguments4 {
/// bitmap4 oa_share_access;
/// bitmap4 oa_share_deny;
/// bitmap4 oa_share_access_want;
/// bitmap4 oa_open_claim;
/// bitmap4 oa_create_mode;
/// };
///
///
/// enum open_args_share_access4 {
/// OPEN_ARGS_SHARE_ACCESS_READ = 1,
/// OPEN_ARGS_SHARE_ACCESS_WRITE = 2,
/// OPEN_ARGS_SHARE_ACCESS_BOTH = 3
/// };
///
///
/// enum open_args_share_deny4 {
/// OPEN_ARGS_SHARE_DENY_NONE = 0,
/// OPEN_ARGS_SHARE_DENY_READ = 1,
/// OPEN_ARGS_SHARE_DENY_WRITE = 2,
/// OPEN_ARGS_SHARE_DENY_BOTH = 3
/// };
///
///
/// enum open_args_share_access_want4 {
/// OPEN_ARGS_SHARE_ACCESS_WANT_ANY_DELEG = 3,
/// OPEN_ARGS_SHARE_ACCESS_WANT_NO_DELEG = 4,
/// OPEN_ARGS_SHARE_ACCESS_WANT_CANCEL = 5,
/// OPEN_ARGS_SHARE_ACCESS_WANT_SIGNAL_DELEG_WHEN_RESRC_AVAIL
/// = 17,
/// OPEN_ARGS_SHARE_ACCESS_WANT_PUSH_DELEG_WHEN_UNCONTENDED
/// = 18,
/// OPEN_ARGS_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS = 20,
/// OPEN_ARGS_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION = 21
/// };
///
///
/// enum open_args_open_claim4 {
/// OPEN_ARGS_OPEN_CLAIM_NULL = 0,
/// OPEN_ARGS_OPEN_CLAIM_PREVIOUS = 1,
/// OPEN_ARGS_OPEN_CLAIM_DELEGATE_CUR = 2,
/// OPEN_ARGS_OPEN_CLAIM_DELEGATE_PREV = 3,
/// OPEN_ARGS_OPEN_CLAIM_FH = 4,
/// OPEN_ARGS_OPEN_CLAIM_DELEG_CUR_FH = 5,
/// OPEN_ARGS_OPEN_CLAIM_DELEG_PREV_FH = 6
/// };
///
///
/// enum open_args_createmode4 {
/// OPEN_ARGS_CREATEMODE_UNCHECKED4 = 0,
/// OPEN_ARGS_CREATE_MODE_GUARDED = 1,
/// OPEN_ARGS_CREATEMODE_EXCLUSIVE4 = 2,
/// OPEN_ARGS_CREATE_MODE_EXCLUSIVE4_1 = 3
/// };
///
///
/// typedef open_arguments4 fattr4_open_arguments;
///
///
/// %/*
/// % * Determine what OPEN supports.
/// % */
/// const FATTR4_OPEN_ARGUMENTS = 86;
///
///
/// const OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION = 0x200000;
///
///
/// const OPEN4_RESULT_NO_OPEN_STATEID = 0x00000010;
///
<CODE ENDS>4. OPEN Grants Either an Open or a Delegation Stateid
The OPEN procedure returns an open stateid to the client to reference the state of the file. The client could also request a delegation stateid in the OPEN arguments. The file can be considered open for the client as long as the count of open and delegated stateids is greater than 0. Either type of stateid is sufficient to enable the server to treat the file as if it were open, which allows READ, WRITE, LOCK, and LAYOUTGET operations to proceed. If the client gets both an open and a delegation stateid as part of the OPEN, then it has to return them both to the server. A further consideration is that during each operation, the client can send a costly GETATTR.¶
If the client knows that the server supports the
OPEN4_
The client is already prepared to not get a delegation
stateid, even if requested. In order to not send an open
stateid, the server MUST indicate that fact with the result
flag of OPEN4_
Note that the OPEN4_
- Only an open stateid with the correct seqid.¶
- Only a delegation stateid with the open stateid now having an incorrect seqid as it needs to be upgraded.¶
- Both an open stateid (which will be upgraded) and a delegation stateid.¶
In this scenario, returning just a delegation stateid would hide
information from the client. If the client already has an open stateid,
then the server SHOULD ignore the
OPEN4_
4.1. Implementation Experience
The CLOSE operation neither explicitly nor implicitly releases any delegation stateids. This is not symmetrical with the OPEN operation, which can grant both an open and a delegation stateid. This specification could have tried to extend the CLOSE operation to release both stateids, but implementation experience shows that is more costly than the approach that has been proposed.¶
Consider a small workload of creating a file with content. This involves
three synchronous operations and one asynchronous operation with
existing implementations
- The first synchronous operation has to OPEN the file.¶
- The second synchronous operation performs the WRITE to the file.¶
- The third synchronous operation has to CLOSE the file.¶
- The asynchronous operation uses DELEGRETURN to return the delegation stateid.¶
<CODE BEGINS>
SEQ PUTFH OPEN GETFH GETATTR
SEQ PUTFH WRITE GETATTR
SEQ PUTFH CLOSE
...
SEQ PUTFH DELEGRETURN
<CODE ENDS>
With the proposed approach of setting the
OPEN_
<CODE BEGINS>
SEQ PUTFH OPEN(OPEN_ARGS_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION)
GETFH GETATTR
SEQ PUTFH WRITE GETATTR
...
SEQ PUTFH DELEGRETURN
<CODE ENDS>This approach reduces the cost of synchronous operations by 33% and the total number of operations by 25%. Contrast that with the alternative proposal of having CLOSE return both stateids, which would not reduce the number of synchronous operations.¶
5. Proxying of Times
When a client is granted a write delegation on a file, it becomes the
authority for the file contents and associated attributes. If the server
queries the client as to the state of the file via a CB_
With the addition of the new
OPEN4_
If a server informs the client via the fattr4_
When the server grants a delegation stateid, it
MUST inform the client by setting the appropriate
flag in the open_
These new attributes are invalid to be used with GETATTR, VERIFY, and NVERIFY, and they
can only be used with CB_
A key prerequisite of this approach is that the server and client are in
time synchronization with each other. Note that while the base NFSv4.2
does not require such synchronization
- before the corresponding time_
access attribute or time_ modify attribute on the file, or¶ - past the current server time.¶
When the time
presented is before the original time, then the update is ignored. When
the time presented is in the future, the server can either clamp the new
time to the current time or return NFS4ERR_
A change in the access time MUST NOT advance the change
time, also known as the time_
Note that each of the possible times (access, modify, and change) are compared to the current time. They should all be compared against the same time value for the current time (i.e., they do not retrieve a different value of the current time for each calculation).¶
If the client sets the OPEN4_
5.1. Use Case for NFSv3 Client Proxy
Consider an NFSv3 client that wants to access data on a server that only supports NFSv4.2. An implementation may introduce an NFSv3 server that functions as an NFSv4.2 client, serving as a gateway between the two otherwise incompatible systems. As NFSv3 is a stateless protocol, the state is not kept on the client, but rather on the NFSv3 server. As the NFSv3 server is already managing the state, it can proxy file delegations to avoid spurious GETATTRs. That is, as the client queries the NFSv3 server for the attributes, they can be served without the NFSv3 server sending a GETATTR to the NFSv4.2 server.¶
5.2. XDR for Proxying of Times
<CODE BEGINS>
///
/// /*
/// * attributes for the delegation times being
/// * cached and served by the "client"
/// */
/// typedef nfstime4 fattr4_time_deleg_access;
/// typedef nfstime4 fattr4_time_deleg_modify;
///
///
/// %/*
/// % * New RECOMMENDED Attribute for
/// % * delegation caching of times
/// % */
/// const FATTR4_TIME_DELEG_ACCESS = 84;
/// const FATTR4_TIME_DELEG_MODIFY = 85;
///
///
/// const OPEN4_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS = 0x100000;
///
/// enum open_delegation_type4 {
/// OPEN_DELEGATE_NONE = 0,
/// OPEN_DELEGATE_READ = 1,
/// OPEN_DELEGATE_WRITE = 2,
/// OPEN_DELEGATE_NONE_EXT = 3, /* new to v4.1 */
/// OPEN_DELEGATE_READ_ATTRS_DELEG = 4,
/// OPEN_DELEGATE_WRITE_ATTRS_DELEG = 5
/// };
<CODE ENDS>6. Extraction of XDR
This document contains the XDR [RFC4506] description of the
new open flags for delegating the file to the client. The XDR description
is embedded in this document in a way that makes it simple for the reader
to extract into a ready
<CODE BEGINS> #!/bin/sh grep '^ *///' $* | sed 's?^ */// ??' | sed 's?^ *///$??' <CODE ENDS>
That is, if the above script is stored in a file called "extract.sh" and this document is in a file called "spec.txt", then the reader can do the following:¶
<CODE BEGINS> sh extract.sh < spec.txt > delstid_prot.x <CODE ENDS>
The effect of the script is to remove leading blank space from each line, plus a sentinel sequence of "///". XDR descriptions with the sentinel sequence are embedded throughout the document.¶
Note that the XDR code contained in this document depends on types
from the NFSv4.2 nfs4_
While the XDR can be appended to that from [RFC7863], the various code snippets belong in their respective areas of that XDR.¶
7. Security Considerations
While this document extends some capabilities for client delegation, there are no new security concerns. The client cannot be queried by other clients as to the cached attributes. The client could report false data for the cached attributes, but it already has this ability via a SETATTR operation.¶
8. IANA Considerations
This document has no IANA actions.¶
9. Normative References
- [RFC2119]
-
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10
.17487 , , <https:///RFC2119 www >..rfc -editor .org /info /rfc2119 - [RFC4506]
-
Eisler, M., Ed., "XDR: External Data Representation Standard", STD 67, RFC 4506, DOI 10
.17487 , , <https:///RFC4506 www >..rfc -editor .org /info /rfc4506 - [RFC7862]
-
Haynes, T., "Network File System (NFS) Version 4 Minor Version 2 Protocol", RFC 7862, DOI 10
.17487 , , <https:///RFC7862 www >..rfc -editor .org /info /rfc7862 - [RFC7863]
-
Haynes, T., "Network File System (NFS) Version 4 Minor Version 2 External Data Representation Standard (XDR) Description", RFC 7863, DOI 10
.17487 , , <https:///RFC7863 www >..rfc -editor .org /info /rfc7863 - [RFC8174]
-
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10
.17487 , , <https:///RFC8174 www >..rfc -editor .org /info /rfc8174 - [RFC8178]
-
Noveck, D., "Rules for NFSv4 Extensions and Minor Versions", RFC 8178, DOI 10
.17487 , , <https:///RFC8178 www >..rfc -editor .org /info /rfc8178 - [RFC8881]
-
Noveck, D., Ed. and C. Lever, "Network File System (NFS) Version 4 Minor Version 1 Protocol", RFC 8881, DOI 10
.17487 , , <https:///RFC8881 www >..rfc -editor .org /info /rfc8881
Acknowledgments
Trond Myklebust, Tom Haynes, and David Flynn all worked on the prototype at Hammerspace.¶
Dave Noveck, Chuck Lever, Rick Macklem, and Zaheduzzaman Sarker provided reviews of the document.¶
Jeff Layton provided experience from an implementation he authored.¶