Internet-Draft Uncacheable Dirent Metadata August 2026
Haynes Expires 25 February 2027 [Page]
Workgroup:
Network File System Version 4
Internet-Draft:
draft-ietf-nfsv4-uncacheable-directories-10
Published:
Intended Status:
Standards Track
Expires:
Author:
T. Haynes
Hammerspace

Adding an Uncacheable Dirent Metadata Attribute to NFSv4.2

Abstract

Network File System version 4.2 (NFSv4.2) clients may cache the file attributes returned by READDIR alongside each directory entry. This caching is inherently best-effort: those attributes belong to the underlying files and change when the files are written, which the directory's change attribute does not track. In some deployments the rate of file writes by other clients makes such caching produce incorrect size and timestamp values often enough to be a deployment problem. This document introduces an uncacheable dirent metadata attribute for NFSv4.2 that allows a server to identify a directory for which an honoring client is required to retrieve dirent metadata from the server on each READDIR rather than serving the response from a local cache.

Note to Readers

Note to RFC Editor: please remove this section prior to publication.

Discussion of this draft takes place on the NFSv4 working group mailing list (nfsv4@ietf.org), which is archived at https://mailarchive.ietf.org/arch/search/?email_list=nfsv4. Source code and issues list for this draft can be found at https://github.com/ietf-wg-nfsv4/uncacheable-directories.

Working Group information can be found at https://github.com/ietf-wg-nfsv4.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 25 February 2027.

Table of Contents

1. Introduction

Clients of remote filesystems may cache the file attributes returned by READDIR alongside each directory entry, to reduce the volume of follow-on GETATTR traffic for entries the client has already seen. This caching is inherently best-effort -- writes to the underlying files can change those attributes at any time, and the directory's change attribute does not track such writes. In some deployments the cost of that staleness is high enough to be a deployment problem; the conditions are described in Section 2.

In this document, the term directory is used to describe the context in which directory entries are retrieved. The uncacheable dirent metadata attribute applies to the caching of dirent metadata -- the file object attributes, such as size and timestamps, returned alongside each entry. It does not prohibit caching of the directory object itself, nor does it affect caching of file data.

When this best-effort caching returns stale size and timestamp information for concurrently modified files, it also undermines the effectiveness of uncacheable file data semantics ([I-D.ietf-nfsv4-uncacheable-files]) in the same deployment: applications can observe inconsistent metadata and data views even when file data caching is disabled.

This document introduces the uncacheable dirent metadata attribute to NFSv4.2 to allow servers to advise clients that caching of dirent metadata is unsuitable. Using the process detailed in [RFC8178], the revisions in this document become an extension of NFSv4.2 [RFC7862]. They are built on top of the external data representation (XDR) [RFC4506] generated from [RFC7863].

2. Deployment Motivation

A class of deployment uses NFSv4.2 to serve a shared directory to many concurrent NFSv4.2 client writers, each writing files within the directory. Workloads of this kind are typical of High-Performance Computing (HPC) environments, where a single output directory may receive results from hundreds or thousands of compute nodes simultaneously, and of large-scale data-ingest pipelines where many producers append to a common landing directory. The files within such a directory have their attributes -- size and timestamps in particular -- modified at a high rate by clients other than the one performing READDIR.

[RFC8881] Section 10.6 permits a client to cache the file attributes returned by READDIR on the same basis as attributes obtained by GETATTR: cached per file, bounded by an upper time boundary, and revalidated against that file's change attribute. In a directory receiving writes from thousands of compute nodes, any nonzero cache lifetime yields stale size and time_modify for most entries most of the time, and revalidating each entry individually costs one GETATTR per entry -- the very traffic that requesting attributes in READDIR exists to avoid. NFSv4.2 gives a server no in-band way to tell a client that the acceptable attribute cache lifetime for the children of a particular directory is zero; mount options express this out of band and per client, not per directory.

The staleness has correctness consequences, not merely cosmetic ones. An incremental backup or an rsync scan that decides what to copy from the size and time_modify reported for each entry will silently skip a file whose cached metadata predates a concurrent write, leaving data uncopied. This attribute lets a server mark the directories where that outcome is likely, so that an honoring client fetches current metadata on each enumeration.

The fattr4_uncacheable_dirent_metadata attribute is the server's mechanism to identify a directory for which this risk is high enough that client-side caching is not safe. When the server sets the attribute on a directory, an honoring client retrieves dirent metadata from the server on each READDIR rather than from a local cache.

3. Definitions

dirent

A directory entry -- the (name, fileid) pair that names a file or subdirectory within a directory. This is what a client maintains for an entry, whatever a given READDIR response carries on the wire; it is the pair POSIX exposes as d_name and d_ino. A dirent itself does not include the file attributes returned alongside it.

dirent metadata

The file attributes (size, mtime, ctime, atime, mode, owner, etc.) returned in a READDIR response alongside each dirent. These attributes belong to the underlying file object, not to the directory; they change when the underlying file is written, which is independent of the directory's change attribute. The term "dirent metadata" in this document is a naming convenience for "the file attributes that arrive in a READDIR response"; it does not assert that those attributes inherit the directory's cache-coherence semantics.

dirent caching

A client-side cache of the dirents themselves -- the (name, fileid) pairs -- used to avoid repeated READDIR traffic. Whether such a cache remains valid is governed by the directory's change attribute: the directory changes when an entry is created, removed, or renamed, and a fileid is stable for as long as its entry names the same object. A fileid is therefore cached with the name rather than with the file attributes: writes to a file change its size and timestamps without touching either the name or the fileid. Nothing in this document constrains dirent caching.

dirent metadata caching

A client-side cache of the dirent metadata returned alongside those entries, used to avoid repeated GETATTR traffic. Because those file attributes are not invalidated by the directory's change attribute (only by writes to the underlying files), this caching is inherently best-effort and subject to staleness whenever the underlying files are modified. This is the caching that the attribute defined in this document constrains.

uncacheable dirent metadata attribute

An NFSv4.2 file attribute that advises clients not to cache dirent metadata associated with file objects, such as size and timestamps.

honoring client

A client that implements this attribute and enforces the always-refetch behavior it defines for a directory on which the attribute is set. The attribute is advisory: a client that does not implement it, or that declines to enforce it, is non-honoring and may continue to cache dirent metadata.

This document assumes familiarity with NFSv4.2 operations, attributes, and error handling as defined in [RFC8881] and [RFC7862].

4. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

5. Caching of Dirent Metadata

The uncacheable dirent metadata attribute constrains what a READDIR on a particular directory does: it directs an honoring client to fetch the entries' file attributes from the server rather than serve them from a local cache. It constrains dirent metadata caching only: an honoring client may continue to cache the dirents themselves, validated by the directory's change attribute as it would be for any other directory, and refetch only their metadata. This matches how clients are typically built, with the dirent cache maintained separately from the attributes obtained for the objects the entries name. A server sets it on the directories where it knows the staleness of cached READDIR attributes is particularly likely and particularly damaging. It is a RECOMMENDED attribute for NFSv4.2, in the attribute-category sense of [RFC8881] Section 5.2 and [RFC7862] Section 12 rather than the BCP 14 sense; a server is not required to support it. If both the client and the server support this attribute, and the attribute is set on a directory, the client MUST retrieve dirent metadata from the server on each READDIR rather than serving the response from a local cache.

Because the attribute governs a READDIR rather than the objects the entries name, it makes no claim about those objects. A file reached through a directory on which the attribute is not set is unaffected, including where the same file is linked into both a directory on which it is set and one on which it is not.

This document specifies the required observable behavior rather than mandating a particular internal implementation strategy. Clients MAY employ more sophisticated mechanisms, such as time-limited caches that revalidate against the server on each READDIR, provided that the externally visible behavior is equivalent to retrieving dirent metadata from the server on each READDIR.

Allowing clients to set this attribute provides a portable mechanism to request that dirent metadata not be cached, without requiring changes to application behavior or out-of-band administrative configuration.

A client can determine whether the uncacheable dirent metadata attribute is supported for a given directory by examining the supported_attrs attribute for that directory's filesystem or by probing support using the procedures described in [RFC8178].

The uncacheable dirent metadata attribute governs the client's caching of READDIR responses for the directory. It does NOT govern:

The uncacheable dirent metadata attribute addresses a different aspect of client-side caching than fattr4_uncacheable_file_data ([I-D.ietf-nfsv4-uncacheable-files]). The file data attribute governs caching of file contents, while the dirent metadata attribute governs caching of file attributes returned by READDIR. The attributes are independent and may be used separately.

This attribute follows the same pattern as fattr4_uncacheable_file_data ([I-D.ietf-nfsv4-uncacheable-files]) applied at the file-data layer. In both cases:

The attribute does NOT make dirent metadata caching reliable for directories where it is not set. Clients MUST NOT interpret the absence of fattr4_uncacheable_dirent_metadata, or its value being false, as a guarantee that cached READDIR attributes are authoritative. As stated in [RFC8881] Section 10.6, all client-cached attributes are subject to staleness; the attribute defined in this document only identifies directories for which staleness is particularly likely and particularly damaging. The base specification separates the two concerns this attribute is often accused of conflating: [RFC8881] Section 10.8.2 governs caching of the directory entries themselves, while Section 10.6 governs caching of the file attributes that arrive alongside them, and this attribute affects only the latter.

This attribute does not define behavior for positive or negative name caching or for caching of LOOKUP results outside the scope of file attributes returned by READDIR.

A directory delegation ([RFC8881] Section 10.9) lets a client cache a directory's entries and the directory's own attributes until the server recalls the delegation. It is not recalled when the attributes of an entry within the directory change ([RFC8881] Sections 10.9.2 and 10.9.4), so a directory delegation does not, by itself, keep the file attributes returned by READDIR fresh. NOTIFY4_CHANGE_CHILD_ATTRS, requested through GET_DIR_DELEGATION, can deliver changed child attributes to a delegated client, but it is not a substitute for this attribute in the deployments of Section 2: GET_DIR_DELEGATION is OPTIONAL and is not implemented by the clients and servers those deployments use; notification cost scales with the number of delegated clients times the number of changes, which a directory written by thousands of clients makes prohibitive (and [RFC8881] Section 10.9.4 permits a server that finds a directory is causing too many notifications to decline to delegate it); and the dirent_notif_delay attribute lets a server bound or refuse child-attribute notification, so a client cannot rely on notification for freshness.

5.1. Uncacheable Dirent Metadata

The fattr4_uncacheable_dirent_metadata attribute is a read-write boolean attribute that applies to directory objects. Authorization to query or modify this attribute is governed by existing NFSv4.2 authorization mechanisms.

Because the attribute applies only to directory objects, a server that receives a GETATTR requesting fattr4_uncacheable_dirent_metadata on an object that is not a directory MUST NOT return the attribute value and MUST NOT set the attribute bit in the result bitmap, as specified for unsupported attributes in [RFC8881] Section 18.7.3. A server that receives a SETATTR requesting fattr4_uncacheable_dirent_metadata on an object that is not a directory MUST return NFS4ERR_ATTRNOTSUPP.

This attribute is set per directory. This document does not define propagation of the attribute to subdirectories created within a directory on which it is set; any such inheritance is a matter of local server policy.

If a directory object has the uncacheable dirent metadata attribute set, the client MUST retrieve dirent metadata from the server on each readdir rather than serving the response from a local cache; that is, each application-level directory read is satisfied by issuing READDIR to the server rather than from cached results. This ensures that the returned metadata reflects the current state of the directory as determined by the server. For such a directory, a client MUST NOT assume that dirent metadata is valid beyond the readdir that produced it. Entries and metadata retrieved during a single enumeration MAY be retained until that enumeration completes, consistent with the snapshot requirement of [RFC8881] Section 10.8.2.

The uncacheable dirent metadata attribute does not modify the semantics of the NFSv4.2 change attribute. Clients MUST continue to use the change attribute to detect directory modifications and to determine when directory contents may have changed, even when dirent metadata caching is suppressed. Suppressing caching of dirent metadata does not remove the need for change-based validation.

This attribute is advisory, so servers SHOULD NOT rely on it for correctness: a client that does not implement it, or that declines to enforce it, may continue to cache dirent metadata. A server cannot distinguish those clients from honoring ones. Observing a GETATTR or a SETATTR of the attribute shows only that a client knows the attribute exists, not that it enforces the always-refetch rule, so such a request is not a basis for treating that client's caching as suppressed.

A directory delegation would let a client serve dirent metadata from its cache without refetching, which is incompatible with the always-refetch rule this attribute defines. Accordingly, if a directory has the uncacheable dirent metadata attribute set and an outstanding directory delegation, the server MUST recall the delegation, after which the client follows the always-refetch rule on each subsequent readdir. A server MUST NOT grant a new directory delegation on a directory while the uncacheable dirent metadata attribute is set on that directory.

6. Example: Directory Enumeration With and Without Dirent Metadata Caching

This example illustrates the difference in client-visible behavior when dirent metadata caching is enabled versus when the uncacheable dirent metadata attribute is set on a directory. In both scenarios, the set of entries in the directory does not change between the two calls; an attribute value of one entry is updated at the server between calls. The difference is whether the second call observes the updated attribute value.

6.1. Classic Directory Enumeration (Dirent Metadata Cached)

In this scenario, the client caches dirent metadata obtained from the server and reuses it for the second readdir.

Application             NFSv4.2 Client        NFSv4.2 Server
-----------             --------------        --------------
readdir("/dir")
   |
   |                     READDIR, size and
   |                     time_modify requested
   |-------------------->------------------------>
   |                     entries:
   |                       a (size=100)
   |                       b (size=200)
   |                       c (size=300)
   |<--------------------<------------------------
   |<-- names a, b, c
                        (attributes retained per
                         entry, bounded by the
                         attribute cache lifetime)

stat("/dir/a")
   |                     (served from the retained
   |                      READDIR attributes)
   |<-- size=100

                                            (concurrent writer extends
                                             a from size=100 to
                                             size=500)

stat("/dir/a")
   |                     (no network traffic; still
   |                      within the cache lifetime)
   |<-- size=100
Figure 1: Dirent Metadata Cached

In this case, Figure 1 shows that the attributes retrieved by the READDIR are retained and reused to satisfy a later stat of entry a. readdir itself yields only names; the size the application observes comes from the retained READDIR attributes. The second stat reflects entry a's size as it was at the time of the READDIR, not the update that occurred at the server between the two calls. This behavior maximizes performance and is what [RFC8881] Section 10.6 permits, but for the duration of the cache lifetime it can result in applications observing dirent attribute values that do not reflect the current state of the server.

6.2. Directory Enumeration With Uncacheable Dirent Metadata

In this scenario, the directory has the uncacheable dirent metadata attribute set. The client retrieves dirent metadata from the server on each READDIR.

Application             NFSv4.2 Client        NFSv4.2 Server
-----------             --------------        --------------
readdir("/dir")
   |
   |                     READDIR, size and
   |                     time_modify requested
   |-------------------->------------------------>
   |                     entries:
   |                       a (size=100)
   |                       b (size=200)
   |                       c (size=300)
   |<--------------------<------------------------
   |<-- names a, b, c

stat("/dir/a")
   |<-- size=100

                                            (concurrent writer extends
                                             a from size=100 to
                                             size=500)

readdir("/dir")
   |
   |                     READDIR, size and
   |                     time_modify requested
   |                     (cache not consulted)
   |-------------------->------------------------>
   |                     entries:
   |                       a (size=500)
   |                       b (size=200)
   |                       c (size=300)
   |<--------------------<------------------------
   |<-- names a, b, c

stat("/dir/a")
   |<-- size=500
Figure 2: Dirent Metadata Not Cached

In this case, Figure 2 shows that each readdir results in a READDIR sent to the server, and that the attributes it returns refresh what a following stat observes. The set of entries returned is unchanged between calls; only the attribute value differs. The client may still cache other information, provided the externally observable behavior is equivalent to retrieving dirent metadata from the server on each READDIR.

6.3. Discussion

This example demonstrates that the uncacheable dirent metadata attribute does not mandate a particular client implementation, but it does require the always-refetch behavior specified in Section 5.1. The attribute ensures that NFSv4.2 clients observe file attribute values reflecting the current state of the server in deployments where staleness of READDIR-returned attributes is known to be a recurring problem.

7. Implementation Status

Note to RFC Editor: please remove this section prior to publication.

There is a prototype Hammerspace server which implements the uncacheable dirent metadata attribute and a prototype Linux client which treats the attribute as an indication to retrieve directory- entry metadata from the server on each READDIR rather than from a local cache.

In the prototype, directories whose contents change at the server at a rate exceeding typical client cache lifetimes are marked with the fattr4_uncacheable_dirent_metadata attribute.

The Linux client decodes the attribute in fs/nfs/nfs4xdr.c into a per-inode flag (nfsi->uncacheable_dirent_metadata, declared in include/linux/nfs_fs.h). The readdir path in fs/nfs/dir.c consults this flag to skip the readdir cache and refetch from the server on each readdir call. Clients may employ more sophisticated mechanisms, such as time-limited caches that revalidate against the server on each READDIR, provided that the externally observable behavior matches the always-refetch semantics described in this document.

The Linux client implementation encodes this attribute as a flag distinct from the companion file-data attribute defined in [I-D.ietf-nfsv4-uncacheable-files]; the two attributes are separated as the two documents specify. That implementation is posted to linux-nfs (patches 4-6 of https://lore.kernel.org/linux-nfs/cover.1785140181.git.snitzer@kernel.org/).

Experience with the prototype indicates that the attribute enables servers to identify directories whose contents change faster than typical NFSv4.2 client cache lifetimes can track, while remaining compatible with existing NFSv4.2 semantics.

8. XDR for the Uncacheable Dirent Metadata Attribute

///
/// typedef bool            fattr4_uncacheable_dirent_metadata;
///
/// const FATTR4_UNCACHEABLE_DIRENT_METADATA   = 88;
///

9. Extraction of XDR

This document contains the external data representation (XDR) [RFC4506] description of the uncacheable dirent metadata attribute. The XDR description is presented in a manner that facilitates easy extraction into a ready-to-compile format. To extract the machine-readable XDR description, use the following shell script:

#!/bin/sh
grep '^ *///' $* | sed 's?^ */// ??' | sed 's?^ *///$??'

For example, if the script is named 'extract.sh' and this document is named 'spec.txt', execute the following command:

sh extract.sh < spec.txt > uncacheable_prot.x

This script removes leading blank spaces and the sentinel sequence '///' from each line. XDR descriptions with the sentinel sequence are embedded throughout the document.

Note that the XDR code contained in this document depends on types from the NFSv4.2 nfs4_prot.x file (generated from [RFC7863]). This includes both nfs types that end with a 4, such as offset4, length4, etc., as well as more generic types such as uint32_t and uint64_t.

While the XDR can be appended to that from [RFC7863], the code snippets should be placed in their appropriate sections within the existing XDR.

10. Security Considerations

This attribute is not a security mechanism. It addresses correctness of client-side caching when client-cached dirent metadata can become stale relative to the current state of the directory at the server. It does not change NFSv4.2 authentication or authorization semantics, and it does not impose access controls on the entries it describes.

Authorization to set or modify the fattr4_uncacheable_dirent_metadata attribute is governed by existing NFSv4.2 authorization mechanisms. Servers MAY restrict modification of this attribute based on local policy, file ownership, or access control rules. This document does not define a new authorization model.

Because the attribute is visible to and affects the caching behavior of all honoring clients, servers should consider the implications of allowing unprivileged users to set or clear it. Setting the attribute on a directory forces honoring clients to abandon READDIR caching and refetch dirent metadata on every enumeration, which can increase load on the server and on other clients. A server MAY restrict modification of the attribute based on administrative configuration, export policy, or ownership.

If the client supports Labeled NFS (see [RFC7204] for background), the client MUST locally enforce the MAC security policies defined by NFSv4.2 ([RFC7862], Section 9). This obligation is independent of whether dirent metadata is cached or refetched.

The uncacheable dirent metadata attribute allows servers to indicate that dirent metadata should not be assumed to remain valid beyond the READDIR that produced it.

11. IANA Considerations

This document has no IANA actions.

NFSv4.2 attribute numbers are assigned by working group coordination rather than through an IANA registry. This document uses attribute number 88, chosen alongside attribute number 87 in [I-D.ietf-nfsv4-uncacheable-files].

12. References

12.1. Normative References

[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
[RFC4506]
Eisler, M., Ed., "XDR: External Data Representation Standard", STD 67, RFC 4506, DOI 10.17487/RFC4506, , <https://www.rfc-editor.org/rfc/rfc4506>.
[RFC7862]
Haynes, T., "Network File System (NFS) Version 4 Minor Version 2 Protocol", RFC 7862, DOI 10.17487/RFC7862, , <https://www.rfc-editor.org/rfc/rfc7862>.
[RFC7863]
Haynes, T., "Network File System (NFS) Version 4 Minor Version 2 External Data Representation Standard (XDR) Description", RFC 7863, DOI 10.17487/RFC7863, , <https://www.rfc-editor.org/rfc/rfc7863>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
[RFC8178]
Noveck, D., "Rules for NFSv4 Extensions and Minor Versions", RFC 8178, DOI 10.17487/RFC8178, , <https://www.rfc-editor.org/rfc/rfc8178>.
[RFC8881]
Noveck, D., Ed. and C. Lever, "Network File System (NFS) Version 4 Minor Version 1 Protocol", RFC 8881, DOI 10.17487/RFC8881, , <https://www.rfc-editor.org/rfc/rfc8881>.

12.2. Informative References

[I-D.ietf-nfsv4-uncacheable-files]
Haynes, T., "Adding an Uncacheable File Data Attribute to NFSv4.2", Work in Progress, Internet-Draft, draft-ietf-nfsv4-uncacheable-files-11, , <https://datatracker.ietf.org/doc/html/draft-ietf-nfsv4-uncacheable-files-11>.
[RFC7204]
Haynes, T., "Requirements for Labeled NFS", RFC 7204, DOI 10.17487/RFC7204, , <https://www.rfc-editor.org/rfc/rfc7204>.

Acknowledgments

Trond Myklebust, Mike Snitzer, Jon Flynn, Keith Mannthey, and Thomas Haynes all worked on the prototype at Hammerspace.

Rick Macklem, Chuck Lever, Dave Noveck, Sorin Faibish, Christoph Hellwig, and Jeff Layton reviewed the document.

Chris Inacio, Brian Pawlowski, Chuck Lever, Zahed Sarker, and Gorry Fairhurst helped guide this process.

Author's Address

Thomas Haynes
Hammerspace