Persistent Memory Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Prior to kernel 5.8 ======

Prior to kernel 5.8 enabling dax is done on an entire file system basis.  It is done with a "-o dax" mount option.

<code>
$ mount -o dax /dev/pmem0p1 /mnt/ext4-pmem0
$ mount -o dax /dev/pmem0p2 /mnt/xfs-pmem0
</code>

====== Kernels 5.8 or later ======

As of kernel 5.8 additional DAX enabling options are available on some file systems, specifically:

===== ext2 =====

When mounting the filesystem, use the "-o dax" option on the command line or add 'dax' to the options in /etc/fstab.  This works to enable DAX on all files within the filesystem.  It is equivalent to the '-o dax=always' behavior below.

===== ext4 and xfs =====

As of kernel 5.8 ext4 and xfs supports the new per-file dax configuration.

==== Summary ====

  - There exists an in-kernel file access mode flag S_DAX that corresponds to the statx flag STATX_ATTR_DAX.  See the manpage for statx(2) for details about this access mode.
  - There exists a persistent flag FS_XFLAG_DAX that can be applied to regular files and directories. This advisory flag can be set or cleared at any time, but doing so does not immediately affect the S_DAX state.
  - If the persistent FS_XFLAG_DAX flag is set on a directory, this flag will be inherited by all regular files and subdirectories that are subsequently created in this directory. Files and subdirectories that exist at the time this flag is set or cleared on the parent directory are not modified by this modification of the parent directory.
  - There exist dax mount options which can override FS_XFLAG_DAX in the setting of the S_DAX flag.  Given underlying storage which supports DAX the following hold:
    * "-o dax=inode"  means "follow FS_XFLAG_DAX" and is the default.
    * "-o dax=never"  means "never set S_DAX, ignore FS_XFLAG_DAX."
    * "-o dax=always" means "always set S_DAX ignore FS_XFLAG_DAX."
    * "-o dax"        is a legacy option which is an alias for "dax=always".
      * __"-o dax" may be removed in the future so "-o dax=always" is the preferred method for specifying this behavior going forward.__
  - The S_DAX policy can be changed via:
    - Setting the parent directory FS_XFLAG_DAX as needed before files are created
    - Setting the appropriate dax="foo" mount option
    -  Changing the FS_XFLAG_DAX flag on existing regular files and directories.  This has runtime constraints and limitations that are described in 6) below.
  - When changing the S_DAX policy via toggling the persistent FS_XFLAG_DAX flag, the change in behaviour for existing regular files may not occur immediately.  If the change must take effect immediately, the administrator needs to:
    - stop the application so there are no active references to the data set the policy change will affect
    - evict the data set from kernel caches so it will be re-instantiated when the application is restarted. This can be achieved by:
       - drop-caches
       - a filesystem unmount and mount cycle
       - a system reboot

    NOTE: Modifications to and the inheritance behavior of FS_XFLAG_DAX remain
    the same even when the filesystem is mounted with a dax option.  However,
    in-core inode state (S_DAX) will be overridden until the filesystem is
    remounted with dax=inode and the inode is evicted from kernel memory.

==== Examples and details ====

There are 2 per-file dax flags.  One is a persistent inode setting (FS_XFLAG_DAX) and the other is a volatile flag indicating the active state of the feature (S_DAX).

FS_XFLAG_DAX is preserved within the file system.  This persistent config setting can be set, cleared and/or queried using the FS_IOC_FS[GS]ETXATTR ioctl (see ioctl_xfs_fsgetxattr(2)) or an utility such as 'xfs_io'.

New files and directories automatically inherit FS_XFLAG_DAX from their parent directory _when_ _created_.  Therefore, setting FS_XFLAG_DAX at directory creation time can be used to set a default behavior for an entire
sub-tree.

To clarify inheritance mount the file system with the inode option.

<code>
$ mount /dev/pmem0p2 /mnt/xfs-pmem0
</code>

Is equivalent to:

<code>
$ mount -o dax=inode /dev/pmem0p2 /mnt/xfs-pmem0
</code>

And here are 3 examples showing how to enable dax on individual files and/or directories.

=== Inheritance Example A: ===

<code>
$ mkdir -p a/b/c
$ xfs_io -c 'chattr +x' a
$ xfs_io -c 'lsattr' a
--------------x- a
$ mkdir -p a/b/c/d
$ mkdir -p a/e
</code>
        Results in:
        
        dax: a,e
        no dax: b,c,d

=== Inheritance Example B: ===

<code>
$ mkdir a
$ xfs_io -c 'chattr +x' a
$ xfs_io -c 'lsattr' a
--------------x- a
$ mkdir -p a/b/c/d
</code>

        Results in:
        
        dax: a,b,c,d
        no dax:

=== Inheritance Example C: ===

<code>
$ mkdir -p a/b/c
$ xfs_io -c 'chattr +x' c
$ xfs_io -c 'lsattr' c
--------------x- c
$ mkdir -p a/b/c/d
</code>

        Results in:
        
        dax: c,d
        no dax: a,b


=== Detecting if a file is using dax ===

The current enabled state (S_DAX) is set when a file inode is instantiated in memory by the kernel.  It is set based on the underlying media support, the value of FS_XFLAG_DAX and the file system's dax mount option.

statx can be used to query S_DAX.  NOTE that only regular files will ever have S_DAX set and therefore statx will never indicate that S_DAX is set on directories.

Continuing with Example C above we create a file foo in a dax enabled directory and it is enabled for dax with the FS_XFLAG_DAX set as well as S_DAX being set.

<code>
$ touch a/b/c/foo
$ xfs_io -c 'lsattr' a/b/c/foo                       # FS_XFLAG_DAX == true
--------------x- a/b/c/foo
$ xfs_io -c 'statx -r' a/b/c/foo | grep attributes   # S_DAX == true
stat.attributes = 0x2000
</code>

Setting the FS_XFLAG_DAX flag (specifically or through inheritance) occurs even if the underlying media does not support dax and/or the filesystem is overridden with a mount option.
Persistent Memory Wiki

User Tools

Site Tools

Page Tools
