User Tools
2mib_fs_dax
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision Next revision Both sides next revision | ||
|
2mib_fs_dax [2017/12/20 20:37] Ross Zwisler created |
2mib_fs_dax [2019/08/06 00:06] Dan Williams |
||
|---|---|---|---|
| Line 16: | Line 16: | ||
| Here are the steps that I've used to successfully get filesystem DAX PMDs: | Here are the steps that I've used to successfully get filesystem DAX PMDs: | ||
| - | 1. First, make sure that our persistent memory block device starts at a 2 MiB aligned physical address. | + | 1. First, make sure that your namespace is in 'fsdax' mode. |
| + | |||
| + | # ndctl list --human | ||
| + | { | ||
| + | "dev":"namespace0.0", | ||
| + | "mode":"fsdax", | ||
| + | "size":"16.73 GiB (17.96 GB)", | ||
| + | "uuid":"179e5b98-96ee-4988-ba9f-ed9383d11598", | ||
| + | "sector_size":512, | ||
| + | "blockdev":"pmem0", | ||
| + | "numa_node":0 | ||
| + | } | ||
| + | |||
| + | 2. Next, make sure that our persistent memory block device starts at a 2 MiB aligned physical address. | ||
| This is important because when we ask the filesystem for 2 MiB aligned and sized block allocations it will provide those block allocations relative to the beginning of its block device. If the filesystem is built on top of a namespace whose data starts at a 1 MiB aligned offset, for example, a block allocation that is 2 MiB aligned from the point of view of the filesystem will still be only 1 MiB aligned from DAX's point of view. This will cause DAX to fall back to 4 KiB page faults. | This is important because when we ask the filesystem for 2 MiB aligned and sized block allocations it will provide those block allocations relative to the beginning of its block device. If the filesystem is built on top of a namespace whose data starts at a 1 MiB aligned offset, for example, a block allocation that is 2 MiB aligned from the point of view of the filesystem will still be only 1 MiB aligned from DAX's point of view. This will cause DAX to fall back to 4 KiB page faults. | ||
| Line 24: | Line 37: | ||
| # cat /proc/iomem | # cat /proc/iomem | ||
| ... | ... | ||
| - | 140000000-57fffffff : Persistent Memory | + | 140000000-57fdfffff : Persistent Memory |
| - | 140000000-57fffffff : namespace0.0 | + | 140000000-57fdfffff : namespace0.0 |
| Our namespace in this case begins at 5 GiB (0x1 4000 0000), which is 2 MiB (0x20 0000) aligned. | Our namespace in this case begins at 5 GiB (0x1 4000 0000), which is 2 MiB (0x20 0000) aligned. | ||
| Line 32: | Line 45: | ||
| # fdisk -l /dev/pmem0 | # fdisk -l /dev/pmem0 | ||
| - | Disk /dev/pmem0: 16.8 GiB, 17966301184 bytes, 35090432 sectors | + | Disk /dev/pmem0: 16.7 GiB, 17964204032 bytes, 35086336 sectors |
| Units: sectors of 1 * 512 = 512 bytes | Units: sectors of 1 * 512 = 512 bytes | ||
| Sector size (logical/physical): 512 bytes / 4096 bytes | Sector size (logical/physical): 512 bytes / 4096 bytes | ||
| I/O size (minimum/optimal): 4096 bytes / 4096 bytes | I/O size (minimum/optimal): 4096 bytes / 4096 bytes | ||
| Disklabel type: dos | Disklabel type: dos | ||
| - | Disk identifier: 0x5af75158 | + | Disk identifier: 0xfd17c8f9 |
| Device Boot Start End Sectors Size Id Type | Device Boot Start End Sectors Size Id Type | ||
| - | /dev/pmem0p1 2048 35090431 35088384 16.7G 83 Linux | + | /dev/pmem0p1 2048 35086335 35084288 16.7G 83 Linux |
| A filesystem built on top of this partition won't be able to provide DAX with 2 MiB aligned block allocations. We instead need to have our partition begin at a 2 MiB aligned boundary: | A filesystem built on top of this partition won't be able to provide DAX with 2 MiB aligned block allocations. We instead need to have our partition begin at a 2 MiB aligned boundary: | ||
| # fdisk -l /dev/pmem0 | # fdisk -l /dev/pmem0 | ||
| - | Disk /dev/pmem0: 16.8 GiB, 17966301184 bytes, 35090432 sectors | + | Disk /dev/pmem0: 16.7 GiB, 17964204032 bytes, 35086336 sectors |
| Units: sectors of 1 * 512 = 512 bytes | Units: sectors of 1 * 512 = 512 bytes | ||
| Sector size (logical/physical): 512 bytes / 4096 bytes | Sector size (logical/physical): 512 bytes / 4096 bytes | ||
| I/O size (minimum/optimal): 4096 bytes / 4096 bytes | I/O size (minimum/optimal): 4096 bytes / 4096 bytes | ||
| Disklabel type: dos | Disklabel type: dos | ||
| - | Disk identifier: 0x276da416 | + | Disk identifier: 0xfd17c8f9 |
| Device Boot Start End Sectors Size Id Type | Device Boot Start End Sectors Size Id Type | ||
| - | /dev/pmem0p1 4096 35090431 35086336 16.7G 83 Linux | + | /dev/pmem0p1 4096 35086335 35082240 16.7G 83 Linux |
| - | 2. Once we have a block device that starts at a 2 MiB aligned persistent memory address, we then need to create a filesystem on top of it that will give us 2 MiB aligned and sized block allocations. Here are the commands to do that with either ext4 or XFS: | + | 3. Once we have a block device that starts at a 2 MiB aligned persistent memory address, we then need to create a filesystem on top of it that will give us 2 MiB aligned and sized block allocations. Here are the commands to do that with either ext4 or XFS: |
| ext4: | ext4: | ||
| Line 61: | Line 74: | ||
| xfs: | xfs: | ||
| - | # mkfs.xfs -f -d su=2m,sw=1 /dev/pmem0 | + | # mkfs.xfs -f -d su=2m,sw=1 -m reflink=0 /dev/pmem0 |
| # mount /dev/pmem0 /mnt/dax | # mount /dev/pmem0 /mnt/dax | ||
| # xfs_io -c "extsize 2m" /mnt/dax | # xfs_io -c "extsize 2m" /mnt/dax | ||
| Line 70: | Line 83: | ||
| [[https://linux.die.net/man/8/xfs_io|xfs_io(8)]] for more details. | [[https://linux.die.net/man/8/xfs_io|xfs_io(8)]] for more details. | ||
| - | 3. Now that we have a filesystem that can give us 2 MiB sized and aligned | + | 4. Now that we have a filesystem that can give us 2 MiB sized and aligned |
| block allocations we just need to create a file that will receive those | block allocations we just need to create a file that will receive those | ||
| allocations. To do this we need to begin with a file that is at least 2 MiB | allocations. To do this we need to begin with a file that is at least 2 MiB | ||
2mib_fs_dax.txt · Last modified: 2020/09/24 00:43 by Darrick Wong
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
