apache/hadoop · error · IOException

Exception while writing data to persistent storage dir: ${re

Error message

Exception while writing data to persistent storage dir: ${realPmemDir}

What it means

Error "Exception while writing data to persistent storage dir: ${realPmemDir}" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/PmemVolumeManager.java:353

    String uuidStr = UUID.randomUUID().toString();
    String testFilePath = realPmemDir.getPath() + "/.verify.pmem." + uuidStr;
    byte[] contents = uuidStr.getBytes(StandardCharsets.UTF_8);
    RandomAccessFile testFile = null;
    MappedByteBuffer out = null;
    try {
      testFile = new RandomAccessFile(testFilePath, "rw");
      out = testFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0,
          contents.length);
      if (out == null) {
        throw new IOException(
            "Failed to map the test file under " + realPmemDir);
      }
      out.put(contents);
      // Forces to write data to storage device containing the mapped file
      out.force();
      return realPmemDir;
    } catch (IOException e) {
      throw new IOException(
          "Exception while writing data to persistent storage dir: " +
              realPmemDir, e);
    } finally {
      if (out != null) {
        out.clear();
      }
      if (testFile != null) {
        IOUtils.closeStream(testFile);
        NativeIO.POSIX.munmap(out);
        try {
          FsDatasetUtil.deleteMappedFile(testFilePath);
        } catch (IOException e) {
          LOG.warn("Failed to delete test file " + testFilePath +
              " from persistent memory", e);
        }
      }
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Inspect the DataNode log for the underlying exception (I/O error, no space) and repair the pmem device.
2. Verify the pmem filesystem is healthy and writable; reformat and remount if corrupted.

When it happens

Trigger: Pmem cache initialization when writing test data into the pmem directory throws, e.g. device full or I/O error.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b3b3c96efb971336. Report an issue: GitHub.