{"record":{"id":"76bbf2377adf8bed","repo":"apache/druid","slug":"failed-on-syncing-fd-d-offset-d-bytes-d","errorCode":null,"errorMessage":"failed on syncing fd [%d], offset [%d], bytes [%d], ret_code [%d], errno [%d]","messagePattern":"failed on syncing fd \\[(.+?)\\], offset \\[(.+?)\\], bytes \\[(.+?)\\], ret_code \\[(.+?)\\], errno \\[(.+?)\\]","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"processing/src/main/java/org/apache/druid/java/util/common/io/NativeIO.java","lineNumber":177,"sourceCode":"  }\n\n  /**\n   * Sync part of an open file to the file system.\n   *\n   * @param fd      The file descriptor of the source file.\n   * @param offset  The offset within the file.\n   * @param nbytes  The number of bytes to be synced.\n   * @param flags   Signal how to synchronize\n   */\n  private static void trySyncFileRange(int fd, long offset, long nbytes, int flags)\n  {\n    if (!initialized || !syncFileRangePossible || fd < 0) {\n      return;\n    }\n    try {\n      int ret_code = sync_file_range(fd, offset, nbytes, flags);\n      if (ret_code != 0) {\n        log.warn(\"failed on syncing fd [%d], offset [%d], bytes [%d], ret_code [%d], errno [%d]\",\n            fd, offset, nbytes, ret_code, Native.getLastError());\n        return;\n      }\n    }\n    catch (UnsupportedOperationException uoe) {\n      log.warn(uoe, \"sync_file_range is not supported\");\n      syncFileRangePossible = false;\n    }\n    catch (UnsatisfiedLinkError nle) {\n      log.warn(nle, \"sync_file_range failed on fd [%d], offset [%d], bytes [%d]\", fd, offset, nbytes);\n      syncFileRangePossible = false;\n    }\n    catch (Exception e) {\n      log.warn(e, \"Unknown exception: sync_file_range failed on fd [%d], offset [%d], bytes [%d]\",\n          fd, offset, nbytes);\n      syncFileRangePossible = false;\n    }\n  }","sourceCodeStart":159,"sourceCodeEnd":195,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/java/util/common/io/NativeIO.java#L159-L195","documentation":"NativeIO.trySyncFileRange calls Linux's sync_file_range(2) via JNA to asynchronously flush a file's dirty pages to disk. A non-zero return code means the kernel rejected the sync; the code logs a warning with ret_code and errno and returns without throwing, since syncing is a best-effort optimization. Unsupported platforms set syncFileRangePossible=false permanently after the first failure.","triggerScenarios":"Calling trySyncFileRange on a file descriptor where sync_file_range returns an error: invalid fd/offset/length alignment on some filesystems, EINVAL on unsupported filesystems (e.g. NFS, tmpfs, overlayfs), or EBADF/EIO on a closed or errored fd.","commonSituations":"Running inside containers with overlayfs backing stores where sync_file_range is unsupported; writing to NFS mounts; using file offsets not aligned to page size on filesystems that require it; kernel versions where the syscall isn't available for the fs.","solutions":["Check the logged errno: EINVAL typically means the filesystem (overlayfs, NFS, tmpfs) doesn't support sync_file_range","Move the data directory to a supported filesystem (ext4/xfs on a real block device)","Ensure fd/offset/length are valid and the file is open when sync is attempted","Treat as advisory: data correctness is unaffected; the call is skipped after repeated unsupported errors"],"exampleFix":"// before: bind-mounting Druid segment cache onto overlayfs/tmpfs in Docker\nVOLUME [\"/opt/druid/var\"]\n// after: use a volume on ext4/xfs\n// docker run -v druid-data:/opt/druid/var ...  # volume backed by block device","handlingStrategy":"fallback","validationCode":"// Only rely on sync_file_range on Linux with a supported local filesystem\nboolean syncSupported = System.getProperty(\"os.name\").equals(\"Linux\")\n    && !mountPoint.startsWith(\"/mnt\") /* exclude NFS/overlay mounts */;","typeGuard":null,"tryCatchPattern":"// Best-effort sync: log and continue\ntry {\n  NativeIO.trySyncFileRange(fd, offset, len, flags);\n} catch (UnsupportedOperationException e) {\n  // platform/filesystem unsupported; skip syncing permanently\n}","preventionTips":["Place Druid data directories on ext4 or xfs block devices, not overlayfs/NFS/tmpfs","Expect the warning on containers whose writable layer is overlayfs","Treat sync_file_range as an optimization; fsync remains the correctness boundary","Match kernel versions that support sync_file_range for the target filesystem"],"tags":["native","linux","filesystem","fsync","jna"],"backgroundTag":"unsupported-operation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}