apache/druid · error · SegmentLoadingException

Unable to remove marker file for

Error message

Unable to remove marker file for [%s]

What it means

Thrown after a successful download when downloadStartMarker.delete() returns false, meaning the marker file could not be removed from disk. Because the marker's presence signals an incomplete download, failing to delete it leaves the segment looking partial and it may be reclaimed by later sweeps.

Solutions

1. Check write/delete permissions on the cache location directory and its contents for the Druid process user.
2. Ensure the filesystem is not read-only or out of inodes/space (df -i, mount output).
3. Stop external cleanup scripts from touching druid cache directories.
4. Retry the load; if the marker persists, delete it manually and re-load the segment.

// before loading, verify location is writable
dPath.toFile().setWritable(true) && Files.isWritable(locationPath) || throw new IllegalStateException("cache location not writable")

try { loadSegment(segment); } catch (SegmentLoadingException e) { if (e.getMessage().contains("Unable to remove marker file")) { checkMountReadOnlyAndAlert(); } }

When it happens

Trigger: loadInLocation completes and loadInLocation(segment, storageDir) succeeds, but File.delete() on the marker returns false — usually filesystem-level causes (permission change, directory removed under the manager, immutable/readonly mount).

Common situations: Cache volume switched to read-only or full; external cleanup job deleting files concurrently; permission mismatches after running as a different user.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Unable to create marker file for
• Was not able to create new download marker for
• Cannot create tempDir
• Could not create temp distribution directory.
• Couldn't kill segment

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/bc137068ce7cbbc6. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/segment/loading/SegmentLocalCacheManager.java:2423

    }

    @GuardedBy("entryLock")
    private void loadInLocationWithStartMarker(final DataSegment segment, final File storageDir)
        throws SegmentLoadingException
    {
      // We use a marker to prevent the case where a segment is downloaded, but before the download completes,
      // the parent directories of the segment are removed
      final File downloadStartMarker = new File(storageDir, DOWNLOAD_START_MARKER_FILE_NAME);
      try {
        FileUtils.mkdirp(storageDir);

        if (!downloadStartMarker.createNewFile()) {
          throw new SegmentLoadingException("Was not able to create new download marker for [%s]", storageDir);
        }
        loadInLocation(segment, storageDir);

        if (!downloadStartMarker.delete()) {
          throw new SegmentLoadingException("Unable to remove marker file for [%s]", storageDir);
        }
      }
      catch (IOException e) {
        throw new SegmentLoadingException(e, "Unable to create marker file for [%s]", storageDir);
      }
    }

    @GuardedBy("entryLock")
    private void loadInLocation(final DataSegment segment, final File storageDir)
        throws SegmentLoadingException
    {
      // LoadSpec isn't materialized until here so that any system can interpret Segment without having to have all the
      // LoadSpec dependencies.
      final LoadSpec loadSpec = jsonMapper.convertValue(segment.getLoadSpec(), LoadSpec.class);
      final LoadSpec.LoadSpecResult result = loadSpec.loadSegment(storageDir);
      if (result.getSize() != segment.getSize()) {
        log.warn(
            "Segment [%s] is different than expected size. Expected [%d] found [%d]",

View on GitHub (pinned to 9b90983fd2)