{"record":{"id":"26f3d47aad6340c3","repo":"apache/druid","slug":"was-not-able-to-create-new-download-marker-for-s","errorCode":null,"errorMessage":"Was not able to create new download marker for [%s]","messagePattern":"Was not able to create new download marker for \\[(.+?)\\]","errorType":"exception","errorClass":"SegmentLoadingException","httpStatus":null,"severity":"error","filePath":"server/src/main/java/org/apache/druid/segment/loading/SegmentLocalCacheManager.java","lineNumber":2418,"sourceCode":"    }\n\n    public File toPotentialLocation(final File location)\n    {\n      return new File(location, relativePathString);\n    }\n\n    @GuardedBy(\"entryLock\")\n    private void loadInLocationWithStartMarker(final DataSegment segment, final File storageDir)\n        throws SegmentLoadingException\n    {\n      // We use a marker to prevent the case where a segment is downloaded, but before the download completes,\n      // the parent directories of the segment are removed\n      final File downloadStartMarker = new File(storageDir, DOWNLOAD_START_MARKER_FILE_NAME);\n      try {\n        FileUtils.mkdirp(storageDir);\n\n        if (!downloadStartMarker.createNewFile()) {\n          throw new SegmentLoadingException(\"Was not able to create new download marker for [%s]\", storageDir);\n        }\n        loadInLocation(segment, storageDir);\n\n        if (!downloadStartMarker.delete()) {\n          throw new SegmentLoadingException(\"Unable to remove marker file for [%s]\", storageDir);\n        }\n      }\n      catch (IOException e) {\n        throw new SegmentLoadingException(e, \"Unable to create marker file for [%s]\", storageDir);\n      }\n    }\n\n    @GuardedBy(\"entryLock\")\n    private void loadInLocation(final DataSegment segment, final File storageDir)\n        throws SegmentLoadingException\n    {\n      // LoadSpec isn't materialized until here so that any system can interpret Segment without having to have all the\n      // LoadSpec dependencies.","sourceCodeStart":2400,"sourceCodeEnd":2436,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/segment/loading/SegmentLocalCacheManager.java#L2400-L2436","documentation":"Thrown when creating the DOWNLOAD_START_MARKER file inside the segment's storage directory fails because File.createNewFile() returned false, meaning a file already exists at that path (typically a stale marker left by an earlier interrupted download). The download cannot safely proceed without a fresh marker.","triggerScenarios":"loadInLocation path where new File(storageDir, DOWNLOAD_START_MARKER_FILE_NAME).createNewFile() returns false — the marker already exists in storageDir, e.g. leftover from a crashed previous download of the same segment.","commonSituations":"Historical restart after unclean shutdown with orphaned download markers; concurrent loads of the same segment racing into the same directory; leftover markers not cleaned by startup sweep.","solutions":["Delete the stale downloadStartMarker (and partial directory) for the segment under the cache location, then retry the load.","Check for concurrent loads of the same segment on this host; ensure only one historical/worker owns the segment.","Restart the historical so the startup cleanup removes orphaned markers and partial dirs.","Verify cache directory permissions allow create/delete operations."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { loadSegment(segment); } catch (SegmentLoadingException e) { if (e.getMessage().contains(\"create new download marker\")) { deleteStaleMarkerAndRetry(segment); } }","preventionTips":["Ensure clean shutdown/restart cycles so stale markers are swept.","Prevent two loaders from downloading the same segment concurrently into the same directory.","Add a startup task that purges leftover DOWNLOAD_START_MARKER files."],"tags":["segment-cache","download-marker","file-already-exists","stale-state"],"backgroundTag":"file-already-exists","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}