apache/druid · error · SegmentLoadingException

Failed to reserve partial metadata for segment[%s] on locati

Error message

Failed to reserve partial metadata for segment[%s] on location[%s] during bootstrap

What it means

Thrown by reservePartialForBootstrap when a valid partial segment layout exists on disk and a range reader opened successfully, but tryReservePartialAt could not reserve the partial metadata entry on the location. Reservation can fail due to eviction pressure, lock contention, or the location refusing to track the new entry during bootstrap.

Source

Thrown at server/src/main/java/org/apache/druid/segment/loading/SegmentLocalCacheManager.java:1716

        // load fetches it fresh rather than reserving an entry that could never fetch anything. The info file goes
        // with it: it asserts that this segment has local cache state, which stops being true here, and nothing else
        // will remove it (no entry was reserved, so there is no unmount hook to fire).
        log.warn(
            "On-disk partial-load layout for segment[%s] in [%s] has no usable range reader (this should not "
            + "happen); deleting it so the segment can be re-loaded.",
            dataSegment.getId(),
            partialDir
        );
        atomicMoveAndDeleteCacheEntryDirectory(partialDir);
        deleteSegmentInfoFile(dataSegment);
        throw new SegmentLoadingException(
            "No usable range reader for partial segment[%s]; its local layout has been reclaimed",
            dataSegment.getId()
        );
      }
      final ReservedPartial reserved = tryReservePartialAt(dataSegment, rangeReader, location, false);
      if (reserved == null) {
        throw new SegmentLoadingException(
            "Failed to reserve partial metadata for segment[%s] on location[%s] during bootstrap",
            dataSegment.getId(),
            location.getPath()
        );
      }
      return reserved.hold;
    }
    return null;
  }

  /**
   * Reserve a {@link CompleteSegmentCacheEntry} for an eagerly-downloaded segment whose files are on disk, so
   * {@link #bootstrap} has something to mount, and hand back the hold it was reserved under. Returns {@code null} when
   * this segment has no complete layout on disk, or already has an entry. Virtual storage only: the legacy path
   * reserves these statically in {@link #getCachedSegments} and never evicts them.
   * <p>
   * Reserving under a hold is what makes the mount safe: reclaim skips held entries only, so an unheld entry can be
   * selected as a victim while a parallel bootstrap thread is still mounting it. Releasing the hold once mounted

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Increase druid.segmentCache.locations maxSize (or free disk) so the location can accept the reservation.
2. Check logs for concurrent eviction activity targeting the same segment during startup; restart after quieting load/eviction churn.
3. Manually remove the affected partial directory so bootstrap skips the broken reservation and the segment re-downloads.
4. Verify location disks are healthy and not over-capacity (du vs configured maxSize).

Example fix

// before
druid.segmentCache.locations=[{"path":"/var/druid/segment-cache","maxSize":10000000000}]
// after
druid.segmentCache.locations=[{"path":"/var/druid/segment-cache","maxSize":50000000000}]

// ensure each cache location's maxSize exceeds the sum of segment sizes on disk
long onDisk = Files.walk(locationPath).mapToLong(p -> p.toFile().length()).sum();
if (onDisk >= maxSize) { throw new IllegalStateException("maxSize too small for existing cache"); }

try { bootstrap(); } catch (SegmentLoadingException e) { if (e.getMessage().contains("Failed to reserve partial metadata")) { cleanupPartialDir(segmentId); } }

When it happens

Trigger: During bootstrap of a location containing a partial segment layout, tryReservePartialAt(dataSegment, rangeReader, location, false) returns null — e.g. the location's tracker cannot add the reservation (size accounting failure, concurrent eviction, or entry already being processed).

Common situations: Cache locations at or near maxSize capacity during historical startup; multiple bootstrap/recovery paths racing over the same segment; disk accounting inconsistencies left by a previous crash.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Location[%s] with available bytes[%,d] cannot reserve segmen
• No usable range reader for partial segment[%s]; its local la
• druid.segmentCache.locations must be set on historicals.
• Failed to write partial info file for segment[%s]
• Failed to load segment[%s] in all locations.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/fd70fb582d58d4c0. Report an issue: GitHub.