{"record":{"id":"1aa91bf2c39e9065","repo":"apache/druid","slug":"failed-to-load-segment-s-in-all-locations","errorCode":null,"errorMessage":"Failed to load segment[%s] in all locations.","messagePattern":"Failed to load segment\\[(.+?)\\] in all locations\\.","errorType":"exception","errorClass":"SegmentLoadingException","httpStatus":null,"severity":"error","filePath":"server/src/main/java/org/apache/druid/segment/loading/SegmentLocalCacheManager.java","lineNumber":2020,"sourceCode":"    final Iterator<StorageLocation> locationsIterator = strategy.getLocations();\n    while (locationsIterator.hasNext()) {\n      final StorageLocation location = locationsIterator.next();\n      if (location.reserve(cacheEntry)) {\n        try {\n          final CompleteSegmentCacheEntry entry = checkComplete(location.getCacheEntry(cacheEntry.id), cacheEntry.id);\n          if (entry != null) {\n            entry.lazyLoadCallback = segmentLoadFailCallback;\n            entry.setOnUnmount(null);\n            entry.mount(location);\n            return entry;\n          }\n        }\n        catch (SegmentLoadingException e) {\n          log.warn(e, \"Failed to load segment[%s] in location[%s], trying next location\", cacheEntry.id, location.getPath());\n        }\n      }\n    }\n    throw new SegmentLoadingException(\"Failed to load segment[%s] in all locations.\", cacheEntry.id);\n  }\n\n  /**\n   * Narrow a {@link CacheEntry} to {@link CompleteSegmentCacheEntry} with a defensive check. Returns null when the\n   * entry is missing. Throws when a non-complete entry (e.g. {@link PartialSegmentMetadataCacheEntry}) is registered\n   * under the same identifier. Callers of {@link #assignLocationAndMount} only operate on the non-virtual-storage\n   * eager path, so encountering a partial entry here is a programming error.\n   */\n  @Nullable\n  private static CompleteSegmentCacheEntry checkComplete(@Nullable CacheEntry entry, SegmentCacheEntryIdentifier id)\n  {\n    if (entry == null) {\n      return null;\n    }\n    if (entry instanceof CompleteSegmentCacheEntry complete) {\n      return complete;\n    }\n    throw DruidException.defensive(","sourceCodeStart":2002,"sourceCodeEnd":2038,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/segment/loading/SegmentLocalCacheManager.java#L2002-L2038","documentation":"Thrown after the manager has attempted to load a segment in every configured cache location and each attempt failed with a SegmentLoadingException. It is the terminal error aggregating per-location failures (each logged as 'Failed to load segment[%s] in location[%s], trying next location').","triggerScenarios":"Any path calling assignLocationAndMount/multi-location load where every location's load attempt throws SegmentLoadingException (download failure, unparseable segment, no space, all locations unhealthy).","commonSituations":"All cache volumes full or unwritable; deep storage unreachable so downloads fail at each location; corrupt segment files in deep storage; all locations' disks failing.","solutions":["Read the per-location warnings logged immediately before this error to find the root cause (they carry the real exception).","Check deep storage connectivity and credentials from the historical node.","Verify cache locations have free space and write permissions; run df and touch a test file in each location path.","Retry the load via coordinator; if the segment is corrupt in deep storage, re-ingest or restore from backup."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { loadSegment(segmentId); } catch (SegmentLoadingException e) { // per-location warnings logged above carry root cause; trigger coordinator re-load or mark segment failed }","preventionTips":["Configure multiple cache locations so a single-volume failure does not exhaust all attempts.","Monitor deep storage connectivity and disk health per location.","Read the preceding per-location warn logs before treating this error as the root cause."],"tags":["segment-cache","segment-loading","all-locations-failed"],"backgroundTag":"file-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}