apache/druid · error · SegmentLoadingException

Unable to move file , not present in either source or…

Error message

Unable to move file %s, not present in either source or target location

What it means

safeMove's moveObject retries the copy; if the source object does not exist (or the target already exists from a prior partial move) it logs and returns assuming the move already happened, but if both checks fail it throws this SegmentLoadingException because the file exists in neither location.

Solutions

Verify the source loadSpec bucket/key actually contains the file (aws s3 ls).
Check whether another kill/move task already removed or relocated the segment; if so the segment can be marked as dropped.
Correct the segment metadata (druid_segments table) if it points to a stale location.
Check S3-compatible store consistency settings if using MinIO/other providers.

Example fix

// query the segments table to confirm location before moving
// SELECT load_spec FROM druid_segments WHERE id = '<segmentId>';

Defensive patterns

Strategy: try-catch

Validate before calling

// check the object exists before attempting a move
try { s3Client.headObject(srcBucket, srcKey); }
catch (NoSuchKeyException e) { /* already moved/killed — reconcile metadata instead of moving */ }

Try / catch

try { mover.move(segment, target); } catch (SegmentLoadingException e) {
  if (e.getMessage().contains("not present in either source or target")) reconcileSegmentMetadata(segment);
  else throw e;
}

Prevention

Ensure only one kill/move task handles a segment at a time
Reconcile druid_segments metadata with actual S3 state after manual deletions
Verify loadSpec bucket/key values after any deep-storage migration

When it happens

Trigger: moveObject copy fails because the source key is absent, doesNotExist handling triggers, and the target object is also absent — i.e., the segment file was deleted or the key is wrong.

Common situations: Segment already killed by another task between listing and move, wrong bucket/baseKey in loadSpec, manually deleted S3 objects, transient eventual-consistency issues with S3-compatible stores.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/be1347269cdaab4c. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentMover.java:218

    }
    final ServerSideEncryptingAmazonS3 s3Client = this.s3ClientSupplier.get();
    final HeadObjectResponse sourceMetadata;
    try {
      sourceMetadata = s3Client.getObjectMetadata(s3Bucket, s3Path);
    }
    catch (S3Exception e) {
      if (e.statusCode() == 404) {
        // Source is gone; succeed silently if it already landed at the target.
        if (s3Client.doesObjectExist(targetS3Bucket, targetS3Path)) {
          log.info(
              "Not moving file [s3://%s/%s], already present in target location [s3://%s/%s]",
              s3Bucket,
              s3Path,
              targetS3Bucket,
              targetS3Path
          );
        } else {
          throw new SegmentLoadingException(
              "Unable to move file %s, not present in either source or target location",
              copyMsg
          );
        }
        return;
      }
      throw e;
    }
    final StorageClass sc = sourceMetadata.storageClass();
    if (StorageClass.GLACIER.equals(sc) || StorageClass.DEEP_ARCHIVE.equals(sc)) {
      throw S3Exception.builder()
          .message(StringUtils.format(
              "Cannot move file[s3://%s/%s] of storage class [%s], skipping.",
              s3Bucket,
              s3Path,
              sc
          ))
          .build();

View on GitHub (pinned to 9b90983fd2)