{"record":{"id":"25d633c8b63afa41","repo":"apache/druid","slug":"read-0-bytes-from-segmentdir-s","errorCode":null,"errorMessage":"Read 0 bytes from segmentDir[%s]","messagePattern":"Read 0 bytes from segmentDir\\[(.+?)\\]","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/worker/shuffle/LocalIntermediaryDataManager.java","lineNumber":311,"sourceCode":"\n    if (!(segment.getShardSpec() instanceof BucketNumberedShardSpec)) {\n      throw new IAE(\n          \"Invalid shardSpec type. Expected [%s] but got [%s]\",\n          BucketNumberedShardSpec.class.getName(),\n          segment.getShardSpec().getClass().getName()\n      );\n    }\n    final BucketNumberedShardSpec<?> bucketNumberedShardSpec = (BucketNumberedShardSpec<?>) segment.getShardSpec();\n\n    //noinspection unused\n    try (final Closer ignoredCloser = closer) {\n      FileUtils.mkdirp(taskTempDir);\n\n      // Temporary compressed file. Will be removed when taskTempDir is deleted.\n      final File tempZippedFile = new File(taskTempDir, segment.getId().toString());\n      final long unzippedSizeBytes = CompressionUtils.zip(segmentDir, tempZippedFile);\n      if (unzippedSizeBytes == 0) {\n        throw new IOE(\n            \"Read 0 bytes from segmentDir[%s]\",\n            segmentDir.getAbsolutePath()\n        );\n      }\n\n      // Try copying the zipped segment to one of storage locations\n      for (int i = 0; i < shuffleDataLocations.size(); i++) {\n        final StorageLocation location = shuffleDataLocations.get(\n            Math.floorMod(cursor.getAndIncrement(), shuffleDataLocations.size())\n        );\n        final String partitionFilePath = getPartitionFilePath(\n            supervisorTaskId,\n            subTaskId,\n            segment.getInterval(),\n            bucketNumberedShardSpec.getBucketId() // we must use the bucket ID instead of partition ID\n        );\n        File destFile = null;\n        if (location.reserve(new FileNameCacheEntry(partitionFilePath, tempZippedFile.length()))) {","sourceCodeStart":293,"sourceCodeEnd":329,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/worker/shuffle/LocalIntermediaryDataManager.java#L293-L329","documentation":"LocalIntermediaryDataManager.addSegment zips a task's segment directory into the intermediary shuffle storage. CompressionUtils.zip returning 0 bytes means the segment directory was empty or unreadable, so no usable segment artifact exists; it throws IOE rather than storing an empty segment file.","triggerScenarios":"addSegment() called with a segmentDir whose contents total zero unzipped bytes — e.g. the directory is empty, files were already cleaned up, or reading the directory fails silently.","commonSituations":"Task directory cleaned concurrently (as in the isCleanedUpAfter3s test that exercises this), a task producing no rows for a segment, or disk/filesystem issues yielding an empty dir.","solutions":["Verify the task actually wrote the segment files into segmentDir before reporting the segment","Re-run the failing task to regenerate the segment","Check for concurrent cleanup (cleanup thresholds, disk reaper) racing with segment publication","Inspect filesystem permissions/health for the task's temp directory"],"exampleFix":"null","handlingStrategy":"try-catch","validationCode":"final long bytes = FileUtils.sizeOfDirectory(segmentDir);\nif (bytes == 0) { throw new IllegalStateException(\"segmentDir \" + segmentDir + \" is empty before zip\"); }","typeGuard":null,"tryCatchPattern":"try { dataManager.addSegment(...); } catch (IOException e) { if (e.getMessage().startsWith(\"Read 0 bytes\")) { rescheduleTask(); } else { throw e; } }","preventionTips":["Ensure tasks flush segment files before publication","Avoid setting cleanup thresholds shorter than task runtime","Monitor worker disk health"],"tags":["druid","indexing","shuffle","io"],"backgroundTag":"empty-required-field","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}