{"record":{"id":"8994d1aaa1c171cf","repo":"apache/druid","slug":"unable-to-transfer-bytes-from-file-s-at-position","errorCode":null,"errorMessage":"Unable to transfer bytes from file[%s] at position[%,d]","messagePattern":"Unable to transfer bytes from file\\[(.+?)\\] at position\\[%,d\\]","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/file/SegmentFileBuilderV10.java","lineNumber":180,"sourceCode":"    this.outputFileName = outputFileName;\n    this.baseDir = baseDir;\n    this.maxContainerSize = maxContainerSize;\n    this.metadataCompression = metadataCompression;\n    this.externalSegmentFileBuilders = new TreeMap<>();\n  }\n\n  @Override\n  public void add(String name, File fileToAdd) throws IOException\n  {\n    try (FileInputStream fis = new FileInputStream(fileToAdd);\n         FileChannel src = fis.getChannel()) {\n      final long size = src.size();\n      try (SegmentFileChannel out = addWithChannel(name, size)) {\n        long position = 0;\n        while (position < size) {\n          final long transferred = src.transferTo(position, size - position, out);\n          if (transferred <= 0) {\n            throw new IOE(\"Unable to transfer bytes from file[%s] at position[%,d]\", fileToAdd, position);\n          }\n          position += transferred;\n        }\n      }\n    }\n  }\n\n  @Override\n  public void add(String name, ByteBuffer bufferToAdd) throws IOException\n  {\n    try (SegmentFileChannel out = addWithChannel(name, bufferToAdd.remaining())) {\n      out.write(bufferToAdd);\n    }\n  }\n\n  @Override\n  public SegmentFileChannel addWithChannel(final String name, final long size) throws IOException\n  {","sourceCodeStart":162,"sourceCodeEnd":198,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/file/SegmentFileBuilderV10.java#L162-L198","documentation":"SegmentFileBuilderV10.add copies an entire source file into the container by repeatedly calling FileChannel.transferTo. If transferTo returns zero or negative bytes at a given position, the builder cannot make progress and throws this IOException rather than looping forever. It signals the source channel yielded no bytes (typically because the source shrank or the channel is not readable at that offset).","triggerScenarios":"Merging delegated files (mergeDelegatedFiles -> add) where src.transferTo(position, size - position, out) returns <= 0, e.g. the source file was truncated or modified between size() and the transfer loop.","commonSituations":"Concurrent modification/deletion of source segment files during merge; a source file replaced with a shorter one mid-build; filesystem errors reporting success without transferring bytes.","solutions":["Check that source files are not being modified or deleted concurrently; snapshot inputs before merging","Verify the source file size with ls/stat matches the size the builder recorded (src.size()); if smaller, re-generate the source file","Retry the merge operation; if it reproduces, inspect filesystem health (disk errors, quotas)"],"exampleFix":"// before\nfinal long transferred = src.transferTo(position, size - position, out);\n// after (guard with re-check)\nif (src.size() != size) { throw new IOE(\"File[%s] changed size during transfer\", fileToAdd); }\nlong transferred = src.transferTo(position, size - position, out);","handlingStrategy":"retry","validationCode":"// Java: check source file size before merging\nlong onDisk = new File(fileToAdd).length();\nif (onDisk != src.size()) { throw new IOException(\"source size changed: \" + fileToAdd); }","typeGuard":null,"tryCatchPattern":"try {\n  builder.add(name, fileToAdd);\n} catch (IOException e) {\n  if (e.getMessage().startsWith(\"Unable to transfer bytes\")) {\n    // re-check source and retry once after re-verifying inputs\n    retryAdd(name, fileToAdd);\n  } else { throw e; }\n}","preventionTips":["Freeze/snapshot input files before merging so they cannot be modified concurrently","Avoid deleting source segments while a merge is in flight","Monitor filesystem health (errors, quotas) on segment directories"],"tags":["io","file-transfer","segment-build"],"backgroundTag":"file-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}