{"record":{"id":"933bbffc977a9dc5","repo":"apache/druid","slug":"unable-to-copy-key-s-to-file-s","errorCode":null,"errorMessage":"Unable to copy key [%s] to file [%s]","messagePattern":"Unable to copy key \\[(.+?)\\] to file \\[(.+?)\\]","errorType":"exception","errorClass":"SegmentLoadingException","httpStatus":null,"severity":"error","filePath":"extensions-contrib/cassandra-storage/src/main/java/org/apache/druid/storage/cassandra/CassandraDataSegmentPuller.java","lineNumber":82,"sourceCode":"\n    try {\n      RetryUtils.retry(\n          () -> {\n            try (OutputStream os = new FileOutputStream(tmpFile)) {\n              ChunkedStorage\n                  .newReader(indexStorage, key, os)\n                  .withBatchSize(BATCH_SIZE)\n                  .withConcurrencyLevel(CONCURRENCY)\n                  .call();\n            }\n            return new FileUtils.FileCopyResult(tmpFile);\n          },\n          Predicates.alwaysTrue(),\n          10\n      );\n    }\n    catch (Exception e) {\n      throw new SegmentLoadingException(e, \"Unable to copy key [%s] to file [%s]\", key, tmpFile.getAbsolutePath());\n    }\n    try {\n      final FileUtils.FileCopyResult result = CompressionUtils.unzip(tmpFile, outDir);\n      log.info(\n          \"Pull of file[%s] completed in %,d millis (%s bytes)\", key, System.currentTimeMillis() - startTime,\n          result.size()\n      );\n      return result;\n    }\n    catch (Exception e) {\n      try {\n        FileUtils.deleteDirectory(outDir);\n      }\n      catch (IOException e1) {\n        log.error(e1, \"Error clearing segment directory [%s]\", outDir.getAbsolutePath());\n        e.addSuppressed(e1);\n      }\n      throw new SegmentLoadingException(e, e.getMessage());","sourceCodeStart":64,"sourceCodeEnd":100,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-contrib/cassandra-storage/src/main/java/org/apache/druid/storage/cassandra/CassandraDataSegmentPuller.java#L64-L100","documentation":"After confirming equal lengths, mergeBloomFilterByteBuffers compares the header bytes (numHashFunctions and numBits fields before START_OF_SERIALIZED_LONGS) of both buffers byte-by-byte. Any difference in those header bytes means the filters use a different number of hash functions or bit layout, and merging would corrupt the result, so it throws this IllegalArgumentException. Note: this check compares every header byte, so even padding differences trip it.","triggerScenarios":"Calling mergeBloomFilterByteBuffers with two buffers of the same serialized length but different numHashFunctions/numBits header values — e.g. same numBits size in bytes but different hash function counts, or header bytes stored incorrectly.","commonSituations":"Filters built with identical bit sizes but different bloomFilterNumHashFunctions; endianness or version differences in how headers were written; merging buffers where one start offset points at the wrong byte, comparing misaligned headers.","solutions":["Standardize bloomFilterNumHashFunctions (and numBits) across all producers of filters that will be merged.","Deserialize both filters and verify getNumHashFunctions() matches before merging.","Double-check bf1Start/bf2Start offsets so header comparisons are aligned to actual filter starts.","Catch IllegalArgumentException and fall back to treating the merge as unsupported for those inputs."],"exampleFix":"// before\n// query A: bloomFilterNumHashFunctions=3, query B: bloomFilterNumHashFunctions=5\nBloomKFilter.mergeBloomFilterByteBuffers(bufA, bufB, 0, 0); // throws\n\n// after\n// use one shared config for all stages\nQuery a = base.clone().withOverriddenContext(ImmutableMap.of(\"bloomFilterNumHashFunctions\", 4, \"bloomFilterNumBits\", 1024));\nQuery b = base.clone().withOverriddenContext(ImmutableMap.of(\"bloomFilterNumHashFunctions\", 4, \"bloomFilterNumBits\", 1024));\n// now merge is safe","handlingStrategy":"validation","validationCode":"boolean sameHeader(ByteBuffer b1, int s1, ByteBuffer b2, int s2) {\n  for (int i = 0; i < BloomKFilter.START_OF_SERIALIZED_LONGS; i++) {\n    if (b1.get(s1 + i) != b2.get(s2 + i)) return false;\n  }\n  return true;\n}","typeGuard":"boolean isCompatibleMergeInput(ByteBuffer b1, int s1, ByteBuffer b2, int s2) {\n  return sameHeader(b1, s1, b2, s2) && serializedLen(b1, s1) == serializedLen(b2, s2);\n}","tryCatchPattern":"try {\n  BloomKFilter.mergeBloomFilterByteBuffers(bf1Buffer, bf2Buffer, start1, start2);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"NumHashFunctions/NumBits\")) {\n    throw new IllegalStateException(\"Filters use different hash-function counts; standardize bloomFilterNumHashFunctions\", e);\n  }\n  throw e;\n}","preventionTips":["Fix one bloomFilterNumHashFunctions value (with numBits) in shared config for every query producing mergeable filters.","Check header bytes equality before merging filters from external tools or other Druid versions.","Validate start offsets so header comparison is not misaligned."],"tags":["bloom-filter","argument-validation","aggregation","druid"],"backgroundTag":"invalid-argument-value","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}