{"record":{"id":"f8d917d4d72c634d","repo":"apache/druid","slug":"metadata-should-be-null-because-batchappenderatord","errorCode":null,"errorMessage":"Metadata should be null because BatchAppenderatorDriver never persists it","messagePattern":"Metadata should be null because BatchAppenderatorDriver never persists it","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"server/src/main/java/org/apache/druid/segment/realtime/appenderator/BatchAppenderatorDriver.java","lineNumber":93,"sourceCode":"\n  @Nullable\n  public Object startJob()\n  {\n    return startJob(AppenderatorDriverSegmentLockHelper.NOOP);\n  }\n\n  /**\n   * This method always returns null because batch ingestion doesn't support restoring tasks on failures.\n   *\n   * @return always null\n   */\n  @Override\n  @Nullable\n  public Object startJob(AppenderatorDriverSegmentLockHelper lockHelper)\n  {\n    final Object metadata = appenderator.startJob();\n    if (metadata != null) {\n      throw new ISE(\"Metadata should be null because BatchAppenderatorDriver never persists it\");\n    }\n    return null;\n  }\n\n  /**\n   * Add a row. Must not be called concurrently from multiple threads.\n   *\n   * @param row          the row to add\n   * @param sequenceName sequenceName for this row's segment\n   *\n   * @return {@link AppenderatorDriverAddResult}\n   *\n   * @throws IOException if there is an I/O error while allocating or writing to a segment\n   */\n  public AppenderatorDriverAddResult add(\n      InputRow row,\n      String sequenceName\n  ) throws IOException","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/segment/realtime/appenderator/BatchAppenderatorDriver.java#L75-L111","documentation":"BatchAppenderatorDriver never persists run metadata (its underlying appenderator's startJob() must therefore return null metadata). If metadata is non-null, an invariant of the batch driver has been violated — likely the appenderator was configured or subclassed to persist metadata — and this ISE is thrown to surface the inconsistency.","triggerScenarios":"Calling BatchAppenderatorDriver.startJob() when the wrapped appenderator's startJob() returns non-null metadata — e.g. a BatchAppenderator instance that loaded persisted metadata from a prior run, or using a non-batch Appenderator implementation with this driver.","commonSituations":"Mixing Appenderator implementations: wrapping a regular Appenderator (which persists metadata) in BatchAppenderatorDriver; stale persisted state from a misconfigured base persist directory being picked up at startJob.","solutions":["Ensure BatchAppenderatorDriver wraps a BatchAppenderator (created via Appenderators.createBatch), not the streaming Appenderator","Point the appenderator's base persist directory at a clean location free of prior jobs' persisted state","If stale state exists, remove the old persist directories for this data source before starting the job"],"exampleFix":"// before\nAppenderator appenderator = Appenderators.createAppendee(...); // streaming appenderator\nBatchAppenderatorDriver driver = new BatchAppenderatorDriver(appenderator, ...);\n// after\nAppenderator appenderator = Appenderators.createBatch(dataSchema, tuningConfig, ...);\nBatchAppenderatorDriver driver = new BatchAppenderatorDriver(appenderator, ...);","handlingStrategy":"try-catch","validationCode":"Object meta = appenderator.startJob();\nif (meta != null) {\n  throw new IllegalStateException(\"BatchAppenderatorDriver requires an appenderator that persists no metadata\");\n}","typeGuard":"boolean isBatchAppenderator(Appenderator a) {\n  return a instanceof BatchAppenderator;\n}","tryCatchPattern":"try {\n  driver.startJob(lockHelper);\n} catch (ISE e) {\n  if (e.getMessage().contains(\"Metadata should be null\")) {\n    cleanPersistDirAndRecreateDriver();\n  }\n  throw e;\n}","preventionTips":["Always pair BatchAppenderatorDriver with a BatchAppenderator (Appenderators.createBatch)","Start batch jobs against a clean base persist directory","Do not reuse an appenderator that previously persisted metadata with the batch driver"],"tags":["appenderator","batch-ingestion","invariant"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}