{"record":{"id":"76039d3cf638aa07","repo":"apache/druid","slug":"the-apache-hadoop-ingestion-task-was-removed-in-dr","errorCode":null,"errorMessage":"The Apache Hadoop ingestion task was removed in Druid 37.0","messagePattern":"The Apache Hadoop ingestion task was removed in Druid 37\\.0","errorType":"exception","errorClass":"DruidException","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/HadoopIndexTaskStub.java","lineNumber":97,"sourceCode":"  }\n\n  @JsonProperty\n  @Override\n  public String getClasspathPrefix()\n  {\n    return classpathPrefix;\n  }\n\n  @Override\n  public String getType()\n  {\n    return TYPE;\n  }\n\n  @Override\n  public boolean isReady(TaskActionClient taskActionClient) throws Exception\n  {\n    throw noHadoop();\n  }\n\n  @Override\n  public void stopGracefully(TaskConfig taskConfig)\n  {\n    throw noHadoop();\n  }\n\n  @Override\n  public TaskStatus runTask(TaskToolbox toolbox)\n  {\n    throw noHadoop();\n  }\n\n  private static String getTheDataSource(Map<String, Object> spec)\n  {\n    final Map<String, Object> dataSchema = (Map<String, Object>) spec.get(\"dataSchema\");\n    if (dataSchema != null) {","sourceCodeStart":79,"sourceCodeEnd":115,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/HadoopIndexTaskStub.java#L79-L115","documentation":"Hadoop-based ingestion was removed in Druid 37.0; HadoopIndexTaskStub is a placeholder that throws on every lifecycle method to fail loudly. Any attempt to start (isReady) a Hadoop index task now throws 'The Apache Hadoop ingestion task was removed in Druid 37.0'. Existing Hadoop specs must be migrated to native batch or MSQ ingestion.","triggerScenarios":"Submitting/resuming a task spec with type \"index_hadoop\" to a Druid 37.0+ cluster; the Overlord calls isReady(taskActionClient) on the stub and it throws.","commonSituations":"Upgrading a cluster to 37.0 while old Hadoop task specs or automated pipelines still submit index_hadoop tasks; archived spec replay; scripts that never migrated off Hadoop ingestion.","solutions":["Migrate ingestion to native batch (index_parallel) or SQL-based (MSQ) ingestion.","Stop submitting/retaining index_hadoop task specs; remove them from automation and periodic schedules.","If Hadoop data must be read, point native batch input at the HDFS/sequence files via the appropriate input source instead."],"exampleFix":"// before\n{\"type\": \"index_hadoop\", \"spec\": {...}}\n// after\n{\"type\": \"index_parallel\", \"spec\": {\"ioConfig\": {\"type\": \"index_parallel\", \"inputSource\": {\"type\": \"hdfs\", \"paths\": \"/data\"}, ...}}}","handlingStrategy":"validation","validationCode":"if (\"index_hadoop\".equals(taskSpec.get(\"type\"))) {\n  throw new IllegalArgumentException(\"index_hadoop removed in Druid 37.0; migrate to index_parallel or MSQ\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  overlord.submitTask(spec);\n} catch (Exception e) {\n  if (String.valueOf(e).contains(\"removed in Druid 37.0\")) {\n    // convert spec to native batch before resubmitting\n  }\n}","preventionTips":["Audit ingestion automation for index_hadoop before upgrading to 37.0","Migrate specs to native batch or SQL-based ingestion","Block submission of removed task types in CI/tooling"],"tags":["hadoop","removed-feature","migration","deprecated-api-usage"],"backgroundTag":"deprecated-api-usage","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}