apache/druid · error · DruidException

The Apache Hadoop ingestion task was removed in Druid 37.0

Error message

The Apache Hadoop ingestion task was removed in Druid 37.0

What it means

Hadoop-based ingestion was removed in Druid 37.0; HadoopIndexTaskStub is a placeholder that throws on every lifecycle method to fail loudly. Any attempt to start (isReady) a Hadoop index task now throws 'The Apache Hadoop ingestion task was removed in Druid 37.0'. Existing Hadoop specs must be migrated to native batch or MSQ ingestion.

Solutions

Migrate ingestion to native batch (index_parallel) or SQL-based (MSQ) ingestion.
Stop submitting/retaining index_hadoop task specs; remove them from automation and periodic schedules.
If Hadoop data must be read, point native batch input at the HDFS/sequence files via the appropriate input source instead.

Example fix

// before
{"type": "index_hadoop", "spec": {...}}
// after
{"type": "index_parallel", "spec": {"ioConfig": {"type": "index_parallel", "inputSource": {"type": "hdfs", "paths": "/data"}, ...}}}

Defensive patterns

Strategy: validation

Validate before calling

if ("index_hadoop".equals(taskSpec.get("type"))) {
  throw new IllegalArgumentException("index_hadoop removed in Druid 37.0; migrate to index_parallel or MSQ");
}

Try / catch

try {
  overlord.submitTask(spec);
} catch (Exception e) {
  if (String.valueOf(e).contains("removed in Druid 37.0")) {
    // convert spec to native batch before resubmitting
  }
}

Prevention

Audit ingestion automation for index_hadoop before upgrading to 37.0
Migrate specs to native batch or SQL-based ingestion
Block submission of removed task types in CI/tooling

When it happens

Trigger: Submitting/resuming a task spec with type "index_hadoop" to a Druid 37.0+ cluster; the Overlord calls isReady(taskActionClient) on the stub and it throws.

Common situations: Upgrading a cluster to 37.0 while old Hadoop task specs or automated pipelines still submit index_hadoop tasks; archived spec replay; scripts that never migrated off Hadoop ingestion.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/76039d3cf638aa07. Report an issue: GitHub.

Appendix: source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/HadoopIndexTaskStub.java:97

  }

  @JsonProperty
  @Override
  public String getClasspathPrefix()
  {
    return classpathPrefix;
  }

  @Override
  public String getType()
  {
    return TYPE;
  }

  @Override
  public boolean isReady(TaskActionClient taskActionClient) throws Exception
  {
    throw noHadoop();
  }

  @Override
  public void stopGracefully(TaskConfig taskConfig)
  {
    throw noHadoop();
  }

  @Override
  public TaskStatus runTask(TaskToolbox toolbox)
  {
    throw noHadoop();
  }

  private static String getTheDataSource(Map<String, Object> spec)
  {
    final Map<String, Object> dataSchema = (Map<String, Object>) spec.get("dataSchema");
    if (dataSchema != null) {

View on GitHub (pinned to 9b90983fd2)