apache/druid · error · org.apache.druid.java.util.common.NotImplementedException

this druid.indexer.logs.type

Error message

this druid.indexer.logs.type [%s] does not support managing task payloads yet. You will have to switch to using environment variables

What it means

TaskPayloadManager.pushTaskPayload is a default interface method that throws NotImplementedException: the configured druid.indexer.logs.type implementation does not support persisting task payloads. Only implementations that override this method (e.g. ones backed by durable storage) can accept payload pushes.

Solutions

Switch druid.indexer.logs.type to an implementation that supports payload management (e.g. an implementation overriding pushTaskPayload)
If payload push is not needed, do not call the payload API and rely on task spec submitted via environment
Implement pushTaskPayload in a custom TaskLogs subclass

Example fix

// before
-Ddruid.indexer.logs.type=file
// after (payload-capable backend)
druid.indexer.logs.type=customPayloadTaskLogs

Defensive patterns

Strategy: fallback

Validate before calling

if (!(taskLogs instanceof TaskPayloadManagerWithPayloads)) {
  logger.warn("configured druid.indexer.logs.type does not support payload push");
}

Type guard

boolean supportsPayloads = taskLogs.getClass().getMethods().stream().noneMatch(m -> m.isDefault() && m.getName().equals("pushTaskPayload"));

Try / catch

try { taskPayloadManager.pushTaskPayload(taskId, file); } catch (NotImplementedException e) { // submit payload at task submission instead of pushing post-hoc }

Prevention

Verify the chosen druid.indexer.logs.type supports payload APIs before enabling payload features
Keep the same TaskLogs backend across overlord and middle managers
Include payload in the initial task spec when the backend is not payload-capable

When it happens

Trigger: Overlord receiving a POST /druid/indexer/v1/taskPayload/{taskId} (or internal payload push) when druid.indexer.logs.type is set to a legacy/simple implementation such as FileTaskLogs or NoopTaskLogs that never overrode pushTaskPayload.

Common situations: Clusters configured with older task-log backends (local file task logs) trying to use the newer task payload API; upgrading the overlord without switching druid.indexer.logs.type to a payload-capable implementation.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/08b7a58a041590b7. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/tasklogs/TaskPayloadManager.java:45

import java.io.File;
import java.io.IOException;
import java.io.InputStream;

/**
 * Something that knows how to push a task payload before it is run to somewhere
 * a ingestion worker will be able to stream the task payload from when trying to run the task.
 */
@ExtensionPoint
public interface TaskPayloadManager
{
  /**
   * Save payload so it can be retrieved later.
   *
   * @return inputStream for this taskPayload, if available
   */
  default void pushTaskPayload(String taskid, File taskPayloadFile) throws IOException
  {
    throw new NotImplementedException(StringUtils.format("this druid.indexer.logs.type [%s] does not support managing task payloads yet. You will have to switch to using environment variables", getClass()));
  }

  /**
   * Stream payload for a task.
   *
   * @return inputStream for this taskPayload, if available
   */
  default Optional<InputStream> streamTaskPayload(String taskid) throws IOException
  {
    throw new NotImplementedException(StringUtils.format("this druid.indexer.logs.type [%s] does not support managing task payloads yet. You will have to switch to using environment variables", getClass()));
  }
}

View on GitHub (pinned to 9b90983fd2)