conductor-oss/conductor · error · NonTransientException

Unable to download payload from external storage path: %s

Error message

Unable to download payload from external storage path: %s

What it means

Thrown by ExternalPayloadStorageUtils.downloadPayload when downloading and deserializing a payload from external storage fails for any non-transient reason — e.g. the object doesn't exist at the path, the downloaded content is not valid JSON, or the storage backend returns an error. TransientException is re-thrown as-is; everything else is wrapped in NonTransientException, meaning retries will not help.

Source

Thrown at core/src/main/java/com/netflix/conductor/core/utils/ExternalPayloadStorageUtils.java:76

    }

    /**
     * Download the payload from the given path.
     *
     * @param path the relative path of the payload in the {@link ExternalPayloadStorage}
     * @return the payload object
     * @throws NonTransientException in case of JSON parsing errors or download errors
     */
    @SuppressWarnings("unchecked")
    public Map<String, Object> downloadPayload(String path) {
        try (InputStream inputStream = externalPayloadStorage.download(path)) {
            return objectMapper.readValue(
                    IOUtils.toString(inputStream, StandardCharsets.UTF_8), Map.class);
        } catch (TransientException te) {
            throw te;
        } catch (Exception e) {
            LOGGER.error("Unable to download payload from external storage path: {}", path, e);
            throw new NonTransientException(
                    "Unable to download payload from external storage path: " + path, e);
        }
    }

    /**
     * Verify the payload size and upload to external storage if necessary.
     *
     * @param entity the task or workflow for which the payload is to be verified and uploaded
     * @param payloadType the {@link PayloadType} of the payload
     * @param <T> {@link TaskModel} or {@link WorkflowModel}
     * @throws NonTransientException in case of JSON parsing errors or upload errors
     * @throws TerminateWorkflowException if the payload size is bigger than permissible limit as
     *     per {@link ConductorProperties}
     */
    public <T> void verifyAndUpload(T entity, PayloadType payloadType) {
        if (!shouldUpload(entity, payloadType)) return;

        long threshold = 0L;

View on GitHub (pinned to cf7c3e4a8a)

Solutions

1. Verify the external storage path exists and is accessible — check the S3/GCP/Azure bucket for the object at the given path.
2. Check storage backend credentials and permissions in the Conductor configuration (access key, secret, IAM role).
3. If the object was deleted by TTL or lifecycle policy, increase the retention period or re-run the workflow to regenerate the payload.
4. Inspect the server log for the underlying cause exception (logged at LOGGER.error with the path) to determine if it's a permissions issue, missing object, or JSON parse error.
5. If the payload is permanently lost, terminate and re-run the affected workflow instance.

// Verify external storage path exists before downloading (if backend supports checks)
// This is backend-specific; for S3 you would use headObject
// Generic pre-check:
if (path == null || path.isEmpty()) {
    throw new IllegalArgumentException("External payload path is null or empty");
}

try {
    Map<String, Object> payload = externalPayloadStorageUtils.downloadPayload(path);
} catch (NonTransientException e) {
    LOGGER.error("Payload at {} is missing or corrupt — workflow may need to be re-run", path, e);
    // Mark the task/workflow as failed or trigger re-execution
    throw e;
}

When it happens

Trigger: Calling downloadPayload with a path that does not exist in external storage, points to corrupted/truncated JSON, or triggers a storage backend error (e.g. S3 403 permission denied). Occurs when the decider loads a task or workflow whose input/output payload was externalized to external storage and the external object is missing or corrupt.

Common situations: External storage bucket/object was deleted or expired (TTL). Permissions changed on the storage backend (S3 IAM policy revoked). The payload path is stale after a storage migration. JSON corruption from a partial write or encoding mismatch. External storage misconfigured (wrong endpoint, credentials expired).

Related errors

• Unable to upload payload to external storage for workflow: %
• Unable to validate variables payload size of workflow: %s
• PLAN_EXECUTE '${config.getName()}': tool '${t.getName()}' fa
• Error parsing the json schema : {message}
• A2A stream to {endpoint} failed: {message}

AI-assisted analysis of conductor-oss/conductor@cf7c3e4a8a (2026-08-14). Data as JSON: /api/errors/932acf63b32ae910. Report an issue: GitHub.