apache/druid · error · IllegalArgumentException

S3 requires the %s parameter if %s is set

Error message

S3 requires the %s parameter if %s is set

What it means

The ad-hoc S3 table function's paths parameter is a shorthand for (prefix = bucket + path), so it is only meaningful when the bucket parameter is also provided. Druid throws this IllegalArgumentException in convertArgsToSourceMap when paths is non-empty but bucket is missing. Without a bucket there is no way to resolve the relative paths.

Source

Thrown at extensions-core/s3-extensions/src/main/java/org/apache/druid/catalog/model/table/S3InputSourceDefn.java:207

        SECURITY_PARAMS
    );
  }

  @Override
  protected void convertArgsToSourceMap(Map<String, Object> jsonMap, Map<String, Object> args)
  {
    jsonMap.put(InputSource.TYPE_PROPERTY, S3StorageDruidModule.SCHEME);
    final List<String> uris = CatalogUtils.getStringArray(args, URIS_PARAMETER);
    final List<String> prefixes = CatalogUtils.getStringArray(args, PREFIXES_PARAMETER);
    final String bucket = CatalogUtils.getNonBlankString(args, BUCKET_PARAMETER);
    final List<String> paths = CatalogUtils.getStringArray(args, PATHS_PARAMETER);
    final String objectGlob = CatalogUtils.getNonBlankString(args, OBJECT_GLOB_PARAMETER);
    final boolean hasUris = uris != null;
    final boolean hasPrefixes = prefixes != null;
    final boolean hasBucket = bucket != null;
    final boolean hasPaths = !CollectionUtils.isNullOrEmpty(paths);
    if (hasPaths && !hasBucket) {
      throw new IAE(
          "S3 requires the %s parameter if %s is set",
          BUCKET_PARAMETER,
          PATHS_PARAMETER
      );
    }
    if ((hasUris && (hasPrefixes || hasBucket)) || (hasPrefixes && hasBucket)) {
      throw new IAE(
          "S3 accepts only one of %s, %s or %s",
          PATHS_PARAMETER,
          BUCKET_PARAMETER,
          PREFIXES_PARAMETER
      );
    }
    if (!hasUris && !hasPrefixes && !hasBucket) {
      throw new IAE(
          "S3 requires one of %s, %s or %s",
          PATHS_PARAMETER,
          BUCKET_PARAMETER,

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Add the bucket parameter alongside paths
2. Convert the paths into full s3:// URIs and pass them via the uris parameter instead
3. Remove the paths parameter if bucket-based prefixes are not intended

Example fix

// before
EXTERNAL(... paths => ARRAY['data/2024/'], ...) 
// after
EXTERNAL(... bucket => 'my-bucket', paths => ARRAY['data/2024/'], ...)

if (paths != null && !paths.isEmpty() && (bucket == null || bucket.isBlank())) {
  throw new IllegalArgumentException("bucket is required when paths is provided");
}

try {
  runExternalQuery(args);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("requires the")) { /* prompt for bucket and retry */ }
}

When it happens

Trigger: Calling the S3 EXTERNAL table function with the PATHS parameter set to a non-empty array while the BUCKET parameter is null or blank.

Common situations: Specifying relative object paths like data/year=2024/ but forgetting the bucket name; converting an old spec that embedded full URIs into a paths-based call and dropping bucket in the process.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• S3 accepts only one of %s, %s or %s
• Function requires a schema: TABLE(%s(...)) (<col> <type>...)
• Cannot translate sqlTypeName[%s] to Druid type for field[%s]
• Invalid parameter type:
• Unsupported query result format:

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/7d3718c9e74295b0. Report an issue: GitHub.