apache/flink · error · IllegalArgumentException

AWS region could not be determined. Set 's3.region' in Flink

Error message

AWS region could not be determined. Set 's3.region' in Flink configuration, AWS_REGION environment variable, or configure ~/.aws/config

What it means

resolveRegion() first honors an explicit s3.region / fs.s3.endpoint-region config, then falls back to the AWS SDK DefaultAwsRegionProviderChain (AWS_REGION/AWS_DEFAULT_REGION env vars, ~/.aws/config profile region, EC2/ECS metadata). If both the explicit config is unset and the provider chain throws, this IllegalArgumentException is raised listing the three remedies. The AWS SDK v2 client built here is region-mandatory, so no region means no client.

Source

Thrown at flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/S3ClientProvider.java:986

                    .refreshRequest(requestBuilder.build())
                    .build();
        }

        private Region resolveRegion(@Nullable String explicitRegion) {
            if (!StringUtils.isNullOrWhitespaceOnly(explicitRegion)) {
                LOG.info("Using configured AWS region: {}", explicitRegion);
                return Region.of(explicitRegion);
            }

            try {
                Region region = DefaultAwsRegionProviderChain.builder().build().getRegion();
                LOG.info("Auto-detected AWS region: {}", region.id());
                return region;
            } catch (Exception e) {
                // Fall through to error
            }

            throw new IllegalArgumentException(
                    "AWS region could not be determined. Set 's3.region' in Flink configuration, "
                            + "AWS_REGION environment variable, or configure ~/.aws/config");
        }
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Set the region explicitly in Flink config: s3.region: us-east-1 (or whatever your S3-compatible service expects; for MinIO any valid region string works, often us-east-1).
2. Alternatively export AWS_REGION=... for the JobManager and TaskManager processes (flink-conf.yaml via env.java.opts.all or the systemd/container env).
3. Or provide ~/.aws/config with [default] region=... on every cluster node — but prefer the Flink option for reproducibility.
4. For S3-compatible stores, also verify fs.s3.endpoint is set; region + endpoint together fully pin the client.

Example fix

# before (flink-conf.yaml, MinIO deployment)
fs.s3.endpoint: http://minio:9000
fs.s3.path.style.access: true

# after
fs.s3.endpoint: http://minio:9000
fs.s3.path.style.access: true
s3.region: us-east-1

// Fail fast with a precise message before S3 init
static void requireRegion(Configuration c) {
    String region = c.getString("s3.region", null);
    if (region == null && System.getenv("AWS_REGION") == null && System.getenv("AWS_DEFAULT_REGION") == null
            && !Files.isReadable(Path.of(System.getProperty("user.home"), ".aws/config"))) {
        throw new IllegalArgumentException("No AWS region source — set s3.region in flink-conf.yaml");
    }
}

When it happens

Trigger: No s3.region in flink-conf.yaml AND no AWS_REGION env on JobManager/TaskManager AND no ~/.aws/config on those hosts AND not running on EC2/ECS with IMDS reachable — then any S3 filesystem init (first S3 path touch) fails. Common in on-prem/minio deployments targeting a custom endpoint without setting the region option.

Common situations: Self-hosted MinIO/Ceph/ObjectStorage with fs.s3.endpoint configured but region forgotten (v2 SDK requires a region even for custom endpoints); bare-metal clusters without ~/.aws/config; containers that drop env vars; IMDS blocked by firewall so the provider chain throws instead of returning a region.

Related errors

• Class {} does not implement AwsCredentialsProvider
• Failed to instantiate credentials provider: {}
• fs.s3.aws.credentials.provider is set but contains no valid
• Unknown encryption type: {}. Supported values: none, sse-s3,
• Failed to put object for key: {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/c50518cca992b58d. Report an issue: GitHub.