apache/dolphinscheduler · error · IllegalArgumentException

bucketName: ${bucketName} is not exists, you need to create

Error message

bucketName: ${bucketName} is not exists, you need to create them by yourself

What it means

exceptionWhenBucketNameNotExists validates that the configured bucket actually exists via s3Client.doesBucketExistV2 and throws IllegalArgumentException when the bucket name is set but no such bucket is reachable in the account/region. DolphinScheduler does not auto-create buckets; the user must create them.

Source

Thrown at dolphinscheduler-storage-plugin/dolphinscheduler-storage-s3/src/main/java/org/apache/dolphinscheduler/plugin/storage/s3/S3StorageOperator.java:216

        S3Object s3Object = s3Client.getObject(bucketName, filePath);
        try (
                InputStreamReader inputStreamReader = new InputStreamReader(s3Object.getObjectContent());
                BufferedReader bufferedReader = new BufferedReader(inputStreamReader)) {
            return bufferedReader.lines()
                    .skip(skipLineNums)
                    .limit(limit)
                    .collect(Collectors.toList());
        }
    }

    void exceptionWhenBucketNameNotExists(String bucketName) {
        if (StringUtils.isBlank(bucketName)) {
            throw new IllegalArgumentException("resource.aws.s3.bucket.name is blank");
        }

        boolean existsBucket = s3Client.doesBucketExistV2(bucketName);
        if (!existsBucket) {
            throw new IllegalArgumentException(
                    "bucketName: " + bucketName + " is not exists, you need to create them by yourself");
        }

        log.info("bucketName: {} has been found, the current regionName is {}", bucketName,
                s3Client.getRegionName());
    }

    @Override
    public List<StorageEntity> listStorageEntity(String resourceAbsolutePath) {
        final String s3ResourceAbsolutePath = transformAbsolutePathToS3Key(resourceAbsolutePath);
        ListObjectsV2Request listObjectsV2Request = new ListObjectsV2Request()
                .withBucketName(bucketName)
                .withDelimiter("/")
                .withPrefix(s3ResourceAbsolutePath);

        List<StorageEntity> storageEntities = new ArrayList<>();
        ListObjectsV2Result listObjectsV2Result;
        do {

View on GitHub (pinned to 02eac45a1b)

Solutions

Create the bucket manually (aws s3 mb s3://<bucket> or via console)
Fix typos and confirm the bucket name matches the region of the S3 client
Verify credentials/IAM policy grants s3:ListBucket so doesBucketExistV2 can see the bucket
Check region configuration (resource.aws.s3.region / endpoint) matches the bucket's region

Example fix

// before (bucket never created)
resource.aws.s3.bucket.name=nonexistent-bucket
// after
$ aws s3api create-bucket --bucket dolphinscheduler-resource --region us-east-1

Defensive patterns

Strategy: validation

Validate before calling

import com.amazonaws.services.s3.AmazonS3;
if (!s3.doesBucketExistV2(bucketName)) {
    throw new IllegalArgumentException("Bucket " + bucketName + " does not exist; create it first");
}

Try / catch

try {
    storageOperator.init();
} catch (IllegalArgumentException e) {
    log.error("Bucket validation failed: {}", e.getMessage()); // create bucket or fix name/region/creds
}

Prevention

Create the bucket before switching resource storage to S3
Match client region config to the bucket's region
Ensure IAM policy includes s3:ListBucket so existence checks succeed
Document bucket name in deployment configuration to avoid typos

When it happens

Trigger: S3StorageOperator construction/startup with a bucket name that does not exist in the target AWS account or region, or that is invisible due to wrong credentials/region/permissions.

Common situations: Typo in resource.aws.s3.bucket.name; bucket created in a different region than the client's configured region; IAM credentials lacking s3:ListBucket so existence check fails; bucket deleted after configuration.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/6b23fb020490fc1f. Report an issue: GitHub.