What causes "bucketName: is not exists, you need to create them by yourself" in apache/dolphinscheduler?

Create the bucket yourself (aws s3 mb s3:// in the target region) — DolphinScheduler requires it to pre-exist

apache/dolphinscheduler · error · IllegalArgumentException

bucketName: <bucketName> is not exists, you need to create t

Error message

bucketName: <bucketName> is not exists, you need to create them by yourself

What it means

When the S3 bucket name is non-blank, S3RemoteLogHandler.checkBucketNameExists calls s3Client.doesBucketExistV2(bucketName). If AWS S3 reports the bucket does not exist (or is inaccessible to the credentials, since head-bucket style checks also fail on permission errors), this IllegalArgumentException is thrown.

Source

Thrown at dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/log/remote/S3RemoteLogHandler.java:115

                }
            }
        } catch (Exception e) {
            log.error("error while getting remote log on S3 {} to {}", objectName, logPath, e);
        }
    }

    protected String readBucketName() {
        return PropertyUtils.getString(Constants.AWS_S3_BUCKET_NAME);
    }

    public void checkBucketNameExists(String bucketName) {
        if (StringUtils.isBlank(bucketName)) {
            throw new IllegalArgumentException("remote.logging.s3.bucket.name is blank");
        }

        boolean existsBucket = s3Client.doesBucketExistV2(bucketName);
        if (!existsBucket) {
            throw new IllegalArgumentException(
                    "bucketName: " + bucketName + " is not exists, you need to create them by yourself");
        }

        log.info("bucketName: {} has been found, the current regionName is {}", bucketName,
                s3Client.getRegionName());
    }
}

View on GitHub (pinned to 02eac45a1b)

Solutions

Create the bucket yourself (aws s3 mb s3://<bucket> in the target region) — DolphinScheduler requires it to pre-exist
Correct remote.logging.s3.bucket.name typos in common.properties
Verify AWS credentials/IAM policy grant s3:HeadBucket or s3:ListBucket on the bucket
Set the S3 client region (remote.logging.s3.region or default AWS region chain) to the bucket's actual region

Example fix

// before
remote.logging.s3.bucket.name=my-logs  // bucket doesn't exist in account

// after
$ aws s3 mb s3://my-logs --region us-east-1
$ aws s3api head-bucket --bucket my-logs  # verify access
remote.logging.s3.bucket.name=my-logs

Defensive patterns

Strategy: validation

Validate before calling

// verify bucket existence/permission with the AWS SDK before enabling
boolean exists = s3Client.doesBucketExistV2(bucketName);
if (!exists) {
    throw new IllegalStateException("S3 bucket " + bucketName + " not found or not accessible; run: aws s3api head-bucket --bucket " + bucketName);
}

Try / catch

try {
    s3RemoteLogHandler.checkBucketNameExists(bucketName);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("is not exists")) {
        log.error("Bucket {} missing/inaccessible: create it and check IAM + region", bucketName);
    }
}

Prevention

Provision buckets in the same region the S3 client is configured for
Grant the worker's IAM role s3:HeadBucket/s3:ListBucket on the log bucket
Run aws s3api head-bucket as a deploy smoke test
Manage bucket names centrally to avoid typos across environments

When it happens

Trigger: checkBucketNameExists is called with a valid non-blank name that doesBucketExistV2 cannot find: bucket never created, wrong region (client resolves region from bucket location), name typo, or credentials lacking s3:ListBucket/s3:HeadBucket on the bucket.

Common situations: Using a bucket name that exists in another AWS account; cross-region access where the S3 client's configured region differs; IAM policies restricting bucket visibility; renamed/deleted buckets in shared environments.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/501d6c1e6f5722fd. Report an issue: GitHub.