{"record":{"id":"4cf4ccccfee7a80a","repo":"windmill-labs/windmill","slug":"no-s3-access-key-secret-key-is-configured-and-no-a","errorCode":null,"errorMessage":"no S3 access key/secret key is configured and no ambient AWS credentials could be loaded through the AWS SDK default chain (env vars, profile, ECS/EC2 instance role): {cause}. If an EC2/ECS instance role is expected to be used, the instance metadata service must be reachable from the process running Windmill — on EC2 the AWS Rust SDK only supports IMDSv2, so when Windmill runs in a Docker container the instance metadata hop limit (HttpPutResponseHopLimit) must be at least 2","messagePattern":"no S3 access key/secret key is configured and no ambient AWS credentials could be loaded through the AWS SDK default chain \\(env vars, profile, ECS/EC2 instance role\\): (.+?)\\. If an EC2/ECS instance role is expected to be used, the instance metadata service must be reachable from the process running Windmill — on EC2 the AWS Rust SDK only supports IMDSv2, so when Windmill runs in a Docker container the instance metadata hop limit \\(HttpPutResponseHopLimit\\) must be at least 2","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"backend/windmill-object-store/src/lib.rs","lineNumber":832,"sourceCode":"        }\n    }\n\n    async fn get(&self) -> anyhow::Result<aws_sdk_sts::config::Credentials> {\n        if let Some((creds, fetched_at)) = self.cached.read().await.as_ref() {\n            if Self::still_valid(creds, fetched_at.elapsed()) {\n                return Ok(creds.clone());\n            }\n        }\n        // The write lock is held across the chain resolution so concurrent requests don't all\n        // hit the metadata service at once.\n        let mut guard = self.cached.write().await;\n        if let Some((creds, fetched_at)) = guard.as_ref() {\n            if Self::still_valid(creds, fetched_at.elapsed()) {\n                return Ok(creds.clone());\n            }\n        }\n        let creds = self.chain.provide_credentials().await.map_err(|e| {\n            anyhow::anyhow!(\n                \"no S3 access key/secret key is configured and no ambient AWS credentials could \\\n                 be loaded through the AWS SDK default chain (env vars, profile, ECS/EC2 instance \\\n                 role): {cause}. If an EC2/ECS instance role is expected to be used, the instance \\\n                 metadata service must be reachable from the process running Windmill — on EC2 the \\\n                 AWS Rust SDK only supports IMDSv2, so when Windmill runs in a Docker container \\\n                 the instance metadata hop limit (HttpPutResponseHopLimit) must be at least 2\",\n                cause = format!(\"{:#}\", anyhow::Error::new(e))\n            )\n        })?;\n        *guard = Some((creds.clone(), std::time::Instant::now()));\n        Ok(creds)\n    }\n}\n\n#[cfg(feature = \"parquet\")]\nlazy_static::lazy_static! {\n    static ref AMBIENT_AWS_CREDS_PROVIDERS: Cache<String, Arc<AmbientAwsCredentials>> =\n        Cache::new(20);","sourceCodeStart":814,"sourceCodeEnd":850,"githubUrl":"https://github.com/windmill-labs/windmill/blob/e474e8803ce2ff5c2df09a58dab51d45f5c922ca/backend/windmill-object-store/src/lib.rs#L814-L850","documentation":"The S3 credential cache in windmill-object-store refreshed ambient AWS credentials via the SDK default chain (env vars, shared profile, ECS/EC2 instance role) and none could be provided. The message distinguishes 'no static access/secret key configured' from 'no ambient credentials resolvable' and includes IMDSv2/hop-limit guidance for Docker-on-EC2 setups.","triggerScenarios":"S3 resource accessed without an access key/secret key in the resource config while get() refreshes credentials and chain.provide_credentials() fails — no AWS_* env vars, no ~/.aws profile, no instance metadata reachable.","commonSituations":"Windmill running in Docker on EC2 with container hop limit 1 (default) blocking IMDSv2 calls; running outside AWS without any AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY; wrong AWS_PROFILE; IMDS unreachable due to security-group/network-policy rules; ECS task missing the task-role IAM permission.","solutions":["Set an access key/secret key explicitly on the S3 resource, or provide AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (and AWS_REGION) env vars to the Windmill containers","If relying on the EC2 instance role, raise the IMDS hop limit: aws ec2 modify-instance-metadata-options --http-put-response-hop-limit 2 (required for Docker containers using IMDSv2)","Verify the instance metadata service is reachable from the container: curl -s http://169.254.169.254/latest/api/token with the IMDSv2 PUT","On ECS, ensure the task definition has a task role and the metadata endpoint (169.254.170.2) is reachable","Check AWS_PROFILE and shared credentials file exist inside the container if using profiles"],"exampleFix":"// before: relying on ambient creds in Docker on EC2 (hop limit 1)\n# no action\n// after\naws ec2 modify-instance-metadata-options \\\n  --instance-id i-0123456789abcdef0 \\\n  --http-put-response-hop-limit 2 \\\n  --http-endpoint enabled\n# and/or set in windmill container env:\n# AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_REGION=us-east-1","handlingStrategy":"fallback","validationCode":"// Pre-flight: fail early if no ambient credentials are resolvable\nasync fn has_ambient_aws_creds(chain: &aws_config::default_provider::credentials::DefaultCredentialsChain) -> bool {\n    chain.provide_credentials().await.is_ok()\n}","typeGuard":"fn s3_resource_has_static_keys(r: &S3Resource) -> bool {\n    r.access_key.as_deref().map(|s| !s.is_empty()).unwrap_or(false)\n        && r.secret_key.as_deref().map(|s| !s.is_empty()).unwrap_or(false)\n}","tryCatchPattern":"match store.get(&path).await {\n    Ok(v) => v,\n    Err(e) if e.to_string().contains(\"no ambient AWS credentials\") => {\n        // surface actionable guidance: set static keys on the S3 resource or fix IMDS hop limit\n        return Err(e.context(\"configure S3 credentials on the resource or fix instance role access\"));\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Prefer explicit access/secret keys on the S3 resource in non-AWS environments","On EC2+Docker, set HttpPutResponseHopLimit >= 2 when creating instances","Set AWS_REGION explicitly — missing region often accompanies credential failures","Smoke-test S3 connectivity after deployment changes","Verify instance profiles/task roles are attached before scaling out"],"tags":["aws","s3","credentials","iam","docker","ec2"],"backgroundTag":"no-aws-credentials-found","analyzedSha":"e474e8803ce2ff5c2df09a58dab51d45f5c922ca","analyzedAt":"2026-09-03T12:38:19.024Z","contentChangedAt":"2026-09-03T12:38:19.024Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}