basecamp/kamal · error · RuntimeError

Could not read #{secrets} from AWS Secrets Manager

Error message

Could not read #{secrets} from AWS Secrets Manager

What it means

The AWS Secrets Manager adapter shells out to `aws secretsmanager batch-get-secret-value --secret-id-list ... [--profile X] --output json`; if that CLI process exits non-zero, Kamal raises RuntimeError. Note the message interpolates the raw CLI output, not the secret names, because the `tap do |secrets|` block variable shadows the secrets array — so the message shows whatever aws printed (often empty, since stderr is not captured).

Source

Thrown at lib/kamal/secrets/adapters/aws_secrets_manager.rb:37

              results["#{secret_name}/#{key}"] = stringify_secret_value(value)
            end
          else
            results["#{secret_name}"] = stringify_secret_value(secret_string)
          end
        rescue JSON::ParserError
          results["#{secret_name}"] = secret["SecretString"]
        end
      end
    end

    def get_from_secrets_manager(secrets, account: nil)
      args = [ "aws", "secretsmanager", "batch-get-secret-value", "--secret-id-list" ] + secrets.map(&:shellescape)
      args += [ "--profile", account.shellescape ] if account
      args += [ "--output", "json" ]
      cmd = args.join(" ")

      `#{cmd}`.tap do |secrets|
        raise RuntimeError, "Could not read #{secrets} from AWS Secrets Manager" unless $?.success?

        secrets = JSON.parse(secrets)

        return secrets["SecretValues"] unless secrets["Errors"].present?

        raise RuntimeError, secrets["Errors"].map { |error| "#{error['SecretId']}: #{error['Message']}" }.join(" ")
      end
    end

    def stringify_secret_value(value)
      value.is_a?(String) ? value : JSON.dump(value)
    end

    def check_dependencies!
      raise RuntimeError, "AWS CLI is not installed" unless cli_installed?
    end

    def cli_installed?

View on GitHub (pinned to eee0083b38)

Solutions

1. Reproduce manually with the exact command from the source: aws secretsmanager batch-get-secret-value --secret-id-list <name> --output json (add --profile if used) and read the stderr
2. Fix credentials: aws sso login (or refresh keys), and verify with aws sts get-caller-identity using the same profile
3. Set the region explicitly, e.g. export AWS_REGION=us-east-1, if the CLI has no default region
4. Upgrade to a current AWS CLI v2 (batch-get-secret-value is not in old v2 releases/v1)

Example fix

# before (failing)
kamal secrets fetch -a aws_secrets_manager --account prod RAILS_MASTER_KEY

# debug + fix (terminal)
aws sts get-caller-identity --profile prod      # verify creds
aws sso login --profile prod                     # refresh if expired
aws secretsmanager batch-get-secret-value --secret-id-list RAILS_MASTER_KEY --profile prod --output json

# Verify AWS auth/CLI health before running kamal
system("aws --version > /dev/null 2>&1") or abort "aws CLI missing"
system("aws sts get-caller-identity #{ENV["AWS_PROFILE"] ? "--profile #{ENV["AWS_PROFILE"]}" : ""} > /dev/null") or abort "AWS credentials invalid/expired"

begin
  results = adapter.fetch(secret_names, account: "prod")
rescue RuntimeError => e
  warn "aws CLI failure — re-run manually: aws secretsmanager batch-get-secret-value --secret-id-list #{secret_names.join(' ')} --profile prod --output json"
  warn e.message
  exit 1
end

When it happens

Trigger: Expired SSO token or missing/invalid AWS credentials; wrong profile name passed via --account; awscli too old to have batch-get-secret-value; no network reachability to secretsmanager.<region>.amazonaws.com; the AWS_REGION/region not configured so the CLI errors out.

Common situations: CI job whose aws SSO session expired overnight; profile typo between deploy scripts; fresh machine with awscli v1 installed; corporate proxy or VPC without an secretsmanager endpoint.

Related errors

• AWS CLI is not installed
• Unknown secrets adapter: #{name}
• #{error['SecretId']}: #{error['Message']}
• Missing required option '--account'
• Failed to login to and unlock Bitwarden

AI-assisted analysis of basecamp/kamal@eee0083b38 (2026-08-21). Data as JSON: /api/errors/eab78a1603d71807. Report an issue: GitHub.