basecamp/kamal · error · RuntimeError

Could not read #{item_name} from Google Secret Manager

Error message

Could not read #{item_name} from Google Secret Manager

What it means

Raised inside the fetch loop of GcpSecretManager#fetch_secrets: for each requested secret it runs `gcloud secrets versions access <version> --secret=<name> --project=... [--account=...] [--impersonate-service-account=...] --format=json` (fetch_secret) and checks $?.success? afterwards. A non-zero exit for any single item (keyed project/secret_name) aborts the whole pull with this message. Note fetch_secret also JSON.parses the output, so some failures surface as JSON::ParserError before this raise.

Source

Thrown at lib/kamal/secrets/adapters/gcp_secret_manager.rb:36

      # - "default|my-service-user@example.com" will use the default user, and enable service account impersonation as my-service-user
      # - "default|my-service-user@example.com,another-service-user@example.com" same as above, but with an impersonation delegation chain

      unless logged_in?
        `gcloud auth login`
        raise RuntimeError, "could not login to gcloud" unless logged_in?
      end

      nil
    end

    def fetch_secrets(secrets, from:, account:, session:)
      user, service_account = parse_account(account)

      {}.tap do |results|
        secrets_with_metadata(prefixed_secrets(secrets, from: from)).each do |secret, (project, secret_name, secret_version)|
          item_name = "#{project}/#{secret_name}"
          results[item_name] = fetch_secret(project, secret_name, secret_version, user, service_account)
          raise RuntimeError, "Could not read #{item_name} from Google Secret Manager" unless $?.success?
        end
      end
    end

    def fetch_secret(project, secret_name, secret_version, user, service_account)
      secret = run_command(
        "secrets versions access #{secret_version.shellescape} --secret=#{secret_name.shellescape}",
        project: project,
        user: user,
        service_account: service_account
      )
      Base64.decode64(secret.dig("payload", "data"))
    end

    # The secret needs to at least contain a secret name, but project name, and secret version can also be specified.
    #
    # The string "default" can be used to refer to the default project configured for gcloud.
    #

View on GitHub (pinned to eee0083b38)

Solutions

1. Reproduce with the CLI to see the true error: `gcloud secrets versions access latest --secret=NAME --project=PROJ` (add --account/--impersonate-service-account as passed to kamal).
2. Grant IAM: `gcloud secrets add-iam-policy-binding NAME --member='serviceAccount:SA' --role='roles/secretmanager.secretAccessor'` for every identity in the chain (including impersonated ones).
3. Fix the name: confirm the secret exists via `gcloud secrets list --project=PROJ` and correct the prefix/name in your kamal secrets list.
4. If using impersonation, also grant roles/iam.serviceAccountTokenCreator on each target SA to the preceding identity.

Example fix

# before: CI service account lacks read on the secret
#   -> RuntimeError: Could not read myproj/RAILS_MASTER_KEY from Google Secret Manager

# after: grant Secret Accessor
gcloud secrets add-iam-policy-binding RAILS_MASTER_KEY \
  --project=myproj \
  --member=serviceAccount:deployer@myproj.iam.gserviceaccount.com \
  --role=roles/secretmanager.secretAccessor

require "open3"

def gcp_secret_readable?(project, name, account: "default")
  _out, _err, status = Open3.capture3("gcloud", "secrets", "versions", "access", "latest",
                                      "--secret=#{name}", "--project=#{project}", "--account=#{account}")
  status.success?
end

missing = wanted.reject { |p, n| gcp_secret_readable?(p, n) }
abort "Unreadable secrets (IAM or missing): #{missing.inspect}" if missing.any?

begin
  adapter.fetch(names, account: account)
rescue RuntimeError => e
  if (m = e.message.match(/Could not read (.+) from Google Secret Manager/))
    raise "GCP read failed for #{m[1]}: check name, project, and roles/secretmanager.secretAccessor for #{account}"
  end
  raise
end

When it happens

Trigger: adapter.fetch(...) after successful auth when: the secret name does not exist in the project; the version alias (e.g. 'latest' or a numeric version) is destroyed/never existed; the authenticated user or impersonated service account lacks secretmanager.versions.access (IAM); the project flag resolved from the secret prefix is wrong; or the impersonation chain (user|sa1,sa2) is misconfigured/unauthorized.

Common situations: deploy.yml referencing GCP secrets by a path whose project slug differs from the real project id; secret created in a different region/replica set; developer account works locally but CI service account lacks the Secret Manager Accessor role; impersonation chain where an intermediate SA dropped the token creator role.

Related errors

• #{error['SecretId']}: #{error['Message']}
• Could not read #{secrets} from Doppler
• could not login to gcloud
• gcloud CLI is not installed
• Could not read #{secrets} from LastPass

AI-assisted analysis of basecamp/kamal@eee0083b38 (2026-08-21). Data as JSON: /api/errors/a3d6fc96514ac63a. Report an issue: GitHub.