hashicorp/terraform · error

refresh Ecs sts token err, fail to get AccessKeySecret: %s

Error message

refresh Ecs sts token err, fail to get AccessKeySecret: %s

What it means

Thrown by getAuthCredentialByEcsRoleName (backend.go:644) when the ECS instance-metadata STS response (from http://100.100.100.200/latest/meta-data/ram/security-credentials/<role>) was valid JSON with Code=="Success", but the JMESPath lookup for the AccessKeySecret field failed. This is the credential-refresh path used when the backend authenticates via ecs_role_name instead of static AK/SK. It means the metadata service returned a structurally unexpected payload that lacked the expected secret key.

Source

Thrown at internal/backend/remote-state/oss/backend.go:695

		return
	}
	code, err := jmespath.Search("Code", data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, fail to get Code: %s", err.Error())
		return
	}
	if code.(string) != "Success" {
		err = fmt.Errorf("refresh Ecs sts token err, Code is not Success")
		return
	}
	accessKeyId, err := jmespath.Search("AccessKeyId", data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, fail to get AccessKeyId: %s", err.Error())
		return
	}
	accessKeySecret, err := jmespath.Search("AccessKeySecret", data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, fail to get AccessKeySecret: %s", err.Error())
		return
	}
	securityToken, err := jmespath.Search("SecurityToken", data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, fail to get SecurityToken: %s", err.Error())
		return
	}

	if accessKeyId == nil || accessKeySecret == nil || securityToken == nil {
		err = fmt.Errorf("there is no any available accesskey, secret and security token for Ecs role %s", ecsRoleName)
		return
	}

	return accessKeyId.(string), accessKeySecret.(string), securityToken.(string), nil
}

func getHttpProxyUrl(rawUrl string) (*url.URL, error) {
	pc := httpproxy.FromEnvironment()

View on GitHub (pinned to c9def3e214)

Solutions

1. curl http://100.100.100.200/latest/meta-data/ram/security-credentials/<ROLE> from the ECS instance and confirm the JSON contains a non-empty AccessKeySecret.
2. Verify the RAM role named by ecs_role_name exists and has at least one trusted policy granting the instance the ability to assume it.
3. If you do not need ECS role auth, switch the backend to static access_key/secret_key or a named profile to bypass the metadata path entirely.
4. Restart/reattach the RAM role to the instance (ECS console -> Instance -> RAM Role) so metadata returns a complete credential set.

Example fix

// before (backend config, metadata path broken)
backend "oss" {
  bucket = "tf-state"
  ecs_role_name = "my-role"
}

// after: use explicit credentials / profile that do not depend on metadata shape
backend "oss" {
  bucket = "tf-state"
  profile = "prod"
}

// Before any terraform op using ecs_role_name, probe the metadata endpoint
// and assert the field is present, so you fail fast with your own message.
package main

import (
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"time"

	"github.com/jmespath/go-jmespath"
)

func checkSTS(role string) error {
	url := "http://100.100.100.200/latest/meta-data/ram/security-credentials/" + role
	c := &http.Client{Timeout: 3 * time.Second}
	resp, err := c.Get(url)
	if err != nil {
		return fmt.Errorf("metadata unreachable: %w", err)
	}
	defer resp.Body.Close()
	body, _ := io.ReadAll(resp.Body)
	var data interface{}
	if err := json.Unmarshal(body, &data); err != nil {
		return fmt.Errorf("metadata not JSON: %w", err)
	}
	if v, err := jmespath.Search("AccessKeySecret", data); err != nil || v == nil {
		return fmt.Errorf("AccessKeySecret missing in STS response")
	}
	return nil
}

// Wrap terraform invocation; treat this as a fatal config error, do not retry.
// In Go, if you call backend methods directly:
if _, _, _, err := getAuthCredentialByEcsRoleName(role); err != nil {
    if strings.Contains(err.Error(), "AccessKeySecret") {
        // stop and surface a config-fix prompt; never auto-retry metadata parse errors
    }
}

When it happens

Trigger: Calling terraform init/plan/apply with an OSS backend whose auth is ecs_role_name, on an ECS instance whose RAM role STS response omits or mis-names the AccessKeySecret field. The JMESPath Search("AccessKeySecret", data) returns a non-nil error (e.g. the value is an unexpected type or the field is absent in a way jmespath treats as an error).

Common situations: The RAM role attached to the ECS instance has no attached policy that grants STS credential issuance; the Alibaba metadata service is mid-deployment returning a partial response; an account-wide metadata schema change; running outside Alibaba Cloud against a mocked metadata endpoint that returns an incomplete body.

Related errors

• refresh Ecs sts token err, fail to get SecurityToken: %s
• there is no any available accesskey, secret and security tok
• error getting bucket: %#v
• state data in OSS does not have the expected content. This
• error getting bucket: %#v

AI-assisted analysis of hashicorp/terraform@c9def3e214 (2026-08-07). Data as JSON: /api/errors/9a428da8f66ba5c6. Report an issue: GitHub.