hashicorp/terraform · error

refresh Ecs sts token err, json.Unmarshal fail: %s

Error message

refresh Ecs sts token err, json.Unmarshal fail: %s

What it means

Returned by getAuthCredentialByEcsRoleName when json.Unmarshal of the metadata response body into interface{} fails. The metadata service returned a 200 but the body is not valid JSON, so the credential payload cannot be parsed.

Source

Thrown at internal/backend/remote-state/oss/backend.go:676

		err = fmt.Errorf("get Ecs sts token err : %s", err.Error())
		return
	}

	response := responses.NewCommonResponse()
	err = responses.Unmarshal(response, httpResponse, "")
	if err != nil {
		err = fmt.Errorf("unmarshal Ecs sts token response err : %s", err.Error())
		return
	}

	if response.GetHttpStatus() != http.StatusOK {
		err = fmt.Errorf("get Ecs sts token err, httpStatus: %d, message = %s", response.GetHttpStatus(), response.GetHttpContentString())
		return
	}
	var data interface{}
	err = json.Unmarshal(response.GetHttpContentBytes(), &data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, json.Unmarshal fail: %s", err.Error())
		return
	}
	code, err := jmespath.Search("Code", data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, fail to get Code: %s", err.Error())
		return
	}
	if code.(string) != "Success" {
		err = fmt.Errorf("refresh Ecs sts token err, Code is not Success")
		return
	}
	accessKeyId, err := jmespath.Search("AccessKeyId", data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, fail to get AccessKeyId: %s", err.Error())
		return
	}
	accessKeySecret, err := jmespath.Search("AccessKeySecret", data)
	if err != nil {

View on GitHub (pinned to c9def3e214)

Solutions

1. curl the metadata URL directly and confirm the body is JSON like {"AccessKeyId":...,"AccessKeySecret":...,"SecurityToken":...,"Code":"Success"}.
2. Disable any HTTP proxy for 100.100.100.200 (set NO_PROXY to include it).
3. Retry for transient truncation; if persistent, fall back to static/STS credentials.
4. Ensure ecs_role_name is attached and returns proper JSON (see error 335).

Example fix

# before: corp proxy returns HTML 200 for metadata
HTTP_PROXY=http://corp-proxy:8080

# after: bypass proxy for metadata host
NO_PROXY=100.100.100.200
HTTP_PROXY=http://corp-proxy:8080

func metadataIsJSON(role string) error {
    u := "http://100.100.100.200/latest/meta-data/ram/security-credentials/" + role
    resp, err := http.Get(u)
    if err != nil { return err }
    defer resp.Body.Close()
    b, _ := io.ReadAll(resp.Body)
    var tmp interface{}
    if err := json.Unmarshal(b, &tmp); err != nil {
        return fmt.Errorf("metadata body is not JSON: %s", string(b))
    }
    return nil
}

if _, err := getAuthCredentialByEcsRoleName(role); err != nil {
    if strings.Contains(err.Error(), "json.Unmarshal fail") {
        // likely a proxy injecting HTML; retry without proxy
        return getAuthCredentialByEcsRoleName(role)
    }
    return err
}

When it happens

Trigger: The metadata service returns HTML/text (e.g. an error page from an intercepting proxy), an empty body, or truncated JSON. Because the status check (error 335) passed first, this is specifically a content-vs-status mismatch.

Common situations: A captive/transparent proxy returning an HTML block page with a 200 status; metadata service returning a plain-text error string; SDK/transport truncating the body.

Related errors

• get Ecs sts token err : %s
• unmarshal Ecs sts token response err : %s
• build sts requests err: %s
• get Ecs sts token err, httpStatus: %d, message = %s
• refresh Ecs sts token err, fail to get Code: %s

AI-assisted analysis of hashicorp/terraform@c9def3e214 (2026-08-07). Data as JSON: /api/errors/e903c542bf8471df. Report an issue: GitHub.