hashicorp/terraform · error

refresh Ecs sts token err, json.Unmarshal fail

Error message

refresh Ecs sts token err, json.Unmarshal fail: %s

What it means

Thrown by getAuthCredentialByEcsRoleName() when json.Unmarshal() fails to parse the metadata service response body as JSON. The metadata service is expected to return a JSON object containing AccessKeyId, AccessKeySecret, SecurityToken, and Code fields. This error indicates the body was not valid JSON.

Solutions

  1. Inspect the raw response body — run 'curl -v http://100.100.100.200/latest/meta-data/ram/security-credentials/<role>' from the ECS instance.
  2. Check for proxies, DNS spoofing, or iptables rules redirecting metadata traffic.
  3. Verify the ECS instance metadata service is functioning correctly by querying other metadata paths.
  4. Fall back to static credentials or STS tokens if the metadata service is consistently returning non-JSON responses.
Defensive patterns

Strategy: try-catch

Validate before calling

// Validate metadata service returns valid JSON before parsing
func fetchAndValidateMetadataJSON(roleName string) (map[string]interface{}, error) {
    url := fmt.Sprintf("http://100.100.100.200/latest/meta-data/ram/security-credentials/%s", roleName)
    resp, err := http.Get(url)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }
    var data map[string]interface{}
    if err := json.Unmarshal(body, &data); err != nil {
        return nil, fmt.Errorf("metadata service returned non-JSON body (%d bytes): %w; raw: %s", len(body), err, string(body))
    }
    return data, nil
}

Try / catch

// Wrap credential refresh with JSON parse error handling and fallback
cred, err := getAuthCredentialByEcsRoleName(roleName)
if err != nil && strings.Contains(err.Error(), "json.Unmarshal fail") {
    // Metadata service returned non-JSON — fall back to environment credentials
    ak := os.Getenv("ALICLOUD_ACCESS_KEY")
    sk := os.Getenv("ALICLOUD_SECRET_KEY")
    if ak != "" && sk != "" {
        cred = fmt.Sprintf("%s:%s", ak, sk) // use env credentials
    }
}

Prevention

When it happens

Trigger: json.Unmarshal(response.GetHttpContentBytes(), &data) fails. The metadata service returned a 200 OK response but the body is not parseable JSON — e.g. HTML, plain text error, empty string, or malformed JSON with syntax errors.

Common situations: Metadata service returning a cached or proxy-injected HTML page instead of JSON. Response body truncated due to network issues. An intermediary (e.g. iptables redirect, transparent proxy) replacing the response. Alibaba Cloud metadata service returning a non-JSON error format for edge cases. Encoding issues (BOM, gzip without decompression).

Related errors

AI-assisted analysis of hashicorp/terraform@d32a084675 (2026-08-11). Data as JSON: /api/errors/e903c542bf8471df. Report an issue: GitHub.

Appendix: source

Thrown at internal/backend/remote-state/oss/backend.go:676

		err = fmt.Errorf("get Ecs sts token err : %s", err.Error())
		return
	}

	response := responses.NewCommonResponse()
	err = responses.Unmarshal(response, httpResponse, "")
	if err != nil {
		err = fmt.Errorf("unmarshal Ecs sts token response err : %s", err.Error())
		return
	}

	if response.GetHttpStatus() != http.StatusOK {
		err = fmt.Errorf("get Ecs sts token err, httpStatus: %d, message = %s", response.GetHttpStatus(), response.GetHttpContentString())
		return
	}
	var data interface{}
	err = json.Unmarshal(response.GetHttpContentBytes(), &data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, json.Unmarshal fail: %s", err.Error())
		return
	}
	code, err := jmespath.Search("Code", data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, fail to get Code: %s", err.Error())
		return
	}
	if code.(string) != "Success" {
		err = fmt.Errorf("refresh Ecs sts token err, Code is not Success")
		return
	}
	accessKeyId, err := jmespath.Search("AccessKeyId", data)
	if err != nil {
		err = fmt.Errorf("refresh Ecs sts token err, fail to get AccessKeyId: %s", err.Error())
		return
	}
	accessKeySecret, err := jmespath.Search("AccessKeySecret", data)
	if err != nil {

View on GitHub (pinned to d32a084675)