{"record":{"id":"7225f75e16c79164","repo":"hashicorp/terraform","slug":"retry-timeout-and-got-an-error-v","errorCode":null,"errorMessage":"retry timeout and got an error: %#v","messagePattern":"retry timeout and got an error: %#v","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/backend/remote-state/oss/backend.go","lineNumber":551,"sourceCode":"}\n\nfunc (a *Invoker) AddCatcher(catcher Catcher) {\n\ta.catchers = append(a.catchers, &catcher)\n}\n\nfunc (a *Invoker) Run(f func() error) error {\n\terr := f()\n\n\tif err == nil {\n\t\treturn nil\n\t}\n\n\tfor _, catcher := range a.catchers {\n\t\tif strings.Contains(err.Error(), catcher.Reason) {\n\t\t\tcatcher.RetryCount--\n\n\t\t\tif catcher.RetryCount <= 0 {\n\t\t\t\treturn fmt.Errorf(\"retry timeout and got an error: %#v\", err)\n\t\t\t} else {\n\t\t\t\ttime.Sleep(time.Duration(catcher.RetryWaitSeconds) * time.Second)\n\t\t\t\treturn a.Run(f)\n\t\t\t}\n\t\t}\n\t}\n\treturn err\n}\n\nvar providerConfig map[string]interface{}\n\nfunc getConfigFromProfile(d *schema.ResourceData, ProfileKey string) (interface{}, error) {\n\n\tif providerConfig == nil {\n\t\tif v, ok := d.GetOk(\"profile\"); !ok || v.(string) == \"\" {\n\t\t\treturn nil, nil\n\t\t}\n\t\tcurrent := d.Get(\"profile\").(string)","sourceCodeStart":533,"sourceCodeEnd":569,"githubUrl":"https://github.com/hashicorp/terraform/blob/d32a084675427f5ac3f7d2868578ef8b2c1dc525/internal/backend/remote-state/oss/backend.go#L533-L569","documentation":"Thrown by Invoker.Run() when a retryable error exhausts all configured retry attempts. The OSS backend registers two catchers: ClientErrorCatcher (matches 'AliyunGoClientFailure', 10 retries, 3s wait) and ServiceBusyCatcher (matches 'ServiceUnavailable', 10 retries, 3s wait). After 10 failed retries (~30s of sleep), the error is returned with %#v formatting of the last error.","triggerScenarios":"Invoker.Run(f) calls f(), which returns an error containing 'AliyunGoClientFailure' or 'ServiceUnavailable' in its message string. The catcher decrements RetryCount from 10, sleeping 3s between retries. When RetryCount reaches 0, this error is returned. The matcher uses strings.Contains, so the error message must contain the exact substring.","commonSituations":"Alibaba Cloud OSS service is genuinely down or degraded in the region. Client-side persistent failure that includes 'AliyunGoClientFailure' in the error (e.g. credential rotation taking longer than 30s). Network partition between the Terraform runner and OSS lasting more than ~30 seconds. RAM permission issue that the SDK reports as ServiceUnavailable.","solutions":["Check Alibaba Cloud status page for ongoing OSS or location service incidents in your region.","Verify credentials are valid and not expired — STS tokens may have rotated.","Check network connectivity and firewall rules to OSS endpoints.","If the error is transient, simply retry the Terraform operation.","For persistent 'AliyunGoClientFailure', upgrade the alibaba-cloud-sdk-go dependency as it may be an SDK-level issue."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Pre-check Alibaba Cloud service health before running Terraform\nfunc checkOSSServiceHealth(region string) error {\n    // Simple DNS + connectivity check\n    endpoint := fmt.Sprintf(\"oss-%s.aliyuncs.com\", region)\n    conn, err := net.DialTimeout(\"tcp\", endpoint+\":443\", 5*time.Second)\n    if err != nil {\n        return fmt.Errorf(\"cannot reach OSS in %s: %w\", region, err)\n    }\n    conn.Close()\n    return nil\n}","typeGuard":null,"tryCatchPattern":"// Wrap Invoker.Run to add outer retry for exhausted inner retries\nfunc runWithOuterRetry(invoker *Invoker, f func() error, outerRetries int) error {\n    var lastErr error\n    for i := 0; i < outerRetries; i++ {\n        err := invoker.Run(f)\n        if err == nil {\n            return nil\n        }\n        if strings.Contains(err.Error(), \"retry timeout\") {\n            lastErr = err\n            time.Sleep(10 * time.Second)\n            continue\n        }\n        return err // non-retryable error\n    }\n    return lastErr\n}","preventionTips":["Check the Alibaba Cloud health status page before large operations.","Verify credentials are valid for the entire duration of the operation (STS tokens may expire mid-run).","Ensure network stability between the Terraform runner and OSS endpoints.","For critical infrastructure, implement an outer retry wrapper around the backend operations."],"tags":["oss","retry","timeout","network","resilience"],"backgroundTag":null,"analyzedSha":"d32a084675427f5ac3f7d2868578ef8b2c1dc525","analyzedAt":"2026-08-11T18:43:52.779Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}