{"record":{"id":"f874f0f458cd437d","repo":"t8y2/dbx","slug":"etcd-history-failed","errorCode":"ETCD_HISTORY_FAILED","errorMessage":"ETCD_HISTORY_FAILED: ","messagePattern":"ETCD_HISTORY_FAILED: ","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"agents/drivers/etcd-go/history.go","lineNumber":201,"sourceCode":"\t_ = client.RequestProgress(progressCtx)\n\tprogressCancel()\n\tselect {\n\tcase <-completed:\n\tcase <-time.After(15 * time.Second):\n\t\treturn nil, errors.New(\"ETCD_HISTORY_TIMEOUT: history replay did not reach the requested revision\")\n\t}\n\n\tfailureMu.Lock()\n\thistoryFailure := failure\n\tfailureMu.Unlock()\n\tif historyFailure != nil {\n\t\tif errors.Is(historyFailure, rpctypes.ErrCompacted) {\n\t\t\tif revision, ok := compactedRevisionOf(client, key); ok {\n\t\t\t\treturn nil, errors.New(\"ETCD_COMPACTED: requested history was compacted at revision \" + longString(revision))\n\t\t\t}\n\t\t\treturn nil, errors.New(\"ETCD_COMPACTED: requested history was compacted\")\n\t\t}\n\t\treturn nil, errors.New(\"ETCD_HISTORY_FAILED: \" + historyFailure.Error())\n\t}\n\n\tcollector.mu.Lock()\n\trows := make([]map[string]any, len(collector.rows))\n\tcopy(rows, collector.rows)\n\ttruncated := collector.truncated\n\tcollector.mu.Unlock()\n\tsort.SliceStable(rows, func(i, j int) bool {\n\t\ta, _ := strconv.ParseInt(rows[i][\"revision\"].(string), 10, 64)\n\t\tb, _ := strconv.ParseInt(rows[j][\"revision\"].(string), 10, 64)\n\t\treturn a > b\n\t})\n\tserialized := make([]any, len(rows))\n\tfor i, row := range rows {\n\t\tserialized[i] = row\n\t}\n\treturn map[string]any{\n\t\t\"events\":           serialized,","sourceCodeStart":183,"sourceCodeEnd":219,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/etcd-go/history.go#L183-L219","documentation":"Generic wrapper for any history-walk failure that is not ErrCompacted. In the history operation, if the retry loop records a historyFailure, the driver returns 'ETCD_HISTORY_FAILED: <underlying error>' at history.go:201, propagating the raw etcd clientv3 error text after the prefix. It means the incremental revision walk over the key's revisions failed server-side or on the wire.","triggerScenarios":"Calling the history operation when an underlying Get/revision-walk fails with a non-compaction etcd error: e.g. ErrGRPCUnavailable (server down), context deadline exceeded (rpcTimeoutSeconds), ErrGRPCPermissionDenied, or a transient gRPC stream failure mid-walk.","commonSituations":"etcd node being restarted or unreachable under load; network partition between driver and cluster; RBAC user lacking read permission on the key; request exceeding the driver's per-RPC timeout on very long history walks.","solutions":["Read the text after the 'ETCD_HISTORY_FAILED: ' prefix to identify the underlying etcd error and apply that error's own remedy.","Retry the history call — transient Unavailable/deadline failures often clear once the etcd member recovers.","Check etcd cluster health (etcdctl endpoint health) and network connectivity to the member.","Verify the client's RBAC role permits reading the key range.","Increase the driver's rpc timeout or lower the history limit to shrink the walk."],"exampleFix":"// before\nconst rows = history(session, key); // fails opaque 'ETCD_HISTORY_FAILED: ...'\n// after\ntry { const rows = history(session, key, { limit: 50 }); }\ncatch (e) {\n  if (String(e).startsWith('ETCD_HISTORY_FAILED:')) retryWithBackoff();\n  else throw e;\n}","handlingStrategy":"retry","validationCode":"const healthy = await checkClusterHealth(); // etcdctl endpoint health equivalent\nif (!healthy) throw new Error('etcd cluster unhealthy; aborting history query');","typeGuard":"function isHistoryFailedError(e) { return String(e && e.message || e).startsWith('ETCD_HISTORY_FAILED:'); }","tryCatchPattern":"try { return history(session, key, { limit: 100 }); }\ncatch (e) {\n  if (isHistoryFailedError(e) && isTransient(String(e).split('ETCD_HISTORY_FAILED: ')[1]))\n    return retryWithBackoff(() => history(session, key));\n  throw e;\n}","preventionTips":["Monitor etcd member health and alert on Unavailable spikes.","Keep history limit modest (<= 500) so walks finish within the RPC timeout.","Verify RBAC read permissions for the keys you query.","Handle the underlying error embedded after the prefix rather than the wrapper generically."],"tags":["etcd","rpc","history","grpc"],"backgroundTag":"etcd-rpc-failure","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}