{"record":{"id":"6e1bf20ffb53b62a","repo":"mlflow/mlflow","slug":"mlflow-flush-failed-reason-attempt-attempt","errorCode":null,"errorMessage":"mlflow: flush failed (${reason}) attempt ${attempt}/${attempts}: ${String(err)}","messagePattern":"mlflow: flush failed \\((.+?)\\) attempt (.+?)/(.+?): (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"libs/typescript/integrations/openclaw/src/service.ts","lineNumber":421,"sourceCode":"    };\n\n    activeTraces.set(sessionKey, trace);\n    evictOldest(activeTraces, MAX_ACTIVE_TRACES);\n\n    return trace;\n  }\n\n  // Flush with exponential backoff retry\n  async function flushWithRetry(reason: string): Promise<void> {\n    const attempts = DEFAULT_FLUSH_RETRY_COUNT + 1;\n    for (let attempt = 1; attempt <= attempts; attempt++) {\n      try {\n        await flushTraces();\n        metrics.flushSuccesses += 1;\n        return;\n      } catch (err) {\n        metrics.flushFailures += 1;\n        log.warn(`mlflow: flush failed (${reason}) attempt ${attempt}/${attempts}: ${String(err)}`);\n        if (attempt >= attempts) {\n          return;\n        }\n        metrics.flushRetries += 1;\n        const delayMs = Math.min(\n          DEFAULT_FLUSH_RETRY_BASE_DELAY_MS * 2 ** (attempt - 1),\n          MAX_FLUSH_RETRY_DELAY_MS,\n        );\n        await sleep(delayMs);\n      }\n    }\n  }\n\n  // =====================================================================\n  // registerHooks — MUST be called during register(), not start().\n  // OpenClaw only accepts api.on() subscriptions during the register phase.\n  // Hooks guard on `initialized` so events before SDK init are silently skipped.\n  // =====================================================================","sourceCodeStart":403,"sourceCodeEnd":439,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/libs/typescript/integrations/openclaw/src/service.ts#L403-L439","documentation":"The OpenClaw integration's `flushWithRetry` attempts to flush pending traces to MLflow with bounded retries and exponential backoff; each failed attempt emits this structured warning including the flush reason, attempt count, and error. After exhausting attempts it gives up, so traces from that flush may be lost.","triggerScenarios":"Calling `stop()`, sweep intervals, or hook-triggered flushes when the MLflow tracking server is unreachable, returns errors (4xx/5xx), or artifact upload fails; retries continue until `attempts` is reached.","commonSituations":"MLflow server down or restarted during shutdown; wrong tracking URI; network timeouts in CI; auth token expired mid-session; flushing large trace payloads hitting request size limits.","solutions":["Check the embedded `String(err)` for the root cause (connection refused vs HTTP status)","Verify the tracking server is reachable and the tracking URI is correct","Increase retry attempts/backoff for shutdown-time flushes","Ensure `stop()` is awaited and the server is available before process exit"],"exampleFix":"// before\nservice.stop() // fire-and-forget flush can fail unobserved\n// after\nawait service.stop() // ensures flushWithRetry completes; inspect warnings if any","handlingStrategy":"retry","validationCode":"// health-check the tracking server before flush-heavy operations\nconst res = await fetch(`${trackingUri}/health`)\nif (!res.ok) throw new Error('tracking server unavailable; flush will retry and may fail')","typeGuard":"function isRetryable(err: unknown): boolean {\n  const s = String(err)\n  return /ECONNREFUSED|ETIMEDOUT|5\\d\\d/.test(s)\n}","tryCatchPattern":"try {\n  await flushWithRetry('stop')\n} catch (err) {\n  console.warn('flush ultimately failed after retries:', String(err))\n}","preventionTips":["Ensure the tracking server is up before stop/shutdown flushes","Await service.stop() and monitor flush warnings","Tune attempts and backoff for large trace volumes"],"tags":["typescript","network","retry","tracing"],"backgroundTag":"flush-retries-exhausted","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}