{"record":{"id":"36c1ff84ab446457","repo":"Crosstalk-Solutions/project-nomad","slug":"recreated-container-readiness-reason","errorCode":null,"errorMessage":"recreated container ${readiness.reason}","messagePattern":"recreated container (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"critical","filePath":"admin/app/services/docker_service.ts","lineNumber":2231,"sourceCode":"        Labels: {\n          ...(containerConfig?.Labels ?? {}),\n          'com.docker.compose.project': 'project-nomad-managed',\n          'io.project-nomad.managed': 'true',\n        },\n        ...(containerConfig?.User && { User: containerConfig.User }),\n        HostConfig: containerConfig?.HostConfig ?? {},\n        ...(containerConfig?.ExposedPorts && { ExposedPorts: containerConfig.ExposedPorts }),\n        ...(recreateEnv.length ? { Env: recreateEnv } : {}),\n        ...(service.container_command ? { Cmd: service.container_command.split(' ') } : {}),\n        ...(process.env.NODE_ENV === 'production' && {\n          NetworkingConfig: { EndpointsConfig: { [DockerService.NOMAD_NETWORK]: {} } },\n        }),\n      })\n      await newContainer.start()\n\n      // Health gate before discarding the old container.\n      const readiness = await this._awaitContainerReady(newContainer)\n      if (!readiness.ready) throw new Error(`recreated container ${readiness.reason}`)\n\n      if (oldInfo) {\n        const oldRef = await this._findContainerByName(oldName)\n        if (oldRef) await this.docker.getContainer(oldRef.Id).remove({ force: true })\n      }\n      service.installed = true\n      service.installation_status = 'idle'\n      await service.save()\n      this.invalidateServicesStatusCache()\n      return { success: true, message: `Service ${serviceName} reconfigured successfully` }\n    } catch (error: any) {\n      logger.error({ err: error }, `[DockerService] recreateCustomAppContainer failed for ${serviceName}`)\n      // Roll back: discard the failed new container and restore the renamed original.\n      try {\n        const failedNew = await this._findContainerByName(serviceName)\n        if (failedNew) {\n          const c = this.docker.getContainer(failedNew.Id)\n          await c.stop({ t: 5 }).catch(() => {})","sourceCodeStart":2213,"sourceCodeEnd":2249,"githubUrl":"https://github.com/Crosstalk-Solutions/project-nomad/blob/0bd1c6f4f9888d577fe232de06ac144bb8337131/admin/app/services/docker_service.ts#L2213-L2249","documentation":"Thrown by DockerService after recreating a container when the new container fails a post-start readiness/health gate. The service starts a replacement container, waits for it to become healthy via _awaitContainerReady, and if readiness.ready is false it aborts the swap and throws, leaving the old container in place (the removal of the old container only happens after the gate passes).","triggerScenarios":"Calling the container recreate/update flow (e.g. app update or reinstall) where the new container starts but its health check fails, times out, or the app inside never becomes reachable; readiness.reason carries the underlying cause (health status, timeout, port not listening).","commonSituations":"Updated image that crashes on boot, missing env vars/migrations for the new version, healthcheck misconfigured or too short a timeout, port conflicts preventing the new container from binding, slow image startup exceeding the readiness wait window.","solutions":["Inspect readiness.reason in the thrown message to identify whether it's a timeout, unhealthy status, or connection refusal, then check `docker logs <new-container>` for the startup failure","Increase the readiness/healthcheck timeout or fix the app's healthcheck definition if the service is just slow to start","Verify the new image version is compatible with existing config/volumes (required env vars, DB migrations)","Check for port conflicts or leftover containers binding the same port","If the old container still runs, the system is safe; fix the root cause and re-trigger the recreate"],"exampleFix":"// before\nawait dockerService.recreateContainer('tipi') // throws: recreated container unhealthy\n\n// after\ntry {\n  await dockerService.recreateContainer('tipi')\n} catch (e) {\n  // old container is still running; diagnose the new one\n  console.error((e as Error).message) // 'recreated container <reason>'\n}","handlingStrategy":"try-catch","validationCode":"// Pre-check image healthcheck definition before recreate\nconst info = await docker.getContainer(name).inspect()\nif (!info.Config.Healthcheck) console.warn('No healthcheck defined; readiness gate may rely on port probe')","typeGuard":"const isReadinessError = (e: unknown) =>\n  (e as Error).message.startsWith('recreated container')","tryCatchPattern":"try {\n  await dockerService.recreateContainer(serviceId)\n} catch (e) {\n  if ((e as Error).message.startsWith('recreated container')) {\n    // old container still serves traffic — log reason, alert, retry later\n    logger.warn('Recreate failed, old container retained:', (e as Error).message)\n    return\n  }\n  throw e\n}","preventionTips":["Define a fast, accurate healthcheck in the service config so the readiness gate measures real readiness","Keep startup time under the readiness timeout (pre-warm caches in build, not runtime)","Test recreates on a staging host before production updates"],"tags":["docker","container-health","deployment","readiness-gate"],"backgroundTag":"container-health-check-failed","analyzedSha":"0bd1c6f4f9888d577fe232de06ac144bb8337131","analyzedAt":"2026-08-27T05:34:15.424Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}