What causes "workflow scheduler graph run failed: {e}" in tinyhumansai/openhuman?

Read the GraphTracingSink event stream tagged workflow:{run_id} to find the exact node and step where the run aborted

workflow scheduler graph run failed: {e}

Error message

workflow scheduler graph run failed: {e}

What it means

Runtime failure of the compiled scheduler graph: graph.run(SchedulerState::default()) returned Err (graph.rs:251). Node bodies lift engine errors through graph_err — select_next_phase/execute_phase failures (ledger writes, spawn errors) become TinyAgentsError::Graph — and the RecursionPolicy backstop (max_visits_per_node = phase_count + 2, max_total_steps = (phase_count + 1) * 3 + 16) aborts a definition that never drains. A phase that merely fails is NOT this error: it persists Failed and routes to done.

Source

Thrown at src/openhuman/agent/orchestration/workflow_runs/graph.rs:251

                    &session,
                    &cancel,
                    model_override,
                    &phase,
                    total_spawned,
                )
                .await
            }
        }
    };

    let graph = build_scheduler_graph(definition.phases.len(), select, run)?.with_event_sink(
        Arc::new(GraphTracingSink::new(format!("workflow:{run_id_owned}"))),
    );

    graph
        .run(SchedulerState::default())
        .await
        .map_err(|e| anyhow!("workflow scheduler graph run failed: {e}"))?;
    Ok(())
}

/// Structure-only [`GraphTopology`] of the workflow scheduler graph for debug /
/// inspection (issue #4249, Phase 4). Built with no-op stub effects — the
/// topology exposes only node names, edges, and routing, never closure bodies.
pub(crate) fn scheduler_graph_topology() -> Result<GraphTopology> {
    let graph = build_scheduler_graph(
        1,
        || async { Ok(PhaseSelection::Terminated) },
        |_phase: WorkflowPhase, _spawned: u32| async { Ok(PhaseExecOutcome::Terminated) },
    )?;
    Ok(graph.topology())
}

View on GitHub (pinned to a221052e0d)

Solutions

Read the GraphTracingSink event stream tagged workflow:{run_id} to find the exact node and step where the run aborted
Validate the WorkflowDefinition before starting: unique phase ids, acyclic dependencies, every phase completable — a valid DAG always drains within the policy budget
If the underlying cause is a ledger/session store error, fix that first (disk space, sqlite lock, workspace permissions)
If the recursion policy tripped, recount visits: dispatch fires once per phase plus one terminal no-phase visit; anything more means the definition loops

Example fix

// before — self-dependency keeps dispatch busy until the backstop fires
{ "id": "build", "depends_on": ["build"] }

// after
{ "id": "build", "depends_on": ["plan"] }

Defensive patterns

Strategy: try-catch

Validate before calling

// Caller-side guard before starting a run: reject definitions that cannot drain
fn definition_is_dag(def: &serde_json::Value) -> bool {
    let Some(phases) = def.get("phases").and_then(|p| p.as_array()) else { return false };
    let ids: std::collections::HashSet<&str> = phases
        .iter().filter_map(|p| p.get("id").and_then(|v| v.as_str())).collect();
    ids.len() == phases.len() // unique ids; also keep depends_on acyclic
}

Try / catch

// Map scheduler failure onto the durable run status instead of propagating raw
if let Err(e) = drive_phases(cfg, def, run_id, cancel, session).await {
    tracing::error!(run_id, "workflow run failed: {e:#}");
    persist_run_failed(run_id, &e).await; // run shows Failed in status + UI
}

Prevention

Validate phase ids are unique and dependencies acyclic before start_workflow_run
Remember the policy budget: dispatch may fire only phase_count + 1 times — design definitions that provably drain
Watch the workflow:{run_id} tracing sink on first runs of a new definition shape

When it happens

Trigger: select_next_phase or execute_phase returning Err (run-ledger sqlite write failure, session error, spawn-budget enforcement erroring); a malformed WorkflowDefinition where dispatch keeps re-selecting phases until a node exceeds phase_count + 2 visits or the run exceeds (phase_count + 1) * 3 + 16 steps; run_phase reached with no selected phase.

Common situations: Hand-authored or LLM-generated workflow definitions with a dependency cycle that still 'progresses'; disk/lock problems on the run ledger; cancellation racing a phase transition; duplicate phase ids re-selecting the same phase forever.

Related errors

AI-assisted analysis of tinyhumansai/openhuman@a221052e0d (2026-08-16). Data as JSON: /api/errors/7150c06f8a354683. Report an issue: GitHub.