What causes "recommendations TOML parse error: {0}" in headroomlabs-ai/headroom?

Make the request from the same host via 127.0.0.1/localhost (e.g. docker exec into the container and curl 127.0.0.1)

headroomlabs-ai/headroom · warning · RecommendationsError

recommendations TOML parse error: {0}

Error message

recommendations TOML parse error: {0}

What it means

An HTTP 404 deliberately returned by the require_loopback FastAPI dependency (loopback_guard.py:203) when the TCP peer is not a loopback address. Debug/admin endpoints (/debug/*, /admin/*, /stats/reset, /v1/telemetry, etc.) are local-only, and 404 rather than 403 is chosen so external scanners cannot distinguish the routes from nonexistent ones.

Source

Thrown at crates/headroom-core/src/transforms/recommendations.rs:262

}

/// Errors surfaced by the loader. Marked non-exhaustive so we can add
/// future variants without breaking callers.
#[derive(Debug, thiserror::Error)]
#[non_exhaustive]
pub enum RecommendationsError {
    /// File doesn't exist on disk.
    #[error("recommendations file not found: {0}")]
    Missing(PathBuf),
    /// Filesystem error other than NotFound.
    #[error("recommendations IO error at {path}: {source}")]
    Io {
        path: PathBuf,
        #[source]
        source: std::io::Error,
    },
    /// TOML parse failure (typed wrapper for ergonomics).
    #[error("recommendations TOML parse error: {0}")]
    Parse(#[from] toml::de::Error),
}

#[cfg(test)]
mod tests {
    use super::*;

    fn sample_toml() -> &'static str {
        r#"
[[recommendation]]
auth_mode = "payg"
model_family = "claude-3-5"
structure_hash = "deadbeef"
skip_compression_recommended = true
strategy_hint = "smart_crusher"
confidence = 0.87
observations = 142

View on GitHub (pinned to 322425c43b)

Solutions

Make the request from the same host via 127.0.0.1/localhost (e.g. docker exec into the container and curl 127.0.0.1)
If remote access is genuinely required, front the endpoint with your own authenticated reverse proxy bound to loopback, or tunnel (ssh -L) to the host
Verify request.client.host is actually loopback — proxies using unix sockets or forwarded connections may present a non-loopback peer

Example fix

# before: from another machine
curl http://192.168.1.20:8080/debug/tasks

# after: on the host itself
curl http://127.0.0.1:8080/debug/tasks

Defensive patterns

Strategy: validation

Validate before calling

import ipaddress, socket
def is_loopback_peer() -> bool:
    try:
        return ipaddress.ip_address(socket.gethostbyname(socket.gethostname())).is_loopback
    except OSError:
        return False
# only target /debug,/admin,/v1/telemetry routes when running on the same host

Try / catch

if response.status_code == 404 and route_exists_in_docs:
    # guard rejection — run from the host instead
    ...

Prevention

Run admin/debug curls from the same host or via ssh -L tunnel
Remember 404 here is intentional camouflage, not a missing route

When it happens

Trigger: Curling http://<lan-ip>:<port>/debug/tasks from another machine, or from a container where the app is reached via a non-loopback docker network address; a reverse proxy forwarding with the original client IP preserved so request.client.host is the remote address.

Common situations: Accessing the admin dashboard through a docker port mapping from another host; putting the proxy behind an external ingress and expecting the debug endpoints to be reachable; health checks from a different pod.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/bc44a9d55182703a. Report an issue: GitHub.