vectordotdev/vector · critical

all json-file keys should be matched

Error message

all json-file keys should be matched

What it means

kubernetes_logs auto-detects the container runtime log format; for Docker json-file lines the parser expects exactly the keys log, stream and time, matching each explicitly. Any other top-level key in the JSON line falls into _ => unreachable!('all json-file keys should be matched'), panicking on the first offending line and killing the log pipeline task.

Source

Thrown at src/sources/kubernetes_logs/parser/docker.rs:86

        Ok(JsonValue::Object(object)) => {
            for (key, value) in object {
                match key.as_str() {
                    MESSAGE_KEY => drop(log.insert(&target_path, value)),
                    STREAM_KEY => log_namespace.insert_source_metadata(
                        Config::NAME,
                        log,
                        Some(LegacyKey::Overwrite(path!(STREAM_KEY))),
                        path!(STREAM_KEY),
                        value,
                    ),
                    TIMESTAMP_KEY => log_namespace.insert_source_metadata(
                        Config::NAME,
                        log,
                        log_schema().timestamp_key().map(LegacyKey::Overwrite),
                        path!("timestamp"),
                        value,
                    ),
                    _ => unreachable!("all json-file keys should be matched"),
                };
            }
            Ok(())
        }
        Ok(_) => Err(ParsingError::NotAnObject { message: bytes }),
        Err(err) => Err(ParsingError::InvalidJson {
            source: err,
            message: bytes,
        }),
    }
}

const DOCKER_MESSAGE_SPLIT_THRESHOLD: usize = 16 * 1024; // 16 Kib

fn normalize_event(
    log: &mut LogEvent,
    log_namespace: LogNamespace,
) -> Result<(), NormalizationError> {

View on GitHub (pinned to 3708c39b12)

Solutions

1. Remove the log-opts that add keys: drop labels/env/tag from daemon.json or the container's log-opt and restart containers so lines are re-emitted in plain json-file shape
2. Verify the emitted shape: head a /var/log/pods/<ns>_<pod>_<uid>/<container>/*.log file and confirm only log/stream/time keys
3. If enrichment keys are required, read those logs with the file source plus a VRL remap instead of kubernetes_logs
4. Upgrade Vector and check release notes - later versions tolerate unknown json-file keys

Example fix

# /etc/docker/daemon.json -- before
{ "log-driver": "json-file", "log-opts": { "labels": "production" } }

# after
{ "log-driver": "json-file" }

# Inspect real log lines before pointing kubernetes_logs at them:
head -n 5 /var/log/pods/<ns>_<pod>_<uid>/<container>/*.log
# json-file lines must contain ONLY log/stream/time keys; then check the daemon:
docker info --format '{{.LoggingDriver}}'; cat /etc/docker/daemon.json
# any log-opts adding keys (labels/env/tag) must be removed

fn is_plain_docker_json(v: &serde_json::Value) -> bool {
    v.as_object()
        .map(|o| o.iter().all(|(k, _)| matches!(k.as_str(), "log" | "stream" | "time")))
        .unwrap_or(false)
}

When it happens

Trigger: Docker daemons configured with --log-opt labels=..., env=... or tag=... - these options inject extra top-level keys into every json-file log line; any runtime or sidecar enriching json-file lines with additional fields triggers the same arm.

Common situations: Nodes where dockerd has log-opts set globally (/etc/docker/daemon.json); per-container log-opt overrides; Docker versions or logging plugins adding new fields to the standard shape read from /var/log/pods.

Related errors

• path and query should never fail to parse
• Serializer does not support JSON
• Paths must always start with a leading forward slash (`/`).
• Only leaf nodes should be allowed to be non-object values.
• Reader encountered unrecoverable error: {e:?}

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/f3778455414a841f. Report an issue: GitHub.