dbt-labs/dbt-core · error

RelationConfig should deserialize

Error message

RelationConfig should deserialize

What it means

Test panic from `RelationConfig::from_time_machine_json(&payload, &ctx).expect("RelationConfig should deserialize")` in serializable_impls.rs:462. RelationConfig is replayed from recorded JSON using a ReplayCallContext (here a Databricks context with a specific RelationType); deserialization fails when the recorded payload does not match what the context-dependent parser expects for that relation type. The time-machine layer throws this because a config payload it recorded can no longer be reconstructed, breaking replay.

Source

Thrown at crates/dbt-adapter/src/time_machine/serializable_impls.rs:462

            (
                RelationType::MaterializedView,
                serde_json::json!({
                    "partitioned_by": {"partition_by": ["event_date"]}
                }),
            ),
            (
                RelationType::StreamingTable,
                serde_json::json!({
                    "comment": {"comment": "streaming events", "persist": true},
                    "partitioned_by": {"partition_by": ["event_date"]}
                }),
            ),
        ];

        for (relation_type, payload) in cases {
            let ctx = databricks_ctx(relation_type);
            let original = RelationConfig::from_time_machine_json(&payload, &ctx)
                .expect("RelationConfig should deserialize");
            let recorded = serialize_object(&original).expect("RelationConfig should serialize");
            let restored = json_to_value_with_context(&recorded, &ctx);
            assert_eq!(
                serialize_object(&restored),
                Some(recorded),
                "RelationConfig should roundtrip for {relation_type:?}"
            );
        }
    }

    /// Reproduces the Databricks same-version replay break.
    ///
    /// `tbl_properties::to_jinja` emits TWO keys for the `tblproperties` component:
    /// the `tblproperties` sub-map (with `pipelines.pipelineId` filtered out) AND a
    /// separate top-level `pipeline_id`. But `component_from_recorded` only reads back
    /// `val.get("tblproperties")` — the `pipeline_id` is silently dropped on deserialize.
    ///
    /// For any DLT-backed relation (streaming tables, materialized views), the recorded

View on GitHub (pinned to 0267ce9170)

Solutions

1. Log the serde/parser error from from_time_machine_json along with the payload and relation type to find the mismatched component.
2. Check that every component key in relation_config_payload() has a registered parser for each tested RelationType in the Databricks context.
3. Update component_from_recorded parsing to cover the new/renamed component key.
4. Regenerate recordings with the current adapter version before replaying.

Example fix

// before
let original = RelationConfig::from_time_machine_json(&payload, &ctx)
    .expect("RelationConfig should deserialize");
// after
let original = RelationConfig::from_time_machine_json(&payload, &ctx)
    .unwrap_or_else(|e| panic!("deserialize failed for {relation_type:?}: {e}; payload={payload}"));

// before replay
if !payload.get("tblproperties").map_or(true, |v| v.is_object()) {
    return Err("malformed RelationConfig payload for relation type");
}

fn is_recorded_relation_config(j: &serde_json::Value) -> bool {
    j.is_object() && KNOWN_COMPONENT_KEYS.iter().any(|k| j.get(k).is_some())
}

match RelationConfig::from_time_machine_json(&payload, &ctx) {
    Ok(cfg) => cfg,
    Err(e) => { log::error("replay config decode failed ({relation_type:?}): {e}"); skip_replay_step() }
}

When it happens

Trigger: Calling `RelationConfig::from_time_machine_json` with a Databricks ctx whose RelationType (Table, View, MaterializedView, StreamingTable) has no matching component parser, or whose payload keys (column_comments, column_tags, comment, tags, tblproperties, partitioned_by) are malformed or unexpected for that type.

Common situations: Adding a new Databricks relation type or config component without updating the time-machine registry; changing the nested component JSON shape (e.g. {"set_tags": {...}}) so the parser no longer recognizes it; replaying recordings produced by another dbt version.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• RelationConfig should serialize
• DatabricksAdapter::list_relations_schemas_by_patterns
• not yet implemented
• should treat persisted view vs temp view as equivalent
• should treat qualified vs unqualified __dbt_tmp in MERGE USI

AI-assisted analysis of dbt-labs/dbt-core@0267ce9170 (2026-09-07). Data as JSON: /api/errors/ee5a9c7b08f0dd26. Report an issue: GitHub.