nautechsystems/nautilus_trader · error · anyhow::Error

CustomData JSON missing data_type

Error message

CustomData JSON missing data_type

What it means

Raised in `add_custom_data` when the parsed CustomData JSON has no `data_type` object, which the function needs to derive `type_name` and `metadata` columns for the `custom` table. It means the serialized CustomData is structurally incomplete — the `data_type` field is absent or not a JSON object.

Source

Thrown at crates/infrastructure/src/sql/queries.rs:1626

        .map_err(|e| anyhow::anyhow!("Failed to load signals: {e}"))
    }

    /// Inserts a `CustomData` entry via the provided `pool`.
    ///
    /// Serializes the model `CustomData` to full JSON and stores it in the JSONB `value` column.
    ///
    /// # Errors
    ///
    /// Returns an error if the SQL INSERT operation fails.
    pub async fn add_custom_data(pool: &PgPool, data: &CustomData) -> anyhow::Result<()> {
        let json_bytes = serde_json::to_vec(data)
            .map_err(|e| anyhow::anyhow!("CustomData must be valid JSON: {e}"))?;
        let value_json: serde_json::Value = serde_json::from_slice(&json_bytes)
            .map_err(|e| anyhow::anyhow!("CustomData value must be valid JSON: {e}"))?;
        let data_type_obj = value_json
            .get("data_type")
            .and_then(|v| v.as_object())
            .ok_or_else(|| anyhow::anyhow!("CustomData JSON missing data_type"))?;
        let data_type_name = data_type_obj
            .get("type_name")
            .and_then(|v| v.as_str())
            .unwrap_or("");
        let metadata_json = data_type_obj
            .get("metadata")
            .cloned()
            .unwrap_or_else(|| serde_json::Value::Object(serde_json::Map::new()));
        let identifier = data_type_obj
            .get("identifier")
            .and_then(|v| v.as_str())
            .unwrap_or("");
        sqlx::query(
            r#"
            INSERT INTO "custom" (data_type, metadata, identifier, value, ts_event, ts_init, created_at, updated_at)
            VALUES ($1, $2, $3, $4, $5, $6, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
            ON CONFLICT (id)
            DO UPDATE SET

View on GitHub (pinned to 18893faf8b)

Solutions

1. Construct CustomData through its proper constructor so `data_type` is always populated.
2. Check `data.data_type()` before calling add_custom_data; bail if None/empty.
3. Migrate legacy records to include `data_type` (type_name and metadata) before re-inserting.
4. Add a pre-insert validation that `serde_json::to_value(data)?.get("data_type").is_some()`.

Example fix

// before
add_custom_data(&pool, &data).await?;
// after
let v = serde_json::to_value(&data)?;
if v.get("data_type").and_then(|d| d.as_object()).is_none() {
    anyhow::bail!("CustomData missing data_type; refusing insert");
}
add_custom_data(&pool, &data).await?;

let v = serde_json::to_value(&data)?;
anyhow::ensure!(
    v.get("data_type").and_then(|d| d.as_object()).is_some(),
    "CustomData must include a data_type object with type_name"
);

fn has_data_type(v: &serde_json::Value) -> bool {
    v.get("data_type").and_then(|d| d.as_object()).is_some()
}

When it happens

Trigger: Calling `add_custom_data` with a CustomData whose `data_type` field is missing (constructed outside the library's constructors, deserialized from a truncated record, or an older schema without `data_type`).

Common situations: Hand-constructing CustomData without its data_type metadata; loading legacy rows saved by an older library version and re-saving them; serde default skipping an unset field.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• canonical backtest result must be a JSON object
• canonical result field '{key}' must be an array
• canonical result summary must be an object
• canonical run configuration ID must be a string or null
• canonical trader ID must be a string

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/5d3ca2f83cd162b8. Report an issue: GitHub.