nautechsystems/nautilus_trader · error · anyhow::Error

Missing type_name in metadata

Error message

Missing type_name in metadata

What it means

Thrown by decode_batch_to_data when the batch schema metadata contains neither a 'type_name' key nor a 'bar_type' key, so the decoder cannot determine which data type to reconstruct. The type_name stored in metadata at write time is what routes the batch to the correct decode_data_batch implementation.

Source

Thrown at crates/persistence/src/backend/custom.rs:188

/// produce an error instead of attempting custom decode.
///
/// # Errors
///
/// Returns an error if decoding fails or the type is unknown (and custom fallback not allowed).
#[expect(
    clippy::implicit_hasher,
    reason = "Arrow schema metadata uses the standard HashMap type"
)]
pub fn decode_batch_to_data(
    metadata: &HashMap<String, String>,
    batch: RecordBatch,
    allow_custom_fallback: bool,
) -> anyhow::Result<Vec<Data>> {
    let type_name = metadata
        .get("type_name")
        .cloned()
        .or_else(|| metadata.get("bar_type").map(|_| "bars".to_string()))
        .ok_or_else(|| anyhow::anyhow!("Missing type_name in metadata"))?;

    match type_name.as_str() {
        "QuoteTick" | "quotes" => Ok(QuoteTick::decode_data_batch(metadata, batch)?),
        "TradeTick" | "trades" => Ok(TradeTick::decode_data_batch(metadata, batch)?),
        "Bar" | "bars" => Ok(Bar::decode_data_batch(metadata, batch)?),
        "OrderBookDelta" | "order_book_deltas" => {
            Ok(OrderBookDelta::decode_data_batch(metadata, batch)?)
        }
        "OrderBookDepth10" | "order_book_depths" => {
            Ok(OrderBookDepth10::decode_data_batch(metadata, batch)?)
        }
        "MarkPriceUpdate" | "mark_price_updates" => {
            Ok(MarkPriceUpdate::decode_data_batch(metadata, batch)?)
        }
        "IndexPriceUpdate" | "index_price_updates" => {
            Ok(IndexPriceUpdate::decode_data_batch(metadata, batch)?)
        }
        "OptionGreeks" | "option_greeks" => Ok(OptionGreeks::decode_data_batch(metadata, batch)?),

View on GitHub (pinned to 18893faf8b)

Solutions

1. Re-write the file through the nautilus catalog writer so type_name metadata is embedded.
2. If the file only has 'bar_type' in metadata, confirm you're on a version whose fallback handles it (this code maps bar_type -> bars).
3. Manually add schema metadata: Schema::new_with_metadata(fields, [("type_name", "...")]) before decoding.
4. Check for intermediate steps (compression, rewriting) that strip schema metadata and preserve it there.

Example fix

// before
let batch = RecordBatch::try_new(schema, columns)?; // schema has no metadata
// after
let mut metadata = HashMap::new();
metadata.insert("type_name".to_string(), "MyCustomData".to_string());
let schema = Arc::new(Schema::new_with_metadata(fields.clone(), metadata));
let batch = RecordBatch::try_new(schema, columns)?;

anyhow::ensure!(metadata.contains_key("type_name") || metadata.contains_key("bar_type"), "batch metadata missing type_name");

fn has_type_name(meta: &HashMap<String, String>) -> bool {
    meta.contains_key("type_name") || meta.contains_key("bar_type")
}

match decode_batch_to_data(&metadata, batch, true) {
    Ok(data) => use(data),
    Err(e) if e.to_string().contains("Missing type_name") => reingest_or_rewrite_file()?,
    Err(e) => return Err(e),
}

When it happens

Trigger: Reading a feather/parquet file whose schema metadata lacks type_name — typically a file not written by the nautilus catalog writer, or metadata stripped during file transformation/copy; also a hand-built RecordBatch passed to decode_custom_batches_to_data without the metadata.

Common situations: External tools (PyArrow/pandas round-trips) that drop Arrow schema metadata; older files written before type_name metadata was introduced; manually constructed batches in tests.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Cannot write {type_name} data with mixed identities: element
• prepare_custom_data_batch called with empty data
• Unknown data type: {type_name}
• Unknown data type: {type_name}; custom decode only allowed i
• Failed to serialize data_type for persistence: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/ea0b57252aa4decc. Report an issue: GitHub.