nautechsystems/nautilus_trader · error · anyhow::Error

Cannot write {type_name} data with mixed identities: element

Error message

Cannot write {type_name} data with mixed identities: element {position} has metadata {:?} but the first element has {first_metadata:?}; write each instrument or bar type separately

What it means

`ParquetDataCatalog::write_to_parquet` requires all elements in one write batch to share identical metadata (instrument/bar identity encoded in `ts_init`-side metadata). If any element's metadata differs from the first element's, the write aborts because later rows would be silently re-labeled with the first element's identity, corrupting the file.

Source

Thrown at crates/persistence/src/backend/catalog.rs:604

    ) -> anyhow::Result<PathBuf>
    where
        T: HasTsInit + EncodeToRecordBatch + CatalogPathPrefix,
    {
        if data.is_empty() {
            return Ok(PathBuf::new());
        }

        let type_name = to_snake_case(std::any::type_name::<T>());
        Self::check_ascending_timestamps(data, &type_name)?;

        // The write directory and schema metadata come from the first element,
        // so mixed identities would silently re-label everything after it
        let first_metadata = data[0].metadata();
        if let Some(position) = data
            .iter()
            .position(|item| item.metadata() != first_metadata)
        {
            anyhow::bail!(
                "Cannot write {type_name} data with mixed identities: element {position} has \
                 metadata {:?} but the first element has {first_metadata:?}; write each \
                 instrument or bar type separately",
                data[position].metadata(),
            );
        }

        let start_ts = start.unwrap_or(data.first().unwrap().ts_init());
        let end_ts = end.unwrap_or(data.last().unwrap().ts_init());

        let batches = self.data_to_record_batches(data)?;
        let schema = batches.first().expect("Batches are empty.").schema();

        let identifier = if T::path_prefix() == "bars" {
            schema.metadata.get("bar_type").cloned()
        } else {
            schema.metadata.get("instrument_id").cloned()
        };

View on GitHub (pinned to 18893faf8b)

Solutions

1. Group the data by metadata (instrument ID / bar type) and issue one `write_to_parquet` call per group.
2. If streaming, flush and start a new write whenever metadata changes mid-stream.
3. Inspect `data[i].metadata()` across the batch to find the first divergent element and split there.

Example fix

// before
catalog.write_to_parquet(&mixed_quotes, type_name, None, None, None, None, None)?;
// after
for (_meta, group) in group_quotes_by_metadata(&mixed_quotes) {
    catalog.write_to_parquet(&group, type_name, None, None, None, None, None)?;
}

fn all_same_metadata(data: &[impl MetadataAware]) -> bool {
    let first = data[0].metadata();
    data.iter().all(|d| d.metadata() == first)
}

match catalog.write_to_parquet(&batch, type_name, None, None, None, None, None) {
    Err(e) if e.to_string().contains("mixed identities") => {
        for group in split_by_metadata(&batch) { write_group(group)?; }
    }
    other => other?,
}

When it happens

Trigger: Calling `write_to_parquet` (directly or via helpers like quote/trade/funding writers) with a slice that mixes instruments, bar types, or otherwise differing metadata within one batch.

Common situations: Batching data for multiple symbols or bar types into a single write call; replaying a mixed stream into one buffer; concatenating buffers from different instruments before writing.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• {type_name} timestamps must be in ascending order
• custom data request requires `instrument_id` metadata
• Unsupported Data::Defi variant for catalog writes
• Unsupported Data variant for catalog writes
• Writing file {filename} with interval ({start_ts}, {end_ts})

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/e4fc44dce6cc1119. Report an issue: GitHub.