databendlabs/databend · error · ValueError

Unsupported format for

Error message

Unsupported format for {source_path!r}. Use format= explicitly or pass pandas/polars/pyarrow data.

What it means

SemaphoreStorage permit keys are encoded as '<seq>/queue/<name>'. PermitKey::parse_key splits the input on the last two '/' and refuses keys that do not yield exactly 3 '/'. This indicates the key passed to parse_key is not a semaphore permit key — either it was malformed at write time, truncated, or a foreign key from the same storage prefix was passed in.

Solutions

  1. Log and inspect the offending key string printed in the message; it must look like '<u64>/queue/<name>'.
  2. Ensure only keys produced by PermitKey::to_key()/its writer path are passed to parse_key; filter foreign keys before parsing.
  3. Check for code that truncates or re-slashes keys (URL-encoding, path joins) between write and read.
  4. If old data was written by an older version with a different encoding, migrate or re-create those permit records.

Example fix

// before
let pk = PermitKey::parse_key(raw_key)?;
// after
if raw_key.matches('/').count() < 2 || !raw_key.contains("/queue/") {
    continue; // skip non-permit keys
}
let pk = PermitKey::parse_key(raw_key)?;
Defensive patterns

Strategy: validation

Validate before calling

fn is_permit_key(key: &str) -> bool { key.rsplitn(3, '/').count() == 3 && key.contains("/queue/") && key.rsplitn(3,'/').last().unwrap_or("").parse::<u64>().is_ok() }
if !is_permit_key(raw) { skip(raw); }

Type guard

fn as_permit_key(key: &str) -> Option<&str> { key.contains("/queue/").then_some(key) }

Try / catch

match PermitKey::parse_key(key) { Ok(pk) => handle(pk), Err(e) if e.kind() == io::ErrorKind::InvalidInput => skip_foreign_key(key), Err(e) => return Err(e.into()) }

Prevention

When it happens

Trigger: Calling PermitKey::parse_key (directly or via storage lookup code) with a string that has fewer than 3 slash-separated segments, e.g. 'abc', 'seq/queue', an empty string, or a key belonging to a different key namespace stored under the same root.

Common situations: Hand-edited or manually constructed keys in the meta store; a version change in the key encoding; iterating raw meta-service keys and assuming all are permit keys; copying keys from logs that were truncated.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/4c73d159a98822ad. Report an issue: GitHub.

Appendix: source

Thrown at src/bendpy/python/databend/__init__.py:149

        source: Any,
        *,
        format: str | None = None,
        pattern: str | None = None,
        connection: str | None = None,
    ) -> "SessionContext":
        if isinstance(source, (str, Path)):
            source_path = str(source)
            source_format = (format or Path(source_path).suffix.lstrip(".")).lower()
            if source_format in {"parquet", "pq"}:
                self.register_parquet(name, source_path, pattern=pattern, connection=connection)
            elif source_format in {"csv"}:
                self.register_csv(name, source_path, pattern=pattern, connection=connection)
            elif source_format in {"json", "ndjson"}:
                self.register_ndjson(name, source_path, pattern=pattern, connection=connection)
            elif source_format in {"txt", "text", "tsv"}:
                self.register_text(name, source_path, pattern=pattern, connection=connection)
            else:
                raise ValueError(
                    f"Unsupported format for {source_path!r}. "
                    "Use format= explicitly or pass pandas/polars/pyarrow data."
                )
            return self

        parquet_path = self._materialize_relation_source(name, source)
        self.register_parquet(name, parquet_path)
        return self

    def from_df(self, source: Any, *, name: str | None = None) -> Relation:
        target = name or _random_name("df")
        self.register(target, source)
        return self.table(target)

    def read_parquet(
        self,
        path: str | Path,
        *,

View on GitHub (pinned to 288d84d76e)