JuliusBrussee/caveman · error · CatalogError

: every source must be HTTPS

Error message

{label}: every source must be HTTPS

What it means

Every entry in a row's sources list must be a string starting with 'https://'. Non-string entries or http:// (insecure) URLs fail validation, since the catalog only accepts secure documentation links.

Solutions

  1. Change the scheme to https:// (update the URL if the site supports TLS)
  2. Add the https:// prefix if the scheme was omitted
  3. Quote the URL in YAML if special characters cause it to parse as a non-string

Example fix

// before
sources:
  - http://docs.example.com/pricing
// after
sources:
  - https://docs.example.com/pricing
Defensive patterns

Strategy: validation

Validate before calling

assert all(isinstance(s, str) and s.startswith("https://") for s in row["sources"]), "all sources must be https"

Type guard

def all_https(srcs):
    return isinstance(srcs, list) and all(isinstance(s, str) and s.startswith("https://") for s in srcs)

Try / catch

try:
    validate_catalog(rows)
except CatalogError as e:
    if "every source must be HTTPS" in str(e):
        upgrade_sources_to_https(row)
    raise

Prevention

When it happens

Trigger: A source entry is "http://example.com/pricing", a bare domain like "example.com/pricing", or a non-string value such as a number or nested object.

Common situations: Copying an http:// link from an older site; forgetting the scheme entirely; YAML interpreting a URL-ish token as a non-string (e.g. a mapping); internal docs links using http.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/a23d22446fab1d37. Report an issue: GitHub.

Appendix: source

Thrown at shared/provider-catalog/validate_catalog.py:160

    for field, value in pricing.items():
        if value is None or field == "long_context_threshold_inclusive":
            if field == "long_context_threshold_inclusive" and value is not None and not isinstance(value, bool):
                raise CatalogError(f"{label}: pricing.{field} must be boolean or null")
            continue
        if isinstance(value, bool) or not isinstance(value, (int, float)):
            raise CatalogError(f"{label}: pricing.{field} must be numeric or null")
        if not math.isfinite(value) or value < 0:
            raise CatalogError(f"{label}: pricing.{field} must be finite and nonnegative")
        if field == "batch_discount_fraction" and value > 1:
            raise CatalogError(f"{label}: pricing.batch_discount_fraction must be <= 1")

    if not isinstance(row["capabilities"], dict):
        raise CatalogError(f"{label}: capabilities must be an object")
    sources = row["sources"]
    if not isinstance(sources, list) or not sources:
        raise CatalogError(f"{label}: sources must be a non-empty list")
    if any(not isinstance(source, str) or not source.startswith("https://") for source in sources):
        raise CatalogError(f"{label}: every source must be HTTPS")

    verified_at = parsed_time(row["verified_at"], label)
    if verified_at > now:
        raise CatalogError(f"{label}: verified_at is in the future")
    if verified_at < now - timedelta(days=120):
        raise CatalogError(f"{label}: verified_at is older than 120 days")
    return identity[0], identity[1], identity[2]


def check_review_markers(text: str, label: str) -> None:
    """Refuse a catalog that still carries an unreviewed sync proposal.

    YAML comments are invisible to the parsed rows, so this reads the raw file:
    a rubber-stamped merge must not be able to advance price provenance while
    the line saying "nobody has confirmed this yet" is still in the file.
    """
    offending = [
        index + 1

View on GitHub (pinned to 3ee70a1026)