langgenius/dify · error · ValueError

Data source type not support

Error message

Data source type not support

What it means

Built-in ValueError (uncaught -> HTTP 500) raised in the batch estimate handler's match/case default arm. document.data_source_type is not one of upload_file / notion_import / website_crawl, so the controller cannot build an ExtractSetting and bails. Indicates a data_source_type value the handler was not updated to support.

Source

Thrown at api/controllers/console/datasets/datasets_document.py:875

                        continue
                    extract_setting = ExtractSetting(
                        datasource_type=DatasourceType.WEBSITE,
                        website_info=WebsiteInfo.model_validate(
                            {
                                "provider": data_source_info["provider"],
                                "job_id": data_source_info["job_id"],
                                "url": data_source_info["url"],
                                "tenant_id": current_tenant_id,
                                "mode": data_source_info["mode"],
                                "only_main_content": data_source_info["only_main_content"],
                            }
                        ),
                        document_model=document.doc_form,
                    )
                    extract_settings.append(extract_setting)

                case _:
                    raise ValueError("Data source type not support")
            indexing_runner = IndexingRunner()
            try:
                response = indexing_runner.indexing_estimate(
                    tenant_id=current_tenant_id,
                    extract_settings=extract_settings,
                    tmp_processing_rule=data_process_rule_dict,
                    doc_form=document.doc_form,
                    doc_language="English",
                    dataset_id=dataset_id_str,
                    session=session,
                )
                return (
                    IndexingEstimateResponse(
                        tokens=0,
                        total_price=0,
                        currency="USD",
                        total_segments=response.total_segments,
                        preview=response.preview,

View on GitHub (pinned to ef8544b173)

Solutions

1. Identify the offending data_source_type via the document row (SELECT data_source_type FROM documents WHERE id=...).
2. Exclude documents of that type from the batch estimate, or handle them separately.
3. If the type is a shipped datasource, file a bug — the handler needs a matching case.
4. Correct corrupted data_source_type values in the DB if they are typos.

SUPPORTED_DATA_SOURCES = {'upload_file', 'notion_import', 'website_crawl'}

def batch_data_sources_supported(docs: list[dict]) -> bool:
    return all(d.get('data_source_type') in SUPPORTED_DATA_SOURCES for d in docs)

# fetch batch documents and guard before estimating
r = requests.get(f"{base}/console/api/datasets/{dataset_id}/batch/{batch}/indexing-status",
                 headers=hdrs)
r.raise_for_status()
if not batch_data_sources_supported(r.json().get('data', [])):
    raise SystemExit('batch contains an unsupported data_source_type; split the batch')

def is_supported_data_source(ds_type: str) -> bool:
    return ds_type in {'upload_file', 'notion_import', 'website_crawl'}

try:
    r = requests.get(f"{base}/console/api/datasets/{dataset_id}/batch/{batch}/indexing-estimate",
                     headers=hdrs)
    r.raise_for_status()
except requests.HTTPError as e:
    if e.response.status_code == 500 and 'Data source type not support' in e.response.text:
        # split the batch to only supported data_source_types and retry
        supported = [d for d in fetch_batch(dataset_id, batch)
                     if d['data_source_type'] in SUPPORTED_DATA_SOURCES]
        # report unsupported ones as a bug if they look like shipped datasources
    else:
        raise

When it happens

Trigger: GET .../batch/{batch}/indexing-estimate over a batch containing a document whose data_source_type is something other than the three handled types (e.g., a newly introduced or custom datasource).

Common situations: New datasource type shipped without updating this estimate handler; legacy row with a typo in data_source_type; custom plugin injecting an unhandled type.

Related errors

• Data source binding not found.
• Data source is not disabled.
• Data source is disabled.
• indexing_estimate_error
• auth_failed

AI-assisted analysis of langgenius/dify@ef8544b173 (2026-08-12). Data as JSON: /api/errors/97be553aa71c7e2b. Report an issue: GitHub.