NanmiCoder/MediaCrawler · error · HTTPException

{e}

Error message

{e}

What it means

HTTP 500 with detail=str(e) — the catch-all handler in the preview branch of GET /api/data/files. Any exception other than JSONDecodeError from the csv/json/pandas readers lands here: pandas parse errors on corrupt xlsx/csv, OSError on permission or encoding failures, or MemoryError on huge files. Because the raw exception string is echoed to the client, the response can leak server-side paths and library internals.

Source

Thrown at api/routers/data.py:156

                import pandas as pd
                # Read first limit rows
                df = pd.read_excel(full_path, nrows=limit)
                # Get total row count (only read first column to save memory)
                df_count = pd.read_excel(full_path, usecols=[0])
                total = len(df_count)
                # Convert to list of dictionaries, handle NaN values
                rows = df.where(pd.notnull(df), None).to_dict(orient='records')
                return {
                    "data": rows,
                    "total": total,
                    "columns": list(df.columns)
                }
            else:
                raise HTTPException(status_code=400, detail="Unsupported file type for preview")
        except json.JSONDecodeError:
            raise HTTPException(status_code=400, detail="Invalid JSON file")
        except Exception as e:
            raise HTTPException(status_code=500, detail=str(e))
    else:
        # Return file download
        return FileResponse(
            path=full_path,
            filename=full_path.name,
            media_type="application/octet-stream"
        )


@router.get("/download/{file_path:path}")
async def download_file(file_path: str):
    """Download file"""
    full_path = DATA_DIR / file_path

    if not full_path.exists():
        raise HTTPException(status_code=404, detail="File not found")

    if not full_path.is_file():

View on GitHub (pinned to d6f7c5bb90)

Solutions

1. Download the file (preview=false) and open it locally to see the real parsing problem.
2. Check API-process read permissions on DATA_DIR and the specific file.
3. Re-save the Excel/CSV from its source application in a supported, well-formed format.
4. As the API maintainer: log the full traceback server-side and return a generic 500 detail instead of str(e).

Example fix

# before
except Exception as e:
    raise HTTPException(status_code=500, detail=str(e))

# after
except Exception as e:
    logger.exception("file preview failed")
    raise HTTPException(status_code=500, detail="Failed to preview file")

const size = (await fetch(`/api/data/files/${path}?preview=false`)).headers.get('content-length');
if (Number(size) > 10_000_000) { /* skip preview, download instead */ }

try { preview = await getPreview(path); } catch (e) { if (e.status === 500) { preview = await downloadRaw(path); /* fall back to raw bytes */ } }

When it happens

Trigger: Previewing a corrupt .xlsx that pandas cannot parse; permission-denied on the data file for the API process; a csv with mixed encoding raising UnicodeDecodeError; read_excel blowing up on a huge or non-spreadsheet file renamed to .xlsx.

Common situations: Excel files saved in legacy .xls format but renamed to .xlsx; files locked by another process on Windows; disk/permission misconfiguration of DATA_DIR; concurrent write by the crawler producing a partially-flushed file.

Related errors

• Invalid JSON file

AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/bae24282dd92b322. Report an issue: GitHub.