{"record":{"id":"34745dca825683c1","repo":"jackwener/OpenCLI","slug":"archive-snapshots-returned-malformed-cdx-payload-34745d","errorCode":null,"errorMessage":"archive snapshots returned malformed CDX payload: snapshot row is missing statuscode/mimetype","messagePattern":"archive snapshots returned malformed CDX payload: snapshot row is missing statuscode/mimetype","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/archive/snapshots.js","lineNumber":118,"sourceCode":"        header.forEach((name, i) => { cols[name] = i; });\n        const timestampCol = requireCdxColumn(cols, 'timestamp');\n        const originalCol = requireCdxColumn(cols, 'original');\n        const statusCol = requireCdxColumn(cols, 'statuscode');\n        const mimetypeCol = requireCdxColumn(cols, 'mimetype');\n\n        return rows.slice(0, limit).map(row => {\n            if (!Array.isArray(row)) {\n                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row must be an array');\n            }\n            const timestamp = String(row[timestampCol] ?? '');\n            const original = String(row[originalCol] ?? '');\n            const status = row[statusCol];\n            const mimetype = row[mimetypeCol];\n            if (!/^\\d{14}$/.test(timestamp) || !original) {\n                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row is missing timestamp/original URL');\n            }\n            if (status == null || mimetype == null || String(status) === '' || String(mimetype) === '') {\n                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row is missing statuscode/mimetype');\n            }\n            return {\n                timestamp,\n                snapshot_url: buildWaybackUrl(timestamp, original),\n                status: String(status),\n                mimetype: String(mimetype),\n                original_url: original,\n            };\n        });\n    },\n});\n","sourceCodeStart":100,"sourceCodeEnd":130,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/archive/snapshots.js#L100-L130","documentation":"Beyond timestamp and original URL, every snapshot row must have non-null, non-empty `statuscode` and `mimetype` cells. The library throws this when either is missing or blank, because these are required output columns and are coerced to strings for the result rows.","triggerScenarios":"A CDX row where `statuscode` or `mimetype` is null or empty string — common for tombstoned captures (redirects to a block page), newly ingested captures with pending metadata, or rows produced by CDX server-side collapsing.","commonSituations":"Removed/robot-blocked captures that still appear in CDX but lack status/mimetype; captures in the middle of archive.org's ingest pipeline; querying very recent or very old edge-case captures.","solutions":["Re-run the command; in-flight captures often gain their metadata shortly after.","Inspect the raw CDX output to identify rows with blank statuscode/mimetype.","Restrict the time range or path to exclude the problematic captures.","If it persists for all rows, check archive.org status and opencli updates."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Ensure statuscode/mimetype cells are populated before consuming results\nfunction rowHasStatusAndMimetype(row, stCol, mtCol) {\n  return row[stCol] != null && row[mtCol] != null && String(row[stCol]) !== '' && String(row[mtCol]) !== '';\n}","typeGuard":"function isPopulated(v) {\n  return v != null && String(v).trim() !== '';\n}","tryCatchPattern":"try {\n  await exec('opencli archive snapshots example.com --limit 20');\n} catch (e) {\n  if (String(e.message).includes('missing statuscode/mimetype')) {\n    // likely a tombstoned/in-flight capture — retry later or narrow range\n  } else throw e;\n}","preventionTips":["Wait and retry — new captures often gain metadata shortly after ingest","Exclude robot-blocked periods via from/to ranges","Query exact paths to reduce tombstoned row exposure","Check status.archive.org if all rows fail"],"tags":["cdx-api","data-quality","schema","wayback-machine"],"backgroundTag":"unexpected-response-schema","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}