{"record":{"id":"3995eb901c082bf5","repo":"jackwener/OpenCLI","slug":"archive-snapshots-returned-malformed-cdx-payload-3995eb","errorCode":null,"errorMessage":"archive snapshots returned malformed CDX payload: snapshot row is missing timestamp/original URL","messagePattern":"archive snapshots returned malformed CDX payload: snapshot row is missing timestamp/original URL","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/archive/snapshots.js","lineNumber":115,"sourceCode":"            throw new CommandExecutionError('archive snapshots returned malformed CDX payload: header row must be an array');\n        }\n        const cols = {};\n        header.forEach((name, i) => { cols[name] = i; });\n        const timestampCol = requireCdxColumn(cols, 'timestamp');\n        const originalCol = requireCdxColumn(cols, 'original');\n        const statusCol = requireCdxColumn(cols, 'statuscode');\n        const mimetypeCol = requireCdxColumn(cols, 'mimetype');\n\n        return rows.slice(0, limit).map(row => {\n            if (!Array.isArray(row)) {\n                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row must be an array');\n            }\n            const timestamp = String(row[timestampCol] ?? '');\n            const original = String(row[originalCol] ?? '');\n            const status = row[statusCol];\n            const mimetype = row[mimetypeCol];\n            if (!/^\\d{14}$/.test(timestamp) || !original) {\n                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row is missing timestamp/original URL');\n            }\n            if (status == null || mimetype == null || String(status) === '' || String(mimetype) === '') {\n                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row is missing statuscode/mimetype');\n            }\n            return {\n                timestamp,\n                snapshot_url: buildWaybackUrl(timestamp, original),\n                status: String(status),\n                mimetype: String(mimetype),\n                original_url: original,\n            };\n        });\n    },\n});\n","sourceCodeStart":97,"sourceCodeEnd":130,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/archive/snapshots.js#L97-L130","documentation":"Each snapshot row must contain a 14-digit `timestamp` (YYYYMMDDhhmmss) and a non-empty `original` URL. This error is thrown when either cell is absent, empty, or the timestamp fails the `/^\\d{14}$/` check, since the library uses them to construct snapshot URLs.","triggerScenarios":"A CDX row has an empty/short timestamp (e.g. collapsed or truncated fields) or a missing `original` column value — typically from `collapse=` style server-side deduplication artifacts, truncated responses, or CDX API changes to field contents.","commonSituations":"Rows with collapsed timestamps of reduced precision; captures whose original URL was redacted/empty; proxy responses cutting cells; querying with flags that trigger CDX field collapsing.","solutions":["Re-run the command to see if the bad row is transient.","Inspect the raw CDX output for the offending row and note its timestamp/original values.","Narrow the query (specific path, tighter from/to, smaller limit) to skip malformed rows.","Update opencli or report the payload upstream if CDX changed its field semantics."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Verify a raw CDX row has the required cells before processing\nfunction rowHasTimestampAndOriginal(row, tsCol, origCol) {\n  return /^\\d{14}$/.test(String(row[tsCol] ?? '')) && Boolean(row[origCol]);\n}","typeGuard":"function hasValidTimestamp(v) {\n  return typeof v === 'string' && /^\\d{14}$/.test(v);\n}","tryCatchPattern":"try {\n  await exec('opencli archive snapshots example.com --limit 20');\n} catch (e) {\n  if (String(e.message).includes('missing timestamp/original URL')) {\n    console.error('CDX returned a capture with blank timestamp/URL — narrow the query');\n  } else throw e;\n}","preventionTips":["Avoid CDX collapsing/dedup flags that produce truncated timestamps","Prefer exact-path queries over wildcard domain queries","Re-run once for transient ingest artifacts","Check raw payload when errors repeat for the same row"],"tags":["cdx-api","data-quality","schema","wayback-machine"],"backgroundTag":"unexpected-response-schema","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}