{"record":{"id":"b7802de69ce74a52","repo":"chenglou/pretext","slug":"could-not-determine-unicode-version-from-derivedbi","errorCode":null,"errorMessage":"Could not determine Unicode version from DerivedBidiClass header","messagePattern":"Could not determine Unicode version from DerivedBidiClass header","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/generate-bidi-data.ts","lineNumber":93,"sourceCode":"  return { start, end }\n}\n\nfunction simplifyBidiType(raw: string): GeneratedBidiType {\n  if (generatedTypeToCode.has(raw as GeneratedBidiType)) return raw as GeneratedBidiType\n  if (bnProjectedTypes.has(raw)) return 'BN'\n\n  const longName = longBidiNames.get(raw)\n  if (longName !== undefined) return longName\n\n  throw new Error(`Unsupported bidi class ${raw}`)\n}\n\nfunction buildPayload(sourceText: string): GeneratedBidiPayload {\n  const sourceLines = sourceText.split(/\\r?\\n/)\n  const versionLine = sourceLines.find(line => line.startsWith('# DerivedBidiClass-'))\n  const versionMatch = versionLine?.match(/^# DerivedBidiClass-(.+)\\.txt$/)\n  if (versionMatch === null || versionMatch === undefined) {\n    throw new Error('Could not determine Unicode version from DerivedBidiClass header')\n  }\n  const unicodeVersion = versionMatch[1]!\n\n  const bidiCodes = new Uint8Array(0x110000)\n  bidiCodes.fill(generatedTypeToCode.get('L')!)\n\n  for (let i = 0; i < sourceLines.length; i++) {\n    const rawLine = sourceLines[i]!\n    let rangeText: string | null = null\n    let typeText: string | null = null\n\n    if (rawLine.startsWith('# @missing:')) {\n      const missingMatch = rawLine.match(/^# @missing:\\s*([0-9A-Fa-f]+(?:\\.\\.[0-9A-Fa-f]+)?)\\s*;\\s*([A-Za-z_]+)/)\n      if (missingMatch === null) continue\n      rangeText = missingMatch[1]!\n      typeText = missingMatch[2]!\n    } else {\n      const line = rawLine.split('#', 1)[0]!.trim()","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/chenglou/pretext/blob/ac49b09b7d83ede19581fa94a8b892b07d309baf/scripts/generate-bidi-data.ts#L75-L111","documentation":"Thrown by buildPayload() in the bidi-data generator when it cannot find a header comment line matching '# DerivedBidiClass-<version>.txt' inside the bundled Unicode source file. The script depends on that exact header to extract the Unicode version string that gets stamped into the generated src/generated/bidi-data.ts. Without it, generation cannot proceed because the version is load-bearing metadata for the output.","triggerScenarios":"Running `bun run generate:bidi-data` when scripts/unicode/DerivedBidiClass-17.0.0.txt is missing, truncated, has had its comment header stripped, or has been replaced with a Unicode release whose header line no longer ends in `.txt`. Also triggered if the file is empty or only contains data rows with no leading comment.","commonSituations":"A maintainer upgrades to a new Unicode version and drops in a fresh DerivedBidiClass file that uses a slightly different header convention; or the source file got corrupted by a line-ending normalizer that mangled the leading `#`; or someone deleted the unicode/ directory.","solutions":["Open scripts/unicode/DerivedBidiClass-17.0.0.txt and confirm the first comment line is exactly of the form `# DerivedBidiClass-17.0.0.txt`.","If the file is missing or wrong, re-download the matching DerivedBidiClass.txt from the Unicode distribution for the version referenced by sourceFile in scripts/generate-bidi-data.ts:204 and place it at that path.","If you intentionally changed the header format, update the regex at scripts/generate-bidi-data.ts:91 to match the new convention.","If upgrading Unicode versions, also update the sourceFile path constant so it points at the new version's file."],"exampleFix":"// before — header missing the trailing .txt\n# DerivedBidiClass-17.0.0\n\n// after\n# DerivedBidiClass-17.0.0.txt","handlingStrategy":"validation","validationCode":"import { readFileSync, existsSync } from 'node:fs'\n\nfunction assertBidiSourceHeader(path: string): void {\n  if (!existsSync(path)) throw new Error(`Missing Unicode source: ${path}`)\n  const text = readFileSync(path, 'utf8')\n  const hasHeader = text.split(/\\r?\\n/).some(l => /^# DerivedBidiClass-.+\\.txt$/.test(l))\n  if (!hasHeader) throw new Error(`DerivedBidiClass header not found in ${path}`)\n}\n\nassertBidiSourceHeader('scripts/unicode/DerivedBidiClass-17.0.0.txt')","typeGuard":null,"tryCatchPattern":"try {\n  await import('./generate-bidi-data.ts')\n} catch (e) {\n  if (e instanceof Error && e.message.includes('DerivedBidiClass header')) {\n    console.error('Unicode source file is missing or its header changed. Re-fetch the matching DerivedBidiClass.txt and re-run.')\n    process.exit(1)\n  }\n  throw e\n}","preventionTips":["Treat scripts/unicode/*.txt as committed, version-pinned inputs; never edit them by hand.","When bumping the Unicode version, update both the sourceFile path constant and the committed .txt file in the same change."],"tags":["build","unicode","data-files","code-generation"],"backgroundTag":null,"analyzedSha":"ac49b09b7d83ede19581fa94a8b892b07d309baf","analyzedAt":"2026-08-12T17:03:16.263Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}