{"record":{"id":"5007f29e6fe2d66e","repo":"nodejs/node","slug":"warning-no-converter-defined-using-codepage-o","errorCode":null,"errorMessage":"###WARNING: No converter defined. Using codepage of system.\n","messagePattern":"###WARNING: No converter defined\\. Using codepage of system\\.\n","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"deps/icu-small/source/tools/toolutil/ucbuf.cpp","lineNumber":499,"sourceCode":"        if(*cp==nullptr || **cp=='\\0'){\n            /* don't have code page name... try to autodetect */\n            ucbuf_autodetect_fs(in,cp,&buf->conv,&buf->signatureLength,error);\n        }else if(ucbuf_isCPKnown(*cp)){\n            /* discard BOM */\n            ucbuf_autodetect_fs(in,&knownCp,&buf->conv,&buf->signatureLength,error);\n        }\n        if(U_SUCCESS(*error) && buf->conv==nullptr) {\n            buf->conv=ucnv_open(*cp,error);\n        }\n        if(U_FAILURE(*error)){\n            ucnv_close(buf->conv);\n            uprv_free(buf);\n            T_FileStream_close(in);\n            return nullptr;\n        }\n        \n        if((buf->conv==nullptr) && (buf->showWarning==true)){\n            fprintf(stderr,\"###WARNING: No converter defined. Using codepage of system.\\n\");\n        }\n        buf->remaining=fileSize-buf->signatureLength;\n        if(buf->isBuffered){\n            buf->bufCapacity=MAX_U_BUF;\n        }else{\n            buf->bufCapacity=buf->remaining+buf->signatureLength+1/*for terminating nul*/;               \n        }\n        buf->buffer=(char16_t*) uprv_malloc(U_SIZEOF_UCHAR * buf->bufCapacity );\n        if (buf->buffer == nullptr) {\n            *error = U_MEMORY_ALLOCATION_ERROR;\n            ucbuf_close(buf);\n            return nullptr;\n        }\n        buf->currentPos=buf->buffer;\n        buf->bufLimit=buf->buffer;\n        if(U_FAILURE(*error)){\n            fprintf(stderr, \"Could not open codepage [%s]: %s\\n\", *cp, u_errorName(*error));\n            ucbuf_close(buf);","sourceCodeStart":481,"sourceCodeEnd":517,"githubUrl":"https://github.com/nodejs/node/blob/1b2de5e052fc0fb95fd7fb6846dcec4ade598e9e/deps/icu-small/source/tools/toolutil/ucbuf.cpp#L481-L517","documentation":"Emitted from ucbuf_open after BOM autodetection and an explicit ucnv_open(*cp) have both failed to produce a converter (buf->conv is still nullptr). With showWarning true it warns that conversion will fall back to the platform's default codepage; reading continues but byte interpretation is implicit and may be wrong.","triggerScenarios":"ucbuf_open called with *cp == nullptr or pointing to an empty string on a file that has no recognizable BOM, or with a *cp value ucnv_open silently returned null for; the converter ends up null and the rest of the pipeline assumes the system default codepage.","commonSituations":"An ICU data-building tool invoked without an encoding/-e flag on a plain ASCII/ANSI file lacking a BOM; deployment across OSes whose default codepage differs (Windows ACP vs Linux UTF-8); data files stripped of their BOM by a transfer pipeline.","solutions":["Always pass an explicit, canonical codepage string to ucbuf_open (e.g. \"UTF-8\") instead of leaving *cp null.","Save the input as UTF-8/UTF-16 with a BOM so ucbuf_autodetect_fs succeeds and sets buf->conv.","Keep showWarning=true during development so this fallback is surfaced, and treat it as a build error.","Detect the encoding upstream with uchardet/file -i and feed that name as *cp."],"exampleFix":"// before: no codepage, no BOM -> silent fallback to system codepage\nconst char* cp = nullptr;\nUCHARBUF* b = ucbuf_open(path, &cp, true, false, &status);\n// after: force UTF-8 explicitly\nconst char* cp = \"UTF-8\";\nUCHARBUF* b = ucbuf_open(path, &cp, true, false, &status);","handlingStrategy":"fallback","validationCode":"// Guarantee a usable converter before ucbuf_open by resolving cp yourself.\n#include <unicode/ucnv.h>\nconst char* resolve_cp(const char* path, const char** cp) {\n    // 1) if caller gave a non-empty, openable cp -> use it\n    if (*cp && **cp) {\n        UErrorCode e = U_ZERO_ERROR;\n        UConverter* c = ucnv_open(*cp, &e);\n        if (U_SUCCESS(e) && c) { ucnv_close(c); return *cp; }\n    }\n    // 2) else try UTF-8 (add BOM to files so this path is reliable)\n    static const char* fb = \"UTF-8\";\n    UErrorCode e = U_ZERO_ERROR;\n    UConverter* c = ucnv_open(fb, &e);\n    if (U_SUCCESS(e) && c) { ucnv_close(c); return fb; }\n    return nullptr; // give up explicitly rather than rely on system codepage\n}","typeGuard":null,"tryCatchPattern":"// This is a warning, not a status failure -- it prints to stderr and continues.\n// Detect it structurally: after ucbuf_open, verify a converter exists.\nUCHARBUF* b = ucbuf_open(path, &cp, /*showWarning=*/true, false, &status);\nif (U_SUCCESS(status) && b && /* buf->conv is null */ cp_was_null_and_no_bom) {\n    fprintf(stderr, \"refusing implicit system-codepage fallback\\n\");\n    ucbuf_close(b);\n    return EXIT_FAILURE;\n}","preventionTips":["Always pass an explicit canonical codepage to ucbuf_open; never rely on null *cp.","Keep a UTF-8 BOM on data files so autodetect succeeds even when *cp is null.","Treat this warning as an error in CI (grep stderr for the message).","Pin the runtime locale/encoding so the 'system codepage' fallback is at least deterministic."],"tags":["icu","encoding","codepage","fallback","toolutil"],"backgroundTag":null,"analyzedSha":"1b2de5e052fc0fb95fd7fb6846dcec4ade598e9e","analyzedAt":"2026-08-13T00:53:24.642Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}