{"record":{"id":"92ec8ab7672f6c0b","repo":"apache/seatunnel","slug":"the-string-type-in-doris-can-only-store-up-to-2gb","errorCode":null,"errorMessage":"The String type in Doris can only store up to 2GB bytes, and the current field [%s] length is [%s] bytes. If it is greater than the maximum length of the String in Doris, it may not be able to write data","messagePattern":"The String type in Doris can only store up to 2GB bytes, and the current field \\[(.+?)\\] length is \\[(.+?)\\] bytes\\. If it is greater than the maximum length of the String in Doris, it may not be able to write data","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-doris/src/main/java/org/apache/seatunnel/connectors/doris/datatype/AbstractDorisTypeConverter.java","lineNumber":235,"sourceCode":"                builder.dataType(DORIS_CHAR);\n            }\n            return;\n        }\n\n        if (column.getColumnLength() <= 65533) {\n            builder.columnType(String.format(\"%s(%s)\", DORIS_VARCHAR, column.getColumnLength()));\n            builder.dataType(DORIS_VARCHAR);\n            return;\n        }\n\n        if (column.getColumnLength() <= MAX_STRING_LENGTH) {\n            builder.columnType(DORIS_STRING);\n            builder.dataType(DORIS_STRING);\n            return;\n        }\n\n        if (column.getColumnLength() > MAX_STRING_LENGTH) {\n            log.warn(\n                    String.format(\n                            \"The String type in Doris can only store up to 2GB bytes, and the current field [%s] length is [%s] bytes. If it is greater than the maximum length of the String in Doris, it may not be able to write data\",\n                            column.getName(), column.getColumnLength()));\n            builder.columnType(DORIS_STRING);\n            builder.dataType(DORIS_STRING);\n            return;\n        }\n\n        throw CommonError.convertToConnectorTypeError(\n                IDENTIFIER, column.getDataType().getSqlType().name(), column.getName());\n    }\n\n    /**\n     * Maximum decimal scale accepted by the target Doris version.\n     *\n     * <p>DECIMALV3 only requires the scale to not exceed the precision, which the caller already\n     * enforces, so the default imposes no further limit. Doris 1.x is stricter and overrides this.\n     *","sourceCodeStart":217,"sourceCodeEnd":253,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-doris/src/main/java/org/apache/seatunnel/connectors/doris/datatype/AbstractDorisTypeConverter.java#L217-L253","documentation":"AbstractDorisTypeConverter logs this warning when converting a SeaTunnel STRING column whose declared column length exceeds Doris's maximum String size (2GB, MAX_STRING_LENGTH). Doris String can hold at most 2,147,483,647 bytes, so a longer declared length cannot be honored; the converter still maps the column to DORIS_STRING and proceeds. Data longer than the Doris limit may fail to write at sink time.","triggerScenarios":"sampleReconvertString is called during schema conversion (e.g. sink table schema creation) with a source column whose getColumnLength() > MAX_STRING_LENGTH (2GB), typical when source metadata reports an unbounded/oversized VARCHAR/TEXT length.","commonSituations":"Syncing from sources like Oracle LONGTEXT, MySQL LONGTEXT, or a VARCHAR with huge declared length; JDBC drivers reporting Integer.MAX_VALUE length for text columns; auto table creation into Doris from such sources.","solutions":["Reduce the declared column length in the source or in a transform before writing to Doris","Cap the column length in the catalog config / type mapping so it stays under 2GB bytes","If data truly exceeds 2GB, split or truncate the field upstream; Doris String cannot store it","Verify actual data sizes; often the declared length is inflated and real values fit fine (warning is safe to ignore then)"],"exampleFix":"// before: source column length reported as 4294967295 (MySQL LONGTEXT)\ncatalogTable.getFieldNames(); // column length 4294967295 -> warning\n// after: constrain via transform/schema override so length <= 2147483647\nrow.field(0, new StringType()); // or cap length in type converter config","handlingStrategy":"validation","validationCode":"if (column.getColumnLength() != null && column.getColumnLength() > 2147483647L) {\n    // shorten/truncate the column or map it manually before Doris sink\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check source column lengths for text fields before configuring a Doris sink","Override oversized text columns explicitly instead of relying on automatic type conversion","Remember Doris String max is 2GB bytes, not characters"],"tags":["doris","type-conversion","string-length","schema"],"backgroundTag":"value-out-of-range","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}