{"record":{"id":"43cf9ed2b049f175","repo":"bytebase/bytebase","slug":"failed-to-unmarshal-field-q","errorCode":null,"errorMessage":"failed to unmarshal field %q","messagePattern":"failed to unmarshal field %q","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"backend/api/v1/document_masking.go","lineNumber":1223,"sourceCode":"}\n\n// maskElasticsearchGetSourceColumn masks a single field value from a GET _source response.\n// In _source responses, each top-level document field becomes a separate column.\n// fieldName is the column name (= field name), valueJSON is the marshaled value.\nfunc maskElasticsearchGetSourceColumn(fieldName, valueJSON string, objectSchema *storepb.ObjectSchema, semanticTypeToMasker map[string]masker.Masker) (string, error) {\n\tvar childSchema *storepb.ObjectSchema\n\tif structKind := objectSchema.GetStructKind(); structKind != nil {\n\t\tif props := structKind.GetProperties(); props != nil {\n\t\t\tchildSchema = props[fieldName]\n\t\t}\n\t}\n\tif childSchema == nil {\n\t\treturn valueJSON, nil\n\t}\n\n\tvar value any\n\tif err := json.Unmarshal([]byte(valueJSON), &value); err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"failed to unmarshal field %q\", fieldName)\n\t}\n\n\tmasked, err := walkAndMaskJSONRecursive(value, childSchema, semanticTypeToMasker)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tout, err := json.Marshal(masked)\n\tif err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"failed to marshal masked field %q\", fieldName)\n\t}\n\treturn string(out), nil\n}\n\n// maskElasticsearchMGetSource masks _source in each doc within a _mget response.\n// The docsColumnJSON is the JSON string from the \"docs\" column.\nfunc maskElasticsearchMGetSource(docsColumnJSON string, objectSchema *storepb.ObjectSchema, semanticTypeToMasker map[string]masker.Masker) (string, error) {\n\tvar docs []any","sourceCodeStart":1205,"sourceCodeEnd":1241,"githubUrl":"https://github.com/bytebase/bytebase/blob/1870550677fe08f0d2a78c07acd27541464eb945/backend/api/v1/document_masking.go#L1205-L1241","documentation":"In a GET _source response, each top-level document field arrives as its own column value. maskElasticsearchGetSourceColumn parses each field's JSON string before walking it with the schema masker; this error means the field value string was not valid JSON.","triggerScenarios":"MaskGetSource API where a column's value string fails json.Unmarshal — e.g. the driver returned a scalar as a raw unquoted string, or the value was truncated/mangled.","commonSituations":"Driver serializing primitive fields inconsistently (string vs JSON-encoded); truncation of long text fields; ES/text extraction tools quoting values differently.","solutions":["Inspect the offending field's raw value (named in the error) to see why it isn't valid JSON.","Ensure the ES driver serializes every field value as a JSON string, including scalars.","Fix data or driver truncation for large text fields.","Retry after correcting the field serialization."],"exampleFix":"// before: column value for field \"note\" is: some unquoted text\n// after: serialize as valid JSON string: \"some unquoted text\"","handlingStrategy":"type-guard","validationCode":"// Before masking a _source field column:\n// if !json.Valid([]byte(valueJSON)) { return valueJSON, nil }","typeGuard":"func isJSONString(s string) bool { return json.Valid([]byte(s)) }","tryCatchPattern":"masked, err := maskElasticsearchGetSourceColumn(colName, value, ...)\nif err != nil && strings.Contains(err.Error(), \"failed to unmarshal field\") {\n    // pass through the original value and log the field name\n}","preventionTips":["Ensure the driver JSON-encodes all field values, including scalars","Prevent truncation of long text fields","Add round-trip tests for GET _source column serialization"],"tags":["elasticsearch","json","get-source"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"1870550677fe08f0d2a78c07acd27541464eb945","analyzedAt":"2026-09-06T21:16:13.665Z","contentChangedAt":"2026-09-06T21:16:13.665Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}