bytebase/bytebase · error

failed to marshal masked responses column

Error message

failed to marshal masked responses column

What it means

After all _msearch response elements are masked, the responses slice is re-serialized with json.Marshal back into the stored column string. Failure here means one of the (possibly masker-modified) values is no longer JSON-serializable, aborting the masking step.

Source

Thrown at backend/api/v1/document_masking.go:1288

func maskElasticsearchMSearchResponses(responsesColumnJSON string, sortFields []string, objectSchema *storepb.ObjectSchema, semanticTypeToMasker map[string]masker.Masker) (string, error) {
	var responses []any
	if err := json.Unmarshal([]byte(responsesColumnJSON), &responses); err != nil {
		return "", errors.Wrap(err, "failed to unmarshal responses column")
	}

	for i, resp := range responses {
		respMap, ok := resp.(map[string]any)
		if !ok {
			continue
		}
		if err := maskElasticsearchMSearchSingleResponse(respMap, sortFields, objectSchema, semanticTypeToMasker); err != nil {
			return "", errors.Wrapf(err, "failed to mask response %d", i)
		}
	}

	out, err := json.Marshal(responses)
	if err != nil {
		return "", errors.Wrap(err, "failed to marshal masked responses column")
	}
	return string(out), nil
}

// maskElasticsearchMSearchSingleResponse masks the hits in a single _msearch response element.
func maskElasticsearchMSearchSingleResponse(respMap map[string]any, sortFields []string, objectSchema *storepb.ObjectSchema, semanticTypeToMasker map[string]masker.Masker) error {
	hitsVal, ok := respMap["hits"]
	if !ok {
		return nil
	}
	hitsObj, ok := hitsVal.(map[string]any)
	if !ok {
		return nil
	}
	hitsArray, ok := hitsObj["hits"].([]any)
	if !ok {
		return nil
	}

View on GitHub (pinned to 1870550677)

Solutions

1. Ensure every masker emits only JSON-basic values (nil, bool, string, float64, json.Number, []any, map[string]any)
2. Inspect the wrapped inner error to locate the offending value
3. Sanitize masker output (convert unknown types to strings or null) before writing back into respMap
4. Add a unit test with the actual payload that reproduces the marshal failure

if _, err := json.Marshal(responses); err != nil { /* sanitize before final marshal */ }

func jsonSafe(v any) bool { return isJSONBasicDeep(v) }

out, err := json.Marshal(responses)
if err != nil {
    sanitizeNonSerializable(responses)
    out, err = json.Marshal(responses)
}

When it happens

Trigger: json.Marshal(responses) fails after maskElasticsearchMSearchSingleResponse has run — typically a masker replaced a value with a channel/func/invalid float, or produced a cyclic or unsupported structure.

Common situations: Custom maskers returning non-JSON Go types; NaN/Inf float results from numeric transformations; internal mutation of response maps with unsupported values.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• failed to marshal masked docs column
• failed to unmarshal responses column
• failed to mask response %d
• failed to parse CreateIssue response
• failed to parse %s response

AI-assisted analysis of bytebase/bytebase@1870550677 (2026-09-06). Data as JSON: /api/errors/6ce2a7a62d4deb5c. Report an issue: GitHub.