{"record":{"id":"23281450e799b284","repo":"projectdiscovery/katana","slug":"response-filtered-by-similarity-detection","errorCode":null,"errorMessage":"response filtered by similarity detection","messagePattern":"response filtered by similarity detection","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"info","filePath":"pkg/output/output.go","lineNumber":185,"sourceCode":"\t}\n\tif options.OutputTemplate != \"\" {\n\t\twriter.outputTemplate, err = fasttemplate.NewTemplate(options.OutputTemplate, \"{{\", \"}}\")\n\t\tif err != nil {\n\t\t\treturn nil, errkit.Wrap(err, \"output: could not create output format template\")\n\t\t}\n\t}\n\treturn writer, nil\n}\n\n// Write writes the result to file and/or screen.\nfunc (w *StandardWriter) Write(result *Result) error {\n\tif result == nil {\n\t\treturn errors.New(\"result is nil\")\n\t}\n\n\t// Skip empty responses (e.g., from similarity filtering)\n\tif result.Response != nil && result.Response.Resp == nil && result.Response.Body == \"\" && result.Error == \"\" {\n\t\treturn errors.New(\"response filtered by similarity detection\")\n\t}\n\n\tif len(w.storeFields) > 0 {\n\t\tstoreFields(result, w.storeFields)\n\t}\n\n\tif !w.extensionValidator.ValidatePath(result.Request.URL) {\n\t\treturn errors.New(\"result does not match extension filter\")\n\t}\n\n\tif !w.matchOutput(result) {\n\t\treturn errors.New(\"result does not match output\")\n\t}\n\tif w.filterOutput(result) {\n\t\treturn errors.New(\"result is filtered out\")\n\t}\n\tif len(w.filterPageType) > 0 && result.Response != nil && result.Response.KnowledgeBase != nil {\n\t\tif pageType, ok := result.Response.KnowledgeBase[\"PageType\"].(string); ok {","sourceCodeStart":167,"sourceCodeEnd":203,"githubUrl":"https://github.com/projectdiscovery/katana/blob/e3e742739c3746f085943ce918fb4e2b8daf6fe6/pkg/output/output.go#L167-L203","documentation":"StandardWriter.Write in pkg/output/output.go returns this error when a Result carries a Response object that is entirely empty (no http.Resp, no body, no error string). The library treats such results as duplicates suppressed by similarity detection and intentionally skips writing them, surfacing a sentinel error instead of silent output.","triggerScenarios":"Calling writer.Write(result) where result.Response != nil but result.Response.Resp == nil, result.Response.Body == \"\", and result.Error == \"\" — i.e. a result whose response was dropped by similarity/duplicate filtering upstream but still reached the writer.","commonSituations":"Running a crawl with similarity/duplicate-response filtering enabled and forwarding every emitted result (including suppressed ones) to a custom output writer; building a custom writer that ignores the filter flag; re-processing previously saved results that had empty bodies.","solutions":["Check result.Response.Resp/Body/Error before calling Write and skip results that are empty — the error is an expected skip signal, not a fault.","Disable or loosen similarity/duplicate filtering in the crawler options so suppressed responses are not emitted.","Treat this specific error as a no-op in your Write loop (log at debug level and continue).","If you genuinely need to record filtered responses, populate result.Error or the body before writing, or bypass the writer's empty-response guard."],"exampleFix":"// before\nif err := writer.Write(result); err != nil {\n    return err\n}\n// after\nif err := writer.Write(result); err != nil {\n    if err.Error() == \"response filtered by similarity detection\" {\n        continue // expected skip for duplicate/empty responses\n    }\n    return err\n}","handlingStrategy":"validation","validationCode":"func writable(r *output.Result) bool {\n    return r != nil && !(r.Response != nil && r.Response.Resp == nil && r.Response.Body == \"\" && r.Error == \"\")\n}","typeGuard":"func hasContent(r *output.Result) bool {\n    return r != nil && r.Response != nil && (r.Response.Resp != nil || r.Response.Body != \"\" || r.Error != \"\")\n}","tryCatchPattern":"if err := writer.Write(result); err != nil {\n    if err.Error() == \"response filtered by similarity detection\" {\n        return nil // expected skip\n    }\n    return err\n}","preventionTips":["Skip results with empty response/body/error before calling Write","Treat this error as an informational skip, not a failure","Keep similarity filtering options consistent with your custom writer's expectations"],"tags":["output","filtering","duplicate-detection","go"],"backgroundTag":"result-filtered-by-duplicate-detection","analyzedSha":"e3e742739c3746f085943ce918fb4e2b8daf6fe6","analyzedAt":"2026-09-03T14:55:13.248Z","contentChangedAt":"2026-09-03T14:55:13.248Z","schemaVersion":2},"datasetVersion":"2026-09-10T17:17:09.494Z"}