{"record":{"id":"90817aa460966668","repo":"apache/druid","slug":"avro-jq-not-supported","errorCode":null,"errorMessage":"Avro + JQ not supported","messagePattern":"Avro \\+ JQ not supported","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/AvroFlattenerMaker.java","lineNumber":161,"sourceCode":"  {\n    if (record.getSchema().getField(key) != null) {\n      return transformValue(record.get(key));\n    } else {\n      return null;\n    }\n  }\n\n  @Override\n  public Function<GenericRecord, Object> makeJsonPathExtractor(final String expr)\n  {\n    final JsonPath jsonPath = JsonPath.compile(expr);\n    return record -> transformValue(jsonPath.read(record, jsonPathConfiguration));\n  }\n\n  @Override\n  public Function<GenericRecord, Object> makeJsonQueryExtractor(final String expr)\n  {\n    throw new UnsupportedOperationException(\"Avro + JQ not supported\");\n  }\n\n  @Override\n  public Function<GenericRecord, Object> makeJsonTreeExtractor(List<String> nodes)\n  {\n    if (nodes.size() == 1) {\n      return (GenericRecord record) -> getRootField(record, nodes.get(0));\n    }\n\n    throw new UnsupportedOperationException(\"Avro + nested tree extraction not supported\");\n  }\n\n  @Override\n  public JsonProvider getJsonProvider()\n  {\n    return avroJsonProvider;\n  }\n","sourceCodeStart":143,"sourceCodeEnd":179,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/AvroFlattenerMaker.java#L143-L179","documentation":"The Avro flattener supports JSON-path and JSON-tree-based flattening via jsonPath, but the JQ-based extraction (makeJsonQueryExtractor) is not implemented for Avro records. The interface method exists (shared flattener API), so it throws UnsupportedOperationException to indicate Avro + JQ is not a supported combination.","triggerScenarios":"Configuring an Avro inputFormat's flattenSpec (or a flattener usage) whose flattener type is \"jq\" — i.e. a spec with flattenSpec using JSON_QUERY-style/jq expressions — causing makeJsonQueryExtractor(String) to be invoked during field extraction.","commonSituations":"Copying a flattenSpec written for Kafka JSON records (which supports jq) to an Avro stream; misremembering that jq flattening works only for JSON-based formats.","solutions":["Change the flattenSpec flattener to \"jsonPath\" or \"json\" (use \"path\"-style expressions) instead of jq.","Remove the flattenSpec and instead reference Avro fields directly (root fields are supported).","Pre-decode Avro to JSON in a separate step if JQ transformations are essential, then ingest the JSON."],"exampleFix":"// before\n\"flattenSpec\": {\"useFieldDiscovery\": true, \"flattener\": {\"type\": \"jq\", \"expr\": \".foo\"}}\n// after\n\"flattenSpec\": {\"useFieldDiscovery\": true, \"flattener\": {\"type\": \"jsonPath\", \"expr\": \"$.foo\"}}","handlingStrategy":"validation","validationCode":"// Validate flattenSpec before submitting an Avro ingestion job\nif (\"jq\".equals(flattenSpec.getFlattener().getType())) {\n  throw new IllegalArgumentException(\"jq flattener is not supported for Avro; use jsonPath or json\");\n}","typeGuard":"boolean avroSupportsFlattener(FlattenerSpec f) {\n  return !\"jq\".equals(f.getType());\n}","tryCatchPattern":"try {\n  extractor = flattenerMaker.makeJsonQueryExtractor(expr);\n} catch (UnsupportedOperationException e) {\n  throw new IllegalArgumentException(\"Use jsonPath flattener for Avro inputs\", e);\n}","preventionTips":["Only use the jq flattener with JSON-based input formats.","Prefer jsonPath for Avro flattening, including nested fields.","Validate flattenSpec.flattener.type at spec-authoring time (CLI/CI lint) before submitting supervisors."],"tags":["java","avro","flattener","unsupported-operation"],"backgroundTag":"unsupported-operation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}