{"record":{"id":"4ed9dd78473cca1f","repo":"apache/druid","slug":"index-metadata-doesn-t-exist-for-segment-s-try","errorCode":null,"errorMessage":"Index metadata doesn't exist for segment [%s]. Try providing explicit rollup, queryGranularity, dimensionsSpec, and metricsSpec.","messagePattern":"Index metadata doesn't exist for segment \\[(.+?)\\]\\. Try providing explicit rollup, queryGranularity, dimensionsSpec, and metricsSpec\\.","errorType":"exception","errorClass":"RE","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java","lineNumber":1037,"sourceCode":"    public void fetchAndProcessIfNeeded()\n    {\n      if (!shouldDownloadSegments()) {\n        // Nothing to do; short-circuit and don't fetch segments.\n        return;\n      }\n\n      multiValuedDimensions = new HashSet<>();\n      final List<Pair<DataSegment, Supplier<ResourceHolder<QueryableIndex>>>> segments = sortSegmentsListNewestFirst();\n\n      for (Pair<DataSegment, Supplier<ResourceHolder<QueryableIndex>>> segmentPair : segments) {\n        final DataSegment dataSegment = segmentPair.lhs;\n\n        try (final ResourceHolder<QueryableIndex> queryableIndexHolder = segmentPair.rhs.get()) {\n          final QueryableIndex index = queryableIndexHolder.get();\n\n          if (index != null) { // Avoid tombstones (null QueryableIndex)\n            if (index.getMetadata() == null) {\n              throw new RE(\n                  \"Index metadata doesn't exist for segment [%s]. Try providing explicit rollup, \"\n                  + \"queryGranularity, dimensionsSpec, and metricsSpec.\", dataSegment.getId()\n              );\n            }\n\n            processRollup(index);\n            processQueryGranularity(index);\n            processDimensionsSpec(index);\n            processMetricsSpec(index);\n            processMultiValuedDimensions(index);\n            processProjections(index);\n          }\n        }\n      }\n    }\n\n    public Boolean getRollup()\n    {","sourceCodeStart":1019,"sourceCodeEnd":1055,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java#L1019-L1055","documentation":"CompactionTask.ExistingSegmentAnalyzer reads segment metadata to infer rollup/queryGranularity/dimensions/metrics. Older segments (or tombstone-adjacent storage) can lack index metadata; without it the analyzer cannot auto-derive the compaction spec, so it throws a Redis/RE telling the user to specify those fields explicitly.","triggerScenarios":"Compacting segments written by very old Druid versions (pre-metadata) or otherwise missing QueryableIndex.getMetadata(); running auto-compaction over segments lacking metadata and no explicit dimensionsSpec/metricsSpec/rollup/queryGranularity in the compaction config.","commonSituations":"Clusters upgraded across major versions with long-lived historical segments; segments produced by external tools; compaction configs relying on auto-detection for legacy data.","solutions":["Provide explicit rollup, queryGranularity, dimensionsSpec, and metricsSpec in the compaction tuningConfig/spec so metadata isn't needed.","Re-ingest the legacy segments with a current Druid version to regenerate metadata, then compact.","Exclude the affected segments/intervals from compaction until they are rewritten."],"exampleFix":"// before: relying on auto-detection\n{ \"type\": \"compaction\", ... } // no granularitySpec\n// after\n{\n  \"granularitySpec\": {\n    \"segmentGranularity\": \"day\",\n    \"queryGranularity\": \"none\",\n    \"rollup\": false\n  },\n  \"dimensionsSpec\": { \"dimensions\": [\"dim1\"] },\n  \"metricsSpec\": [{ \"type\": \"count\", \"name\": \"count\" }]\n}","handlingStrategy":"fallback","validationCode":"// pre-check one segment\nQueryableIndex idx = ...;\nif (idx == null || idx.getMetadata() == null) {\n  // require explicit rollup/queryGranularity/dimensionsSpec/metricsSpec in compaction config\n}","typeGuard":null,"tryCatchPattern":"try { runCompaction(spec); } catch (RE e) { if (e.getMessage().contains(\"Index metadata doesn't exist\")) { add explicit schema fields and rerun; } else throw e; }","preventionTips":["For segments from old Druid versions, always specify the full schema in compaction specs.","Re-ingest legacy data to regenerate index metadata before enabling auto-detection.","Spot-check QueryableIndex.getMetadata() on a sample segment before configuring auto-compaction."],"tags":["druid","compaction","segment-metadata","legacy"],"backgroundTag":"missing-required-config-field","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}