{"record":{"id":"34a32f93d594a23a","repo":"prestodb/presto","slug":"mv-missing-too-much-data","errorCode":"MV_MISSING_TOO_MUCH_DATA","errorMessage":"%s misses too many partitions or is never refreshed and may incur high cost. Consider refreshing with predicates first.","messagePattern":"(.+?) misses too many partitions or is never refreshed and may incur high cost\\. Consider refreshing with predicates first\\.","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-main-base/src/main/java/com/facebook/presto/sql/analyzer/StatementAnalyzer.java","lineNumber":1072,"sourceCode":"\n        private Map<SchemaTableName, Expression> analyzeAutoRefreshMaterializedView(\n                RefreshMaterializedView node,\n                QualifiedObjectName viewName)\n        {\n            MaterializedViewStatus viewStatus = session.getRuntimeStats().recordWallTime(\n                    RuntimeMetricName.GET_MATERIALIZED_VIEW_STATUS_TIME_NANOS,\n                    () -> metadataResolver.getMaterializedViewStatus(viewName, TupleDomain.all()));\n            Map<SchemaTableName, MaterializedViewStatus.MaterializedDataPredicates> missingPartitionsPerTable =\n                    viewStatus.getPartitionsFromBaseTables();\n\n            if (viewStatus.isFullyMaterialized() || missingPartitionsPerTable.isEmpty()) {\n                warningCollector.add(new PrestoWarning(SEMANTIC_WARNING,\n                        format(\"Materialized view %s is already fully refreshed\", viewName)));\n                return ImmutableMap.of();\n            }\n            if ((viewStatus.isNotMaterialized() || viewStatus.isTooManyPartitionsMissing()) &&\n                    !SystemSessionProperties.isMaterializedViewAllowFullRefreshEnabled(session)) {\n                throw new PrestoException(MV_MISSING_TOO_MUCH_DATA,\n                        format(\"%s misses too many partitions or is never refreshed and may incur high cost. \" +\n                                \"Consider refreshing with predicates first.\", viewName.toString()));\n            }\n\n            return MaterializedViewUtils.generatePredicatesForMissingPartitions(missingPartitionsPerTable, metadata);\n        }\n\n        private Optional<RelationType> analyzeBaseTableForRefreshMaterializedView(Table baseTable, Optional<Scope> scope)\n        {\n            checkState(analysis.getStatement() instanceof RefreshMaterializedView, \"Not analyzing RefreshMaterializedView statement\");\n\n            RefreshMaterializedView refreshMaterializedView = (RefreshMaterializedView) analysis.getStatement();\n            QualifiedObjectName viewName = createQualifiedObjectName(session, refreshMaterializedView.getTarget(), refreshMaterializedView.getTarget().getName(), metadata);\n\n            // Use AllowAllAccessControl; otherwise Analyzer will check SELECT permission on the materialized view, which is not necessary.\n            StatementAnalyzer viewAnalyzer = new StatementAnalyzer(analysis, metadata, sqlParser, new AllowAllAccessControl(), session, warningCollector);\n            Scope viewScope = viewAnalyzer.analyze(refreshMaterializedView.getTarget(), scope);\n","sourceCodeStart":1054,"sourceCodeEnd":1090,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-main-base/src/main/java/com/facebook/presto/sql/analyzer/StatementAnalyzer.java#L1054-L1090","documentation":"Presto throws this while executing REFRESH MATERIALIZED VIEW when the materialized view is missing too many of its source partitions (or has never been refreshed at all) and the session does not allow a full refresh. A full refresh in that state would rebuild the entire view, which can be extremely expensive. The engine refuses and suggests running a partitioned (predicated) refresh instead.","triggerScenarios":"Running REFRESH MATERIALIZED VIEW on a view whose connector-reported status returns isNotMaterialized() or isTooManyPartitionsMissing(), while SystemSessionProperties.isMaterializedViewAllowFullRefreshEnabled(session) is false.","commonSituations":"First-ever refresh of a newly created materialized view whose base tables are large; a view that has not been refreshed for a long time so most partitions became stale/missing; clusters where the allow-full-refresh session property is deliberately disabled to protect resources.","solutions":["Run a refresh with predicates over subsets of partitions: REFRESH MATERIALIZED VIEW mv WHERE <partition column> BETWEEN ... and iterate until fully caught up.","Set the session property SET SESSION allow_full_refresh_for_materialized_view = true (or the equivalent SystemSessionProperties flag) if a full refresh is acceptable.","Schedule regular incremental refreshes so the missing-partition count never crosses the threshold.","Verify the connector's staleness/partition-missing reporting is correct (e.g., stale partition metadata) if the view is actually fresh."],"exampleFix":"// before\nREFRESH MATERIALIZED VIEW sales_mv;\n// after\nREFRESH MATERIALIZED VIEW sales_mv WHERE day >= DATE '2026-08-01';\n-- or enable full refresh for this session:\nSET SESSION allow_full_refresh_for_materialized_view = true;\nREFRESH MATERIALIZED VIEW sales_mv;","handlingStrategy":"validation","validationCode":"// Check staleness before refreshing\nRow stale = query(\"SELECT count(*) FROM \" + mvStatusTable + \" WHERE missing_partitions > threshold\");\nboolean needsPredicatedRefresh = stale.count > 0 && !session.getAllowFullRefresh();","typeGuard":null,"tryCatchPattern":"try {\n    refreshMaterializedView(mv);\n} catch (PrestoException e) {\n    if (\"MV_MISSING_TOO_MUCH_DATA\".equals(e.getErrorCode().getName())) {\n        refreshWithPartitionPredicates(mv); // incremental fallback\n    } else throw e;\n}","preventionTips":["Schedule frequent incremental refreshes with WHERE predicates on partition columns.","Do not blindly enable allow_full_refresh on large views; budget refresh cost first.","Monitor missing-partition counts per materialized view and alert before the threshold is hit."],"tags":["presto","materialized-view","refresh","semantic-analysis"],"backgroundTag":"materialized-view-full-refresh-blocked","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}