{"record":{"id":"094ef947f85c038d","repo":"instructure/canvas-lms","slug":"kafka-events-has-brokers-configured-but-required-topic-s","errorCode":null,"errorMessage":"kafka_events has brokers configured but required topic(s) missing from DynamicSettings: #{missing.join(\", \")}","messagePattern":"kafka_events has brokers configured but required topic\\(s\\) missing from DynamicSettings: #(.+?)","errorType":"validation","errorClass":"ConfigError","httpStatus":null,"severity":"error","filePath":"lib/canvas/kafka_events/config.rb","lineNumber":85,"sourceCode":"        \"socket.keepalive.enable\": \"true\",\n        \"linger.ms\": \"50\",\n        \"message.timeout.ms\": message_timeout_ms.to_s,\n        \"queue.buffering.max.messages\": queue_buffering_max_messages.to_s,\n        \"reconnect.backoff.ms\": reconnect_backoff_ms.to_s,\n        \"reconnect.backoff.max.ms\": reconnect_backoff_max_ms.to_s,\n      }.compact\n    end\n\n    def validate!\n      if sasl_configured? && !sasl_credentials_complete?\n        raise ConfigError,\n              \"kafka_events connection requires both sasl_username and sasl_password when SASL is in use\"\n      end\n\n      missing = Events.topic_keys.reject { |key| topic_for(key).present? }\n      return if missing.empty?\n\n      raise ConfigError,\n            \"kafka_events has brokers configured but required topic(s) missing from DynamicSettings: #{missing.join(\", \")}\"\n    end\n\n    private\n\n    def load_operational_config\n      parsed = YAML.safe_load(DynamicSettings.find(tree: :private)[\"kafka_events.yml\", failsafe: nil] || \"{}\")\n      parsed.is_a?(Hash) ? parsed : {}\n    rescue => e\n      Rails.logger.warn(\"Kafka operational settings load error: #{e.message}\")\n      {}\n    end\n\n    def load_connection_config\n      Rails.application.credentials.kafka_events&.with_indifferent_access || {}\n    end\n  end\nend","sourceCodeStart":67,"sourceCodeEnd":103,"githubUrl":"https://github.com/instructure/canvas-lms/blob/1c9f0bb8013ed69c4f2efe11fd483025469b7e6c/lib/canvas/kafka_events/config.rb#L67-L103","documentation":"Canvas::KafkaEvents::Config#validate! checks every required topic key (Events.topic_keys) resolves to a non-empty topic name via topic_for (sourced from DynamicSettings). If brokers are configured but any required topic name is missing, ConfigError is raised listing the missing topic keys.","triggerScenarios":"Instantiating/validating Config when kafka brokers are set but DynamicSettings lacks topic names for one or more required event topic keys (e.g. after adding a new event type without provisioning its topic key).","commonSituations":"New Canvas version requires new topics; consul/DynamicSettings tree not updated; typo in topic key under the kafka_events namespace; environment missing the topics: block.","solutions":["Add the listed topic names under kafka_events topics in DynamicSettings for the environment","Create the corresponding topics in the Kafka cluster and match names in config","Run the canvas task/console to dump Canvas::KafkaEvents::Config required topic keys and verify each is present","After adding new event types in an upgrade, re-sync the config tree from the documented sample"],"exampleFix":"# before (DynamicSettings kafka_events)\nbrokers: [kafka1:9092]\n// after\nbrokers: [kafka1:9092]\ntopics:\n  live_events: canvas-live-events\n  migrations: canvas-migrations","handlingStrategy":"validation","validationCode":"missing = Canvas::KafkaEvents::Events.topic_keys.reject { |k| config.topic_for(k).present? }\nraise \"missing topics: #{missing.join(',')}\" if missing.any?","typeGuard":"ok = topic_keys.all? { |k| topics.is_a?(Hash) && topics[k.to_s].present? }","tryCatchPattern":"begin\n  Canvas::KafkaEvents::Config.validate!\nrescue Canvas::KafkaEvents::ConfigError => e\n  provision_missing_topics(e.message)\n  raise\nend","preventionTips":["Keep a checked-in reference kafka_events settings file listing all topic keys","Automate topic provisioning to match required keys after upgrades","Add a config linter that compares Events.topic_keys with provisioned settings"],"tags":["kafka","configuration","dynamic-settings"],"backgroundTag":"missing-required-config-field","analyzedSha":"1c9f0bb8013ed69c4f2efe11fd483025469b7e6c","analyzedAt":"2026-09-15T20:33:18.891Z","contentChangedAt":"2026-09-15T20:33:18.891Z","schemaVersion":2},"datasetVersion":"2026-09-23T02:17:17.105Z"}