{"record":{"id":"08b9c154ec790a69","repo":"tensorflow/models","slug":"distribution-strategy-is-not-specified","errorCode":null,"errorMessage":"Distribution strategy is not specified.","messagePattern":"Distribution strategy is not specified\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"official/legacy/bert/run_pretraining.py","lineNumber":157,"sourceCode":"      steps_per_loop=steps_per_loop,\n      epochs=epochs,\n      sub_model_export_name='pretrained/bert_model',\n      explicit_allreduce=explicit_allreduce,\n      pre_allreduce_callbacks=pre_allreduce_callbacks,\n      post_allreduce_callbacks=post_allreduce_callbacks,\n      allreduce_bytes_per_pack=allreduce_bytes_per_pack,\n      train_summary_interval=train_summary_interval,\n      custom_callbacks=custom_callbacks)\n\n  return trained_model\n\n\ndef run_bert_pretrain(strategy, custom_callbacks=None):\n  \"\"\"Runs BERT pre-training.\"\"\"\n\n  bert_config = configs.BertConfig.from_json_file(FLAGS.bert_config_file)\n  if not strategy:\n    raise ValueError('Distribution strategy is not specified.')\n\n  # Runs customized training loop.\n  logging.info('Training using customized training loop TF 2.0 with distributed'\n               'strategy.')\n\n  performance.set_mixed_precision_policy(common_flags.dtype())\n\n  # Only when explicit_allreduce = True, post_allreduce_callbacks and\n  # allreduce_bytes_per_pack will take effect. optimizer.apply_gradients() no\n  # longer implicitly allreduce gradients, users manually allreduce gradient and\n  # pass the allreduced grads_and_vars to apply_gradients().\n  # With explicit_allreduce = True, clip_by_global_norm is moved to after\n  # allreduce.\n  return run_customized_training(\n      strategy,\n      bert_config,\n      FLAGS.init_checkpoint,  # Used to initialize only the BERT submodel.\n      FLAGS.max_seq_length,","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/tensorflow/models/blob/e006f5f0d534913e49c1f1dae87364039fa607e2/official/legacy/bert/run_pretraining.py#L139-L175","documentation":"Guard in run_bert_pretrain: the strategy argument is None/falsy, so no distribution strategy exists for pre-training. It usually means main() did not build a strategy (e.g. distribution_utils.get_distribution_strategy returned None or was skipped) before calling the pre-training routine.","triggerScenarios":"Thrown at official/legacy/bert/run_pretraining.py:157 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Create the strategy in main: strategy = distribution_utils.get_distribution_strategy(FLAGS.strategy_type) and pass it into run_bert_pretrain","Verify the strategy_type/ tpu flags so a non-None strategy is produced"],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"e006f5f0d534913e49c1f1dae87364039fa607e2","analyzedAt":"2026-08-24T14:09:15.576Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}