{"record":{"id":"a1c6c8ffa0b99812","repo":"xai-org/x-algorithm","slug":"the-value-of-top-feedforward-specified-top-feedf","errorCode":null,"errorMessage":"The value of top_feedforward specified ({top_feedforward}) does not match that in the checkpoint for {clip_model_type} ({config['top_feedforward']}).","messagePattern":"The value of top_feedforward specified \\((.+?)\\) does not match that in the checkpoint for (.+?) \\((.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"clip/model.py","lineNumber":123,"sourceCode":"        self._device = \"cuda:0\" if self.use_gpu else \"cpu\"\n        if self.use_gpu:\n            self._prepare_model_for_gpu()\n\n        self.clip_model.logit_scale.to(self._device)\n\n        if clip_model_type in self.TWITTER_MODELS:\n            map_location = None if self.use_gpu else torch.device(\"cpu\")\n            local_model_path = io_utils.maybe_download_file(\n                self.TWITTER_MODELS[clip_model_type]\n            )\n            print(f\"loading model from: {local_model_path}\")\n            checkpoint_contents = torch.load(\n                local_model_path, map_location=map_location\n            )\n\n            config = checkpoint_contents[\"config\"]\n            if config[\"top_feedforward\"] != top_feedforward:\n                raise ValueError(\n                    f\"The value of top_feedforward specified ({top_feedforward}) does not match that in the \"\n                    f\"checkpoint for {clip_model_type} ({config['top_feedforward']}).\"\n                )\n\n            if self._multi_gpu:\n                image_state_dict = checkpoint_contents[\n                    \"image_encoder_state_dict_multi_gpu\"\n                ]\n                text_state_dict = checkpoint_contents[\n                    \"text_encoder_state_dict_multi_gpu\"\n                ]\n            else:\n                image_state_dict = checkpoint_contents[\"image_encoder_state_dict\"]\n                text_state_dict = checkpoint_contents[\"text_encoder_state_dict\"]\n\n            self.image_encoder.load_state_dict(image_state_dict, strict=True)\n            self.text_encoder.load_state_dict(text_state_dict, strict=True)\n","sourceCodeStart":105,"sourceCodeEnd":141,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/clip/model.py#L105-L141","documentation":"When loading a fine-tuned checkpoint, Model.__init__ compares the top_feedforward hyperparameter you pass to the value stored in the checkpoint's config; a mismatch means the caller's model architecture would differ from the trained one, so weight loading would be invalid and it raises.","triggerScenarios":"Instantiating Model with top_feedforward=2048 while the checkpoint for that clip_model_type was trained with top_feedforward=1024; changing the hyperparameter in config without retraining; loading someone else's checkpoint with your own defaults.","commonSituations":"Hyperparameter tuning changed top_feedforward but old checkpoints are still loaded; shared checkpoint files across experiments with different heads; defaults in code drifted from the exported config.","solutions":["Open the checkpoint's config dict and read config['top_feedforward'], then pass exactly that value","Or do not override top_feedforward; let the code take the checkpoint's value","If you intentionally changed the architecture, retrain and export a new checkpoint","Keep a single source of truth for hyperparameters alongside checkpoints"],"exampleFix":"# before\nModel(clip_model_type='ViT-B/32-finetuned', top_feedforward=4096)\n# after\nModel(clip_model_type='ViT-B/32-finetuned', top_feedforward=1024)  # from checkpoint config","handlingStrategy":"validation","validationCode":"ckpt = torch.load(local_model_path, map_location='cpu')\nexpected_ff = ckpt['config']['top_feedforward']\nassert top_feedforward == expected_ff, f\"checkpoint has {expected_ff}\"","typeGuard":null,"tryCatchPattern":"try:\n    Model(clip_model_type=t, top_feedforward=ff)\nexcept ValueError as e:\n    if 'top_feedforward' in str(e):\n        ff = torch.load(path)['config']['top_feedforward']\n        Model(clip_model_type=t, top_feedforward=ff)\n    else:\n        raise","preventionTips":["Read hyperparameters from the checkpoint config instead of duplicating them","Store training config alongside checkpoints and load it","Add tests that construct models from every shipped checkpoint"],"tags":["python","clip","pytorch","checkpoint","hyperparameter-mismatch"],"backgroundTag":"checkpoint-config-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}