{"record":{"id":"9b1b1d497deda070","repo":"open-mmlab/mmdetection","slug":"please-run-accumulate-first","errorCode":null,"errorMessage":"Please run accumulate() first","messagePattern":"Please run accumulate\\(\\) first","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"mmdet/datasets/api_wrappers/cocoeval_mp.py","lineNumber":290,"sourceCode":"            stats = np.array(stats)\n            return stats\n\n        def _summarizeKps():\n            stats = np.zeros((10, ))\n            stats[0] = _summarize(1, maxDets=20)\n            stats[1] = _summarize(1, maxDets=20, iouThr=.5)\n            stats[2] = _summarize(1, maxDets=20, iouThr=.75)\n            stats[3] = _summarize(1, maxDets=20, areaRng='medium')\n            stats[4] = _summarize(1, maxDets=20, areaRng='large')\n            stats[5] = _summarize(0, maxDets=20)\n            stats[6] = _summarize(0, maxDets=20, iouThr=.5)\n            stats[7] = _summarize(0, maxDets=20, iouThr=.75)\n            stats[8] = _summarize(0, maxDets=20, areaRng='medium')\n            stats[9] = _summarize(0, maxDets=20, areaRng='large')\n            return stats\n\n        if not self.eval:\n            raise Exception('Please run accumulate() first')\n        iouType = self.params.iouType\n        if iouType == 'segm' or iouType == 'bbox':\n            summarize = _summarizeDets\n        elif iouType == 'keypoints':\n            summarize = _summarizeKps\n        self.stats = summarize()\n","sourceCodeStart":272,"sourceCodeEnd":297,"githubUrl":"https://github.com/open-mmlab/mmdetection/blob/cfd5d3a985b0249de009b67d04f37263e11cdf3d/mmdet/datasets/api_wrappers/cocoeval_mp.py#L272-L297","documentation":"COCOevalMp.summarize() requires that accumulate() has been run first, because summarize reads self.eval['counts'] etc. populated by accumulate. If self.eval is falsy (not yet computed, or empty after evaluating zero results), it raises Exception('Please run accumulate() first'). This mirrors upstream pycocotools COCOeval.summarize behavior in the multiprocessing variant.","triggerScenarios":"Calling summarize() directly after evaluate() but before accumulate(); or calling summarize when evaluate() produced no detections (self.eval empty), which yields the same guard trip even if accumulate was nominally called on empty results.","commonSituations":"Custom eval loops that reorder COCOeval steps; evaluating a checkpoint that predicts nothing (untrained model, wrong classes) so evalImgs is empty; multiprocessing wrapper (cocoeval_mp) used with a subset that skipped accumulation.","solutions":["Ensure the canonical order: coco_eval.evaluate(); coco_eval.accumulate(); coco_eval.summarize()","If it still fires with empty predictions, check that the model actually detects anything (log number of dets) and that ann labels/CATEGORY ids match","Catch the exception when running evaluations where zero predictions are legitimate, and report metrics as N/A"],"exampleFix":"# before\ncoco_eval.evaluate()\nstats = coco_eval.summarize()  # Exception: Please run accumulate() first\n# after\ncoco_eval.evaluate()\ncoco_eval.accumulate()\nstats = coco_eval.summarize()","handlingStrategy":"validation","validationCode":"coco_eval.evaluate()\nif not coco_eval.eval.get('counts', None) and not coco_eval.evalImgs:\n    raise RuntimeError('evaluate() produced no results; check predictions/annotations')\ncoco_eval.accumulate()\ncoco_eval.summarize()","typeGuard":"def is_ready_to_summarize(coco_eval) -> bool:\n    return bool(getattr(coco_eval, 'eval', None)) and bool(getattr(coco_eval, 'params', None))","tryCatchPattern":"try:\n    stats = coco_eval.summarize()\nexcept Exception as e:\n    if 'accumulate' in str(e):\n        coco_eval.accumulate()\n        stats = coco_eval.summarize()\n    else:\n        raise","preventionTips":["Always call evaluate -> accumulate -> summarize in that order, ideally wrapped in one helper","Warn when total detections == 0 after evaluate, since empty evalImgs leads here even with correct ordering","Unit-test eval helpers against a tiny known-good prediction set"],"tags":["mmdetection","coco-eval","evaluation","state-machine","multiprocessing"],"backgroundTag":"method-call-order-violation","analyzedSha":"cfd5d3a985b0249de009b67d04f37263e11cdf3d","analyzedAt":"2026-08-27T20:54:20.183Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}