coco_dt = coco_gt.loadRes(coco_results)
print('result loaded.')
coco_eval = COCOeval(coco_gt, coco_dt, iou_type)
# Use the COCO default keypoint OKS sigmas unless overrides are specified
if kpt_oks_sigmas:
coco_eval.params.kpt_oks_sigmas = np.array(kpt_oks_sigmas)
coco_eval.evaluate()
loadRes load segmentation mask as uncompressed RLEs, which indicates in cocoapi code,
but evaluate() call, reading mask as polygons.
So how to make them same? Why doesn't using the same API?
@niazahamd89 Hi, I have found that, loadRes
is loading from uncompressed RLEs is reasonable, since when evaluation all results contains in memory, if using mask image array that would be very big.
And when evaluate, actually, it loads Gt and Dt all as polygons, it will convert to mask internal.
So seems reasonable here, am I understand right?
you are right @jinfagang but what i want to clear is i train my model on coco train and validation set for pose estimation now i want to evaluate my model to find the AP and AR. i am going to use coco evaluation metric for this purpose, in demo.ipynb the first file i need is the Gt file which i downloaded from coco site the second file i need the Dt file now i am confuse how i can generate this Dt file? my training is complete the model generate weights file in .ckpt format and i test the model is able to plot pose in pictures.
there are some result.json files in cocoapi result folder https://github.com/philferriere/cocoapi/tree/master/results basically the same file person_keypoints_val2014_fakekeypoints100_results.json i want to generate for my model to evaluate.
did you went through this process? did you generate these files for your model evaluation?
@xuluxulu Hi, I also have a question about the variable score
in the json file. I think score
means model confidence score in object detection. But I don't know what it means in person keypoints detection. Have you solved this problem? Any reply or help would be appreciate.
I figured it out several days ago by debug mode. For keypoints problem, It means the average confidence score for all visible keypoints. Like the code here, it means the average confidence score of all visible keypoints(t_s > in_vis_thre
), in which t_s
is the confidence of a single keypoint. t_s
comes from this file, so it actually represents the confidence score.