Bugs of Batch Inference Code
Hi !
Hi! When running batch inference code, the order of the returned matching_scores is incorrect. For example, if the input batch order is (0, 1, 2), the returned result might be (0, 2, 1). This is due do the early stopping mechanism of Transformers Layers in the paper, which shuffles the batch index. Yet the code of recovering the batch index is incorrect.
Here is the Bug-Fixed code.
at modeling_lightglue.py, line639
idx_map = {early_stops_indices[i].item(): i for i in range(early_stops_indices.size()[0])} # debug
idx_new = [idx_map[i] for i in range(early_stops_indices.size()[0])] # debug
matches, matching_scores, final_pruned_keypoints_indices, final_pruned_keypoints_iterations = (
tensor[idx_new] # tensor[early_stops_indices], debug
for tensor in [
matches,
matching_scores,
final_pruned_keypoints_indices,
final_pruned_keypoints_iterations,]
)
Hi
@cm5105
,
Thanks for highlighting this issue, do you have a reproducible example so I can investigate on my side and see the changes ?
Hi @cm5105 ,
Thanks for highlighting this issue, do you have a reproducible example so I can investigate on my side and see the changes ?
@stevenbucaille
Sry, the data belongs to the company and I cannot share it externally.
However, there is a way to reproduce the bug: find three pairs of images where the first two pairs are very different, but the third pair is very similar. Then run batch inference and, within the _concat_early_stopped_outputs function, print the early_stops_indices. In this case, early_stops_indices will most likely be [4, 5, 0, 1, 2, 3]. This happens because the third pair of images will trigger the early stopping mechanism first. Then you can see the changes.