The chance-corrected framing is a useful correction to the raw-accuracy headline. With 20 scenarios and one trial each, a 55% vs 75% difference has a fairly wide confidence interval (a Wilson interval on n=20 is roughly +/-20 points), so repeating each pair a few times at temperature above 0 might show how much of the 25 to 50 cliff is noise. The find_contact result also made me wonder about two things: whether its description is broader than the others (a generic 'look up a person' tool can act as an attractor for any request that mentions a name), and whether menu order matters. Shuffling tool order between runs would separate a genuine attractor from position bias. Did you try either, or test a retrieval step that pre-filters the menu to the top-k tools?