Just place your detector rails on top of a colored block (wool, concrete, concrete powder, terracotta, glazed terracotta) and it will not activate when players are passing by without them holding that exact item in their main hand.

Discord