Numbers & keys.. they don't always match.
Particularly with NPCs who have additional gossip or act as vendors - in addition to being questgivers.
Say, and this is merely an example really, I went to Rax Sixtrigger in Stormheim, who is a questgiver and offers more than one dialog option, then the numbers supposed to represent the keys don't match (or vice versa, anyway).
https://imgur.com/a/FyoiI
Option 2 is selected by pressing 3, option 3 is selected by pressing 4.
Pressing 1 correctly clicks the quest button/frame.
Ideally this would correspond to pressing 1, 2, 3 instead, same as the numbers.
