I'm with you - and also not part of gun culture, so maybe that's it. The term pistol makes me think of dueling at dawn ...
Those would have been revolvers, but ya know, close enough for government work. Honestly, people usually name them by model so you know what they're talking about in context. Unless you were just going out for target practice and took several, then you'd probably say handguns.