Yeah, I figure most that don't flag will either be looking for a fight, or are well trained in pickpocketing.

Even people that claim they won't flag, will likely flag on all their alts, and it's people being lazy with their alts that even gives you any decent targets at all anymore.