Why is it Sie instead of Ihnen here?
Kann ich Sie kurz sprechen?
Shouldn't it be like this:
Kann ich Ihnen kurz sprechen?
Can I speak TO YOU or WITH YOU for a moment? Both should be in dative case.
The German verb phrase in question does neither use zu nor mit as you can easily see. It's jemanden sprechen, hence accusative case. There's also the verb phrases mit jemandem sprechen and zu jemandem sprechen but neither of both is relevant here.
It's true that German dative objects can often be replaced by to X in English. But that's not always the case as English knows indirect objects. And neither can an English to X always be replaced by a dative object in German. Those are only rules of thumb.
It's true that this is a somewhat special construction. "Jemanden sprechen" (instead of "mit jemandem") means to talk to someone in a formal context, when this person has a particular office or duty or capacity, and you are pursuing a goal which relates to that.