> ... learn, when a verb goes with Dativ and when with Akkusativ?

Use that rule with direct and indirect object as you already know.

> Or in the words of the rule above, how does one know that ich is a
> undirect object for antworten while a direct object for fragen?

You just can imagine the Dativ concerning `antworten` in a way that someone gives information back to someone, and you reduce it from `Antwort geben` to `antworten`. But that's just a guess from my feel for language. A reply gives something back, so that may be your virtual direct object. The same goes with `imponieren`, "make a good impression on someone".