"Am Ende der Welt" is (from my experience) just as common as "Am Arsch der Welt". The latter is much more vulgar, yet they mean about the same: some place in the middle of nowhere (but not necessarily distant!).
Thus this does not necessarily refer to the South pole, but could - depending on context - also be a hard-to-reach part of your local town (when e.g. talking about reaching it via public transport). I'd be careful and very reluctant to use "am Arsch der Welt" outside spoken context within a small group of peers whom I know well to appreciate or cope with an occasional vulgar expression used in jest.
I don't see a single unambiguous word which describes Südpol (South Pole) or Antarktis (Antarctica) other than those two geographical words.
As a slighly more poetical expression, and quite a bit clearer than just "am Ende der Welt" one can use "am anderen Ende der Welt" or "auf der anderen Seite der Welt" which from a middle European perspective must be somewhere on the Southern hemisphere. Mind, it's broad enough that I've been using that to refer to Australia or New Zealand (depending on context) without second thought.