By suffixing verbs with "-ung" we can build nouns, e.g.:
prüfen - Prüfung
meinen - Meinung
richten - Richtung
All those nouns seem to have a female gender. Is that always true or do we have exceptions from that rule?
AFAIK this is always true: all nouns ending with suffix "-ung" have female gender. There are some notes on the conversion verb - noun using that suffix at canoo.net. Furthermore, elexiko allows for searching for words sharing a certain suffix. The search returned zero matches for nouns ending on "ung" with male or neutral gender; only with gender set to female, elexiko returns matches (73, to be precise). So i assume there are in fact no nouns with other than female gender ending on "ung".
I think that this is true. You have also some suffixes that require a male gender.