The phrase is, in essence, correct - it's not "tests" but "proves", because the idea is that if I say "well that's an excpetion", then this means that there must be an underlying rule of which the exception is an exception. This is a Roman legal principle, so the whole saying is derived from Latin:
exceptio probat regulam in casibus non exceptis
which translates to "the exception confirms the rule in cases not excepted". This then gives you the meaning as above - however we (in German or English) tend to not use the second part of the phrase.
So far Wikipedia knowldege. Now, what's odd is that the real source of this (which is supposedly Cicero's "Pro Balbo") does not pop up (on the first google page) when you search for the above Latin phrase. Digging a bit deeper, this is what I found:
Quod si exceptio facit ne liceat, ubi necesse est licere.
in Cicero, Pro Balbo, Caput XII,
which translates to "If the exception makes such an action unlawful, where there is no exception the action must necessarily be lawful." - i.e. a twisted wording of the above (translation not by me).
Another version is
Exceptio firmat regulam in casibus non exceptis
the only difference being that "probare" (prove) is exchanged by "firmare" (confirm, strengthen). Now having a look at this phrase, you get more lucky. This was discussed in normal Roman law throughout the ages, see e.g. here or here.
In conclusio: Above, you find the original phrase used by Cicero. The other versions (exceptio propat/firmat regulam in casibus non exceptis) are derived from there, but were around since Cicero and they mean what Cicero meant and what is more aptly be translated as "Ausnahmen bestätigen die Regel" or "exceptions prove the rule".