In this context "an die zwanzig" means approximately "nearly, close to twenty". It's different from um "around", but either way, both mean "not exactly". In the same manner "so" can be used alone, "so zwanzig".
The combination of both "so an" is peculiar, but hardly unique; Many such modal collocations exist and they mean whatever I want them to mean; That is, if I wanted to be precise I'd better choose another expression.This could in principle allow for various reinterpretation to have happened and invitesbspeculation. First of all note that approximately "ad-prox-imate" reflects the same idea and "they are at two times a year now" implies a potential for increase, even.
Other instances notwithstanding, I'd compare this "so" to Eng "say" ("e.g.", "not precisely") as in "twenty eggs say", "suppose you had some eggs, say twenty", but we have as well "sagen wir zwanzig", while "so" has a general proto-indo-european etymology in *so. I'm just not sure whether this collocation had any cognates. So a few further notes may help to extract my understanding of it:
I suppose that's an orphaned variant of subjunctive "sein", Old English "sea", but I'm not so sure; also compare some and perhaps selbst "even".
It's notable that "an die" requires an article, but a sole "so" doesn't.
It's also notable that "so an die 20, 30 Mann" modulo the numbers is a fixed expression as the primary example of the collocation that is otherwise quite rare (therefore cp. "some" and "assembly"). If "an" in this expression derived from "Ahn", then could "so Ahn" mean "the men by themselves, without further ado", "ohne Anhang, ohne entfernte Verwandte"? I don't know. In contrast, other pronouns in similar positions are not rare ("gegen fünf Uhr", "nach drei", "unter zehn").
We also see also "Es fühlt sich so an wie ...", next to "~ an wie" "~ an so wie" etc, in which "an" is part of "anfühlen" (cp "em-pathy", and "empfehlen"?) while "so" is a determiner, or interrogative particle; "so ... wie" may bracket an expression (an outfix operator like "as ...as", or Fr "ne ... pas"), so "so an die fünf mal wie ..." is imaginable.