The preposition does not matter in my experience. Having grown up in Southern Germany, geduldig mit is a little more common than the geduldig gegenüber, but both are correct. Dito for Geduld mit/gegenüber.
The benefactor of patience is more interesting here, I think. While there might be special texts using it that way, I don't think you can have Geduld with something that is not a person, institution, group etc. One of the commenters pointed out you can have Geduld mit/gegenüber diseases, the weather, natural events etc, but I disagree in the sense that I have not heard or read someone using it that way and find it odd.
This might have changed in time.
Particularly the given example "Die Geduld gegenüber den weltlichen Plagen" sounds quite strange to me. My reference point is 2015 Standard German as spoken in Munich though, so in special domains or in past versions of the language, it might have been valid to connect Geduld mit with a thing, concept etc.