Adding the suffix -weise, or -erweise, which is basically a Genitive plus weise, will create an adverb that often has a slightly different meaning than the adverb/adjective that you used for it. Thus, you shoudld only add it, if that is what you want to express.
Er sagt mir freundlich, warum er mein Bier getrunken hat.
This freundlich is used as an adverb that directly describes how the action is performed... it could be translated as in a friendly way.
Er sagt mir freundlicherweise, warum er mein Bier getrunken hat.
Freundlicherweise is more of a comment-adverb for the whole activity in this case. Other options for the same slot would be zum Glück or endlich. It has an ironic quality and I don't really know how to translate it to English.
Anyway... the same idea applies to other adverbs like großzügigerweise, dummerweise, glücklicherweise, and so on. Without -erweise they quite literally qualify the verb, with -erweise they are more of a comment adverb.
While you can add -erweise to every lexical adjective in theory, it doesn't always make sense to do so. Either because the adverb is already commenting, or because a commenting adverb (which is what -erweise creates) doesn't make much sense, or at least no difference to the one without the suffix.
In case of regelmäßigerweise a reading as a comment adverb doesn't make much sense.