Can it really be used to mean "homonym" outside the scope of the game?
People who know the game will use it informally, not in written speech and it can easily happen that people don't know the game. Personally, I will only use it among my family who played the game with me when I was a child.
Maybe it is comparable to people using "Simon says" in informal speech.
I would like to offer a different point of view from the existing answers. The word »Teekesselchen« only refers to the game, whereas »Teekessel« could be both the game and a tea kettle (not a teapot). Of course anybody who knows German well enough notices that it’s a diminutive form of »Teekessel« and might very well imagine a small tea kettle (especially if they don’t know the game), but it would only be in a very specific context involving a small tea kettle where the word carries that meaning.
I can’t give a similar example off the top of my head because e.g. »Schnittchen« is the diminutive form of »Schnitte« (slice of bread) and not »Schnitt« (cut).
More specifically towards the question sentence in the question body: No, outside of the game you wouldn’t use it a a synonym for »Homonym«, unless you’re talking to younger kids who are familiar with the game (and thus the concept of the same word carrying two or more meanings), but not the word »Homonym«.
I wasn't aware it's a "game" (other than noun being a "game" with the goal of listing as many nouns as you can think of, and verb being a "game" with the goal of listing as many verbs as you can think of, etc.), but the term was routinely used where I went to elementary school in South-Western Germany many years ago and seemed to be familiar to elementary school teachers among my relatives from Northern Germany, as well.
I am not sure Teekesselchen is an exact synonym for homonym, though; I always felt ein Teekesselchen already denotes the pair of homonymous words.