Yes, at least in Standard German it is limited to lunch and dinner. For dinner, the analogous expression is "zu Abend essen". But you wouldn't say "*zu Frühstück essen".
Also I found this DaF forum, which points out that you wouldn't say "*zu Mittag kochen" (prepare lunch), but you can say "ich koche zu Mittag Suppe".
For other contexts, "am Mittag" is okay. The preferred way however is "mittags" or "jeden (Montag, Dienstag, …) Mittag" for regular activities around noon, and a specific "heute Mittag", "morgen Mittag", "gestern Mittag" for any of these 3 days.
According to general rules, it would logically be zum. However, it's one of the various cases of a Nullartikel. I don't think it's very logical but just idiomatic.
Similarly, you can ask:
Was wollen wir zu Ostern essen?
Schenken wir uns zu Weihnachten etwas?
But:
Zur Pensionierung backe ich dir einen Kuchen.
Zum Geburtstag viel Glück.
On an interesting albeit confusing side note: Acc. to wiktionary, "zu Mittag essen" is Austrian. My personal experience tells me that people in West Germany, e.g. Cologne, also say "zu Mittag essen". In contrast, Duden states that it is the alternative "mittagessen" that is Austrian.