There are lots of remnants that can be traced back to Celtic languages in German - Especially in place names and names of rivers (Rhein, Main, Lech, Inn, even Donau). A typical example could be place names in southern Germany that start on "Kräh-" (like Krähbühl, for example) and don't normally trace back to the birds, but rather to the Celtic craig (rock).
But there are also just "normal" words that can be traced back to Celtic, like
- Amt - ambactos - "office" or "charge"
- Apfel - afal(?) - "apple"
- Eibe - eburo - "yew"
- Beil - bitlon - "hatchet"
- Eisen - isarnon - "iron"
- Geisel - gistlos - "hostage"
- Glocke - cloccos - "bell"
- Leder - letro - "leather"
- Leinen - linno - "linnen"
Some of them made it into today's English as well, probably from the same Celtic roots. From a lot of these terms it is not always easy to trace them directly to Celtic origin or whether they have to be treated as "re-imports" from other languages.
Obviously, there are Celtic terms like Menhir and Dolmen that are used for the same thing in German - I wouldn't consider them as direct holdovers of Celtic in German, however.
Wikipedia has a page of Celtic remnants in various European languages, including the interesting claim the 20-based number system in French was actually of Celtic origin.