The building is the same in both cases. It is called »Gefängnis« as you already know. But the kind of residence has different names:
- Untersuchungshaft or U-Haft
imprisonment on remand
When you are under suspicion, but not yet convicted
(verb "untersuchen" = "to investigate")
- Strafhaft
imprisonment for sentence
When you are validly convicted
(verb "strafen" = to punish)
In both cases you are a »Häftling« (prisoner, inmate). If you are in Untersuchungshaft, you are a »Untersuchungshäftling« or »U-Häftling«. If you are convinced, you are a »Strafhäftling« or »Sträfling«.
There are Gefängnisse where both kinds of Häftlinge are locked-in. Only a few buildings are only for long-time inmates, i.e. only for Sträflinge. But the majority of Gefängnisse are mixed.
There are also »Hafträume« (also: »Zellen«) (cells) in police stations, but inmates may only stay there for one day before they will be transported to a Gefängnis. Those buildings are not called Gefängnis, but Polizeistation or Wachstube. Their main purpose is to be a police station, i.e. a place where police officers have their offices.