You are right. The correct translation of
Sehr geehrte Damen und Herren
(dear) ladies and gentlemen
So, »to whom it may concern« seems not to match with »Sehr geehrte Damen und Herren«. But there is a problem with the correct German translation of »to whom it may concern« which is:
Wen auch immer es betrifft
The problem is, that this phrase is extremely impolite.
In »sehr geehrte Damen und Herren« you use a grammatical feature, that is very old and can be found in other languages (like Latin for example) as distinct grammatical case, but in German it is just a stylistic feature: The vocative. This case/feature is used for the only purpose to directly address the recipient of your message. You can't use a vocative for anything else. And you use it to honor the recipient of the message. It is very polite and friendly to start a conversation by addressing the other person directly using a vocative. This is like spreading a red carpet in front of him or her.
But in »wen auch immer es betrifft« you do not address the person you want to talk to. The word »wen« is in accusative case, and it addresses a third person, i. e. anybody else but the person you really want to talk to. This is, as if the person you want to talk to stands directly in front of you, but you look over her or him and shout to the crowd that surrounds you. This is very unfriendly, not to say aggressive.
So, although »wen auch immer es betrifft« is the best matching German translation of »to whom it may concern«, but you never should use it. Use the friendly and respectful phrase »sehr geehrte Damen und Herren« instead.
Btw: I also think that the English phrase is impolite for the same reasons (my personal opinion).