I don't get how the word "spätestens" makes sense in this context? Does it mean that they can increase the price in the first two months, but not later then that?
Yes, I would also understand that this means that the other party can increase the price until two months after the contract was signed.
This is a strange wording in AGBs. I could imagine that in some particular long-running contract, where the final costs are unclear at signature time, but not in general. Also, it is accepted that prices based on offers might change (usually by at most 10%) if the offer was to low and there is a reason given for it. (i.e. the bricklayer found that he had to rebuild the whole structure below the new wall he was supposed to build).
Apparently, this clause tries to reserve the right of the contractor to change pricing after contract closure. It is, however, put somewhat awkwardly. Whether it does or doesn't make sense - depends on the type of contract. You would normally assume a fixed price is only valid up to a certain time limit, thus limiting price validity exactly the other way round.
I suspect there is a little "bis" gone astray in the sentence that could have made the clause a bit more clear:
Preiserhöhungen bis spätestens zwei Monate nach Vertragsabschluss müssen wir uns vorbehalten.
Would mean "we might be forced to increase prices and reserve our right to do that, but limited to two months after contract signage."