Could someone please explain how to say "of", as in "the book of the man"?
I suspect it is fairly simple but I don't understand the differnce between:
der Buch des Mann
der von dem Mann
(assuming both are correct).
You got it almost right - it's "Das Buch des Mannes" or, more archaic, "Des Mannes Buch".
This is the genitive case, which, in spoken language, is nowadays often replaced by the simpler (but in my opinion still wrong - I recommend you don't use any of these!) dative case, which would be "Das Buch vom / von dem Mann", or even worse, "Dem Mann sein Buch".