What's the exact difference and how should those words be used?
P.S.: Is it when we talk about air travel and land travel?
Abflug (masc.) is the departure of a means of transport that can fly (e.g. an airplane). Abfahrt (fem.) of those that can be driven on the ground (e.g. a car). So, when talking about air travel, use the former, when talking about land travel, the latter. As a neutral word for all kind of travels you could use "Abreise" (fem.).
The literal meaning has been described by AGuyCalledGerald already...
I'd like to add the figure of speech meanings, as far as I know them:
Abflug can be a general departure in "slang"
basically means that I will leave that bar, party or BBQ You can also use it tell people to go away.
Abfahrt is used to describe an intense part of a DJ Set or Song... for example a loooong trance break and then the snare roll and then finally the base kicks in; that would be a reason to scream "Abfaaaaahrt"
For completion Abfahrt is also the correct word in a ski resort both for the downhill track and the action of riding it down. This meaning uses ab as down rather than of