I have two ideas and two guesses to add to the other contributors'contributors’ answers:
German'sGerman’s generic compounding (especially of nouns) which allows juxtaposition of components often without any markers ( example:
Erschwernis
,Zulage
->Erschwernis, Zulage →Erschwerniszulage
Erschwerniszulage (complication
complication,extra pay
->extra pay due to complication
extra pay → extra pay due to complication) puts an extra onus on the listener as the phonetic stress patterns of the words involved are altered.German is a head final language when it comes to subordinate clauses. this might cause a problem because preceding lexical material needs to be retained until the head is encountered. While this does not seem to impose a significant burden onto a native speaker, a foreign language appears to be processed differently turning sentences like in the example below to a somewhat hard problem (note that there are no morphological clues and the semantics can be disambiguated by the verb only)
German'sGerman’s rich morphology (at least in comparison to English) may hinder the phonetic recognition of spoken language (this is however just a wild guess)
German'sGerman’s liberal word order may impede a non-native speaker to rely on patterns in sentence structure (again I'mI’m just guessing).
example:
Die Löwin, die die Trappe fraß. ( the lioness that devoured the bustard )
vs.
Die Löwin, die die Trappe überflog. ( the lioness that the bustard flew over )