Strictly speaking your book is right about it expressing a hypothesis. What you should notice is that just as in English, the conditional is often implied or even unclear. In this way the subjunctive can be used to convey certain moods. In your example the implied conditional could read as follows:
Das hätte ich nicht gedacht, hättest du es mir nicht gesagt!/I wouldn’t have thought so, hadn’t you told me!
Now, why does the native speaker not say the following?
Das habe ich nicht gedacht/I didn’t think that
The last sentence is grammatically correct, but it simply and dryly says „I didn’t think that“, without any connotation. Such sentence rarely occurs in practice. Normally we tell someone such things, in order to hint at something else.
Das hätte ich nicht gedacht/I wouldn’t have thought so
is often a veiled way to say something else, such as:
I wouldn’t have thought so! You look damn old (young) for a student!/ Das hätte ich nicht gedacht! Für einen Studenten sehen Sie verdammt alt (jung) aus!
I wouldn’t have thought so! I was sure you wouldn’t be admitted to university!/Das hätte ich nicht gedacht! Ich war mir sicher, daß Sie die Zulassung nicht erhalten würden!
I wouldn’t have thought so! You talk more like a truck driver!/Das hätte ich nicht gedacht! Sie sprechen eher wie ein Lastkraftwagenfahrer!
(I apologise for stereotyping. Sometimes stereotypes are memorable and cross culturally self explanatory)
Similarly, the subjunctive can express moods of restraint and politeness, just like in English. For example:
Ich hätte gern einen Tee/I would like a tea
implies something like:
Ich hätte gern einen Tee, wenn es nicht so mühsam für dich wäre, einen zu kochen./I would like a tea, if it weren’t such a trouble for you to make it.
In this way I express politeness, restraint and consideration.
