Yes, there is a subtle and difficult-to-grasp difference and there even is a third possibility.
First of all, a neutral form can be used for something in this colour:
das Weiße des Auges – the white part of the eye
das Schwarze der Zielscheibe – the black part of a target; the bullseye
The second neutral form (note the different inflection without -e) can be used to refer to a specific shade of a colour or light of that colour:
Ich mag das Weiß der Tapete. – I like the wallpaper’s white (that exact shade of white).
Das Weiß der Tapete blendete. – The wallpaper’s white was glaring (e.g., because the rest of the room was dark).
Ich mag das Blau ihrer Augen. – I like the blue of her eyes.
The female form refers to a tint of a colour, the extent to which something is tinted in a colour or the fact that something is in a colour. In the English translation usually -ness is appended:
Die Weiße seiner Haut war beunruhigend. – The whiteness of his skin was alarming. (Note that usually Blässe is used instead of Weiße.)
die Schwärze der Nacht – the blackness of the night
Röte stieg ihr ins Gesicht. – Her face turned red (she blushed).
Finally, the female form sometimes refers to a dye of that colour:
Druckerschwärze – printer’s ink
Malerweiße – white dye used to paint walls
I hope I did not miss anything.
