For most of the people who don't have German as a mother tongue these three sounds might be the most difficult. How to pronounce something that is not even found on most of the keyboards.

The ä is still not too difficult. In normal spoken language, it cannot be differentiated between ä and e. They have both the same IPA-sign, which are [ε] for a short ä and [ε:] for a long ä. The short ä sounds like in the English words let, men or gender.
The ö is a sound similar to an o, but the lips are put a bit more together, it's actually the sound that is in murder, burden, fur.
The ü might be the most difficult one because there is nothing alike in the English language. Imagine that u meets a long ee, lips formed like you would like to whistle and then instead of whistling put some voice into it.

That's about the theory here have some examples:

ähnlich similar   Ärger   trouble   sägen   to saw

öffentlich public   Öl   oil   Möbel   furniture

(this sound also exist ins French: survivre (to survive), sur (sure), curer (to cure)) über over   merkwürdig   weird   Lüge   lie

Since it's always the question what is the difference between the normal vowel and the umlaut, have an ear at the following examples.

a, A	ä, Ä	o, O	ö, Ö	u, U	ü, Ü