From Walter Sinnott-Armstrong's excellent Stanford Encyclopedia of Philosophy entry on “consequentialism“:
Some critics argue that not all pleasures are valuable, since, for example, there is no value in the pleasures of a sadist while whipping a victim. Other opponents object that not only pleasures are intrinsically valuable, because other things are valuable independently of whether they lead to pleasure or avoid pain. For example, my love for my wife does not seem to become less valuable when I get less pleasure from her because she gets some horrible disease. Similarly, freedom seems valuable even when it creates anxiety, and even when it is freedom to do something (such as leave one's country) that one does not want to do. Again, many people value knowledge of other galaxies regardless of whether this knowledge will create pleasure or avoid pain.
I find all of these objections totally persuasive. Is there any reason for resisting them other than a prior commitment to hedonism?