I find it interesting that Stephen King spends quite a bit of time defending the importance of correct grammar in his excellent and inspiring On Writing, yet his editor allowed him to misuse "which" consistently throughout.
"Which" and "that" are both used to introduce subordinate clauses: groups of words that tell us more about the word or phrase they modify. However, "that" generally tells which of several examples of a thing is being referred to. (Often, when used before verbs, it gets dropped out of sentences these days.) "Which" is used to give an extra tidbit of information.
The Canadian Press Stylebook explains this in more technical terms when it says: "That often introduces an essential clause – one that... cannot be omitted. (...) Which introduces a non-essential or parenthetical clause – one that adds information that could be omitted without changing meaning."
Check out these examples:
"The puppy that died was the smallest of the litter." (There were several puppies. Only one died.)
"The puppy, which died, was the smallest of the litter." (By the way, the small puppy died.)
"The bowl that is full of soup is on the counter." (There are lots of bowls, but only one has soup.)
"The bowl, which is full of soup, is on the counter." (There's only one bowl. Watch out! It's full of soup.)
"The book that I am reading is interesting." (This means "of all my books, the one I'm reading right now..." These days, it's apt to be changed to "The book I'm reading...")
"The book, which I am reading, is interesting." (A bit sarcastic, possibly to be read in a Houselike tone: Can't you see I'm reading, you moron?)
Notice that if you drop out the red bits, you still get the essential meaning of the sentence, but if you drop out the purple bits, you have trouble knowing which dog, which bowl, which book.
If you still can't tell, speak the sentence out loud. If you hear yourself making a comma-sized pause before the "which-or-that" part of the sentence, the word that you want is "which".