From WoWWiki

Quest givers are NPCs that provide quests for players. When the quests are completed, the quest giver will usually bestow XP, or, if the character is the maximum level obtainable, money. Sometimes, quest givers will include an item or two as a token of their thanks. Questgiving NPCs will not attack monsters or the opposing faction unless they have been attacked first. A quest giver who currently has a quest for you will have a golden exclamation point (!) above their head.

As of Patch 2.3, your pointer will also change when you move it over a quest giver:

interact with quest giver quest giver out of range

Some items and fixed devices can be quest givers also. Usually, you need to right-click the item or fixed device to get the quest. Items like this will usually say "This Item Begins a Quest" in their tooltip. The only way to tell if a Fixed device begins a quest is to interact with it.

Most quest givers are quest enders (who you turn the quest items to or receive the quest reward from) also.