Death Coil is a "horrify" effect (one of two horrify effects in the game, the other being a shadow priest's Psychic Horror). As such, fear countering abilities will not be effective against it.
Most creatures that are Fear immune are also immune to Death Coil. This includes nearly all undead.
It is a binary spell, and as such it can only hit or miss. Neither critical hits nor partial resists are possible.
Typically, Death Coil is followed up with the Fear spell against melee attackers, especially in PvP as the short horrify effect will prevent you from suffering casting pushback or lose your Fear spell to an interrupting skill such as Kick or Pummel.
It can be used to interrupt enemies' spellcasting.
While in a five-man-instance this spell can be particularly useful when used against mobs aggroing healers or players that recently crowd controled the target, buying the CCers or the tank time to take them out of combat or pick them up respectively. Despite for the fear only lasts three seconds, depending on the given situation you have to consider the odds for a misfortunate fear, which could easily lead to adds and as a result to a wipe.
Because of the high health return of this spell it can be used in conjunction with life tap for mana/health management.
Background
Death Coil was premiered in Warcraft II, where it was used by the Horde unit called the Death knight. It was used in much the same way as it is now, except against either friend or foe. The spell reappeared in Warcraft III as one of the powers of those death knights which serve the Lich King. This version of the spell dealt damage to living enemies, while healing their undead allies.
Patch changes
Hotfix (2009-04-30): "The Warlock ability Death Coil now heals you for 300% of the damage done, up from 100%. [2]"
Patch 2.1.0 (22-May-2007): Is now subject to diminishing returns in the Horror category. Currently, only Death Coil is in this category.
Patch 1.8.0 (10-Oct-2005): Cooldown reduced, mana cost increased, and now causes a short Fear effect that is not diminished.
Patch 1.6.0 (12-Jul-2005): Health leech effects no longer give you health while you are a ghost.