Note that he WILL interrupt spells with a casting time, such as Healing Touch, and will continue to do so to every such spell you try to cast. Goggeroc also periodically throws his target up into the air with a forceful uppercut.
Clear out the three kobolds guarding the earthen first, as he will aggro immediately on release. Rogues and druids in cat form should be able to release him and stealth quickly enough to avoid this, however, and thus start the fight on their own terms (so to speak).
Prior to patch 2.3.0, Goggeroc was elite, making him much more difficult to solo.