Important note: You must be in Caer Darrow at the time you equip the trinket. If you are not in Caer Darrow when you equip the trinket, then later come to Caer Darrow, none of the ghosts will be visible. If this happens, simply take off then re-equip the trinket and the ghosts will appear. Consult the minimap location to check if you are in the right place. Also, if you equip the trinket while not in Caer Darrow, the game will display the warning message "You need to be in Caer Darrow".
Also, sometimes the equip of the item may not register. The solution is to equip the item in your second trinket slot to activate it.