Stand still and swing at it with a sword right before it hits you. Ghosts are knocked back several tiles after being hit, so walking towards them may speed up the process of killing them. Otherwise, wait until they slowly float back towards you for another strike.