Click the small X button in the upper left corner of the icon or cursor box to permanantly delete it.
