If you click on the little flag (or whatever icon you used for the marker), a box with that icon will appear next to it. Click inside that box and it will bring up the same editing menu as when you first placed the marker and there is an option to remove it there.