Two things that should be much more obvious in Gnome, if not actually defaults:
xmodmap -e "remove lock = Caps_Lock"
xmodmap -e "keycode 106 = "
In other words, disable caps lock, and disable the insert key. Why these aren't even offered as choices in the keyboard configuration menu is simply befuddling, not to mention the time wasted by many, many people if Google search results are any indication.