Since Safari 6 (on Mac), the delete key no longer doubles as the browser Go Back action. To re-enable this feature, run the following snippet in Terminal and restart Safari.

defaults write com.apple.Safari com.apple.Safari.ContentPageGroupIdentifier.WebKit2BackspaceKeyNavigationEnabled -bool YES