Wearing a hat comes from ancient codes of honor. When you entered the place where your commander was, you removed your helmet to show trust and respect. The salute was to show that you were not carrying your weapon ready to use. Like a handshake - that's why we use the right hand, the majority of people are right-handed - the hand you hold your weapon in.
The removal of head-gear shows that you trust and respect those in that confined area not to strike at your head - and they can see your eyes.
Today, a baseball cap, frontways, in a shaded restaurant, means your face is not very easy to see. It's like talking on your cellphone while checking out at a store - shows no respect to the person assisting you.
As for cultures that require headcoverings near constantly, it's appropriate to leave them on (Sikh, some Jewish peoples, Muslims, etc), but do you notice that those headcoverings don't cover the eyes? I'm trying to think if there are any that do shield the eyes.... not coming up with anything, but that may be because it's late.