When I was young and stupid I lost my ear defenders but was determined I wasn't going to miss a massive GPMG live fire - you can guess the rest. I was pretty much deaf for three days. I did some research and apparently there is a gene variant which means you either have "sensitive" or "insensitive" ears. Insensitive (like mine, thank God) you suffer the deafness, your hearing comes back, on a booth/beep test you seem fine but classically you have trouble if there is an intermediate noise; you have real problems listening to people talking in a loud pub/nightclub.
Sensitive ears - one loud exposure and your hearing is f*****d.
The little tiny rubber defenders didn't give proper protection, but you could pretty much hear with them in and they were a lot better than nothing. Yellow foam nearly deafens you.
I'd be tempted to say a defender in your right ear at all times is a good compromise. Stick one in the left in a long contact if there is a lull. Wearing nothing is all well and good but once firing starts you'll be totally deafened anyway.
Some of the commercially available defenders claim to be able to filter out the dangerous stuff, but of course you run into all kinds of problems if it isn't issue.