The groans are more because of how overplayed and often overly simplistic puns can be. They're the realm of the quintessential "dad jokes", which were funny when you were a child but then become embarrassing and annoying that he still expects you to laugh when you're not six years old anymore.

But, less violations of grammar and more violations of sets and definitions, but that's a semantic point... but puns are semantic... a metasemantic point... which is just like... we may have achieved a new level of pedantry.