You can engage in defamation without speaking. This is not a good comparison.
If you are speaking and during that time defame somebody then yes...I'd say it's entirely sensical to assert that your speech is being regulated. Defamation is the act, just as stabbing or shooting is the act. You still need to do it with something, with a weapon.
Going in circles man.