’People gossip about me.
I am sorry for what I have done;
I have taken three thin steps
And landed myself in the tavern of my mistress’ - Tsangyang Gyatso.

People gossip about you, me and everyone else. It's inevitable.

How affected you are by it is however anything but inevitable. Yeah under the sway of the 8 worldly dharmas far too much weight is given to what people think and say about us, or worse, what we think they think and say about us. If these people are in the group we've at that time labeled friends then the whole thing takes on another level of perceived importance.

Amongst all the hypocrisy, delusion, pretense and samaya damage, what someone you'll never meet again thinks comes pretty low on the scale of importance. Perspective is a great thing if kept loosely.