I don't think I'd see it like that - some friends of mine recently married in a huge church and, knowing the congregation, invited anyone around to come to the actual ceremony. I don't believe that anyone regarded that as rude.

Besides, just saying "it's rude" doesn't really explain why it has come to be regarded as such, which was what I would be interested in.