His point is that if a man does not possess goodness, then he has no business participating in ceremonial rituals because he would only violate their true meaning with inappropriate displays of ostentatious wealth and power or false reverence and piety.
Although he doesn’t mention them by name, Confucius is probably indirectly criticizing the ritual excesses of the Three Families with this comment. He is also suggesting that only people who were born with an innate sense of goodness have the capacity to further cultivate their characters through the practice of the rites, while those born without it (presumably those pesky Three Families again) have no hope at all of improving themselves even if they do their best to follow them.
Nature, or native substance as Confucius often calls it, thus wins out over nurture. Even if a man is highly talented, without goodness as his anchor a man will apply his abilities for his own ends rather than the overall benefit of society.