Is it right to just stick to basic items as you get older?

From an average person's perspective, patterned items tend to be quite polarizing.
I wonder if wearing solid-colored tops and bottoms is the way to at least play it safe.