I think you are contradicting yourself when you say Brutalism is about raw basic building blocks or about doing a job and nothing else. Trying “blue with pink, yellow with grey” isn’t doing a job and nothing else. It’s trying something different for the sake of trying something different.
Brutalism is a reaction to the ordered, refined, and clean structure of modern web design. Brutalism isn’t design it’s art. Whenever you’re ‘reacting’ to something you’re creating an artistic expression. Design is the pursuit or intent of solving problems for users. “blue with pink, yellow with grey” doesn’t solve problems of readability or cognitive overload. In fact, they make those issues worse.