That screams to me of a codebase that isn’t using modular practices. If your CSS isn’t modular, then it absolutely should all be in one file.
…but your CSS should be (mostly) modular. Use SMACSS and/or BEM methodologies. Need to edit the “media” module? Open media.scss. Need to edit the “nav” module? Open nav.scss. It should be no more complicated then that.