As for task runner npm is much simpler solution compared to gulp/grunt/broccoli. And what is more interesting that neither of gulp/grunt/broccoli is actually building your stuff. They delegate this work to other tools: babel, uglify, browserify, webpack, system.js, scss etc. Those are build tools.
After I switched to webpack I haven’t had a need to setup much of build tasks. You still need to configure loaders and it may be cumbersome sometimes. But you configure specialized build tool, not your task runner, that take care of dependency tracking, watching etc.
Personally I feel that gulp/grunt/broccoli are too general tools, that aren’t doing well neither as task runner nor as build tool. They are in the middle and that is the problem.