This may be a minor distinction but I don’t see caffeine as the problem. It’s the attachment to caffeine that is problematic especially when it becomes an addiction, Caffeine is a mild drug and can safely be consumed in moderation by many productive (and mindful) people.
There is nothing inherently evil in the ingestion of substances which act on the brain or nervous system. It’s our relationship to them that needs to be considered.
The framing of a substance, in this case caffeine, as an inherent antagonist towards health, mindfulness or enlightenment is surrendering of agency to an external influence.
Enjoying a cup of coffee, or smoking a cigarette, doesn’t preclude the possibility of living a mindful, healthy and productive life.