Don’t set scheduler host to localhost in Docker
This post will make no sense to almost everyone. But I just wasted a full day trying to connect to a dask cluster hosted in a Docker container, failing again and again. I’m sure this makes sense to people who know their stuff, but DO NOT SET the host parameter of your Dask scheduler to “localhost”. You’ll not be able to connect to it from another container no matter what you do.
Instead, just leave it blank and it will host at the machine IP (something like 172.24.0.3 rather than 127.0.0.1) and you can then connect to it from another container or the host machine using localhost:8786
[assuming you opened that port with ports: — '8786:8786'
(docker compose)]. If anyone can explain that to me, I’d be much obliged.