Parallel Run Commands On Multiple Servers

If maintaining servers is one of your routine jobs, occasionally you will need to run commands on multiple servers. Say restart service, update firewall rules, grep files, etc.

It’s a bit boring and inefficient to manually ssh and run commands one by one.

Permanent Link: http://dennyzhang.com/parallel_run_commands

Manual steps are slow. If you have multiple servers, you will complain. If your network to servers is doggy, you will complain more. To automate this, people usually write a tiny bash script running a loop on the servers.

for server in $server_list; do
    ssh -i $ssh_key_file \
        -o StrictHostKeyChecking=no \
        -p $ssh_port root@$server
done

It works nicely. From my working experience, I do see two major improvement points:

1. It can be faster, if we can run remote commands parallelly not sequentially. The more servers involved, the bigger difference it would be. In certain scenarios, it even means less downtime or faster response.
2. Track command history and console output. This will be helpful for trouble shooting, since it enables us to better audit servers’ change history,

Using ansible, we can easily fix point #1 like below.

# install package
sudo pip install ansible

# define a list of hosts to manage
cat > my_hosts <<EOF
104.131.134.190
107.170.211.46
107.170.242.31
EOF

# parallel run remote ssh commands
ansible all -i my_hosts \
--private-key=$ssh_key_file \
-u root -m script -a "test.sh"

For the audit purpose in point #2, we can wrap up above ansible command as a Jenkins job: RunCommandOnServers.