Salt features two timeout parameters called
timeout and
gather_job_timeout that are relevant during the execution of Salt commands and jobs - it does not matter whether they are triggered using the command line interface or API. These two parameters are explained in the following article.
This is a normal workflow when all minions are well reachable:
A salt command or job is executed:
salt '*' test.pingSalt master publishes the job with the targeted minions into the Salt PUB channel.
Minions take that job and start working on it.
Salt master is looking at the Salt RET channel to gather responses from the minions.
If Salt master gets all responses from targeted minions, then everything is completed and Salt master will return a response containing all the minion responses.
If some of the minions are down during this process, the workflow continues as follows:
If
timeout is reached before getting all expected responses from the minions, then Salt master would trigger an additional job (a Salt find_job job) targeting only pending minions to check whether the job is already running on the minion.
Now
gather_job_timeout is evaluated. A new counter is now triggered.
If this new find_job job responses that the original job is actually running on the minion, then Salt master will wait for that minion’s response.
In case of reaching gather_job_timeout without having any response from the minion (neither for the initial
test.ping nor for the find_job job), Salt master will return with only the gathered responses from the responding minions.