rsync question

[kim_bcs]

Hi,

We have the following setup.

A bunch of units out in the field all connecting to a central hub via a wifi link. We have a script on the hub which pulls files from each of the units via rsync.

So the script looks something like:

rsync -av unit1:/location/of/file/ /place/file/here
rsync av unit2:/location/of/file/ /place/file/here
etc

Now the problem is that when/if one unit is down, rsync will timeout and not carry on pulling the files from the units below that one in the list. I have tried playing with rsync and included --timeout=60, but this does not work when ssh is not available on a unit.

Does anyone have any ideas on how we can get rsync to carry on with the next unit if the unit above it in the list is not available?

Thanks

 

[SarelSeemonster]

I'm assuming Bash. Do you perhaps have

set -e

anywhere in the script, or does the shebang look like

#!/bin/bash -e

?

 

[DrJohnZoidberg]

Maybe something like this:

#!/bin/bash

# Unit 1

if ping -c 1 unit 1 | grep Unreachable > /dev/null 

then
{
echo "Can't reach host"
}
else

{
rsync -av unit1:/location/of/file/ /place/file/here
}

fi

# Unit 2

if ping -c 1 unit 2 | grep Unreachable > /dev/null 

then
{
echo "Can't reach host"
}
else

{
rsync -av unit2:/location/of/file/ /place/file/here
}

fi

This code is probably not in the correct syntax, but its just the idea.

 

[SarelSeemonster]

From the Bash manpage:

-e Exit immediately if a pipeline (which may consist of a single simple command), a subshell command enclosed in parentheses, or one of the commands executed as part of a command list enclosed by braces exits with a non-zero status.

Which will sound like exactly the opposite of what you're trying to achieve, therefore if you do have set -e somewhere in the script, removing it should prevent Bash from exiting when one of the commands that it executes fails.

Alternatively, with set -e still present (or added if it wasn't there before), you can try to add || true to the end of each rsync command so it will make the command pipeline have a true return value even if the command fails so the set -e option won't kill the script.

Hope that makes sense.

 

[sidai]

Hey,

If you still didn't get it right ( I had similar problem and the set -e and other stuff didn't work)
Try this:

....
....commands...
rsync -av unit1:/location/of/file/ /place/file/here;
rsync av unit2:/location/of/file/ /place/file/here;
...more commands...
(note the ";" after the command)

Now, if one of the commands fail, it will continue to the next one.
Or try this:

rsync -av unit1:/location/of/file/ /place/file/here &>> /log/file/if/you/want;
rsync av unit2:/location/of/file/ /place/file/here 2> /error/log/file/location;

 

[DrJohnZoidberg]

Hey,

If you still didn't get it right ( I had similar problem and the set -e and other stuff didn't work)
Try this:

....
....commands...
rsync -av unit1:/location/of/file/ /place/file/here;
rsync av unit2:/location/of/file/ /place/file/here;
...more commands...
(note the ";" after the command)

Now, if one of the commands fail, it will continue to the next one.
Or try this:

rsync -av unit1:/location/of/file/ /place/file/here &>> /log/file/if/you/want;
rsync av unit2:/location/of/file/ /place/file/here 2> /error/log/file/location;

Thanks for this. I didn't know that.