Transferring Data and Other Tips

When moving data from Zeus to Bluefire, you need to use the -z bbcp option to reverse the direction of the authentication.  Otherwise the transfer will fail.

Also, due to the nature of bbcp and AIX vs Linux, the optimal bbcp window size and streams may be different depending on the direction you are going.  If you are going from Bluefire to Zeus, it looks like the more streams, the better.  If you are going the other direction, that doesn't appear to be the case, and using only 4 streams may be best. In any case, it may take some experimentation to find the optimal options for each direction.

Note: so from zeus to bluefire, one might use: -s 4 -w 16M   or -s4 -w 2M. So to get data from zeus back to NCAR, from bluefire, one would do something like:

    bbcp -P 1 -s 4 -w 16M -z .

and then transfer it to the local machine.


-P ......... popen command mode. This option is similar to the -O option, but causes all output to be directed to stdout, which is normally redirected to a process that starts HSI with the popen(3) system call. It also results in setting "quiet" (no extraneous messages) mode, disabling verbose response message, and disabling interactive file transfer messages.

Update: CISL has some simple examples of bbcp  using yellowstone's data transfer nodes: