Notes on how to install and execute Swift on various common systems for the AMWG analysis project.

General Information

The general instructions regarding downloading and compiling Swift are at the Swift: Downloads page. In general:

  • Remember to download the appropriate cog branch and swift branch. Don't try to build the current swift from the older cog trunk, or in reverse.
  • Note that the retrieval of swift from Subversion requires https, but some Subversion clients may be built without https support. In that case, I've been checking out cog and swift on Twister, and then scping/rsyncing the entire tree to the remote system.

Retrieving and Compiling the Source Code

(Keystrokes from the Swift documentation)

$ cd SwiftSupport # (base of the swift installation, usually under Matthew's scratch directory)

$ svn co https://cogkit.svn.sourceforge.net/svnroot/cogkit/trunk/current/src/cog
$ cd cog/modules
$ svn co https://svn.ci.uchicago.edu/svn/vdl2/trunk swift
$ cd swift
$ ant dist

$ cd SwiftSupport # (base of the swift installation, usually under Matthew's scratch directory)
$ ln -s path swift
(e.g., swift -> /users/ncar/mattheww/SwiftSupport/cog/modules/swift/dist/swift-svn)

Source Code edits

Pre-compilation

  • Edit the coaster provider to have longer timeouts:
    diff cog/modules/provider-coaster/resources/worker.pl cog/modules/provider-coaster/resources/worker.bak
    126,127c126
    < #my $IDLETIMEOUT = ( $#ARGV <= 2 ) ? (4 * 60) : $ARGV[3];
    < my $IDLETIMEOUT = (60*60*24);
    ---
    > my $IDLETIMEOUT = ( $#ARGV <= 2 ) ? (4 * 60) : $ARGV[3];
    

Dynamic/runtime

  • Edit the coaster provider to include debugging information when run in-process with Swift:
    cog/modules/provider-coaster/resources/log4j.properties
    cog/modules/swift/dist/swift-svn/etc
    log4j.logger.org.globus.cog.abstraction.coaster=INFO
    log4j.logger.org.globus.cog.abstraction.impl.common.task.TaskImpl=DEBUG
    

System-Specific Notes

Twister

Package

Status/Version

Comment

Subversion

System Installed

Subversion available by default and works properly

Java

Custom Installed

See environment variables below

export JAVA_HOME=/ptmp/mattheww/SwiftSupport/jdk1.6.0_21
export PATH=/ptmp/mattheww/SwiftSupport/jdk1.6.0_21/bin/:$PATH
export PATH=/fs/local/bin:$PATH # to get Apache ant

Swift support files are installed in /ptmp/mattheww/SwiftSupport

ReSET Data-Intensive Prototype

Package

Status/Version

Comment

Subversion

Java

Custom Installed

See environment variables below (shared with Frost installation!)

NetCDF

Custom netcdf-3.6.3

/ptmp/mattheww/SwiftSupport/netcdf-3.6.3-icc

NCO

Custom nco-3.9.9

/ptmp/mattheww/SwiftSupport/nco-3.9.9-icc

export JAVA_HOME=/ptmp/mattheww/SwiftSupport/jdk1.6.0_21
export PATH=/ptmp/mattheww/SwiftSupport/jdk1.6.0_21/bin/:$PATH
export PATH=/ptmp/mattheww/SwiftSupport/netcdf-3.6.3-icc/bin:$PATH
export PATH=/ptmp/mattheww/SwiftSupport/nco-3.9.9-icc/bin:$PATH

Dash

Package

Status/Version

Comment

Subversion

Not Available

Subversion does not support https; copying tree from Twister

Java

Complicated

Works on compute nodes, but can't compile on head node

Swift support files are installed in /users/ncar/mattheww/SwiftSupport

Nautilus

Package

Status/Version

Comment

Subversion

System Installed

Subversion available by default and works properly

Java

System Installed

module load java && module load java-jdk

NCO

System Installed

*module load nco

Swift support files are installed in /gpfs/medusa/matthew/SwiftSupport

  • No labels