Notes on how to install and execute Swift on various common systems for the AMWG analysis project.
General Information
The general instructions regarding downloading and compiling Swift are at the Swift: Downloads page. In general:
- Remember to download the appropriate cog branch and swift branch. Don't try to build the current swift from the older cog trunk, or in reverse.
- Note that the retrieval of swift from Subversion requires https, but some Subversion clients may be built without https support. In that case, I've been checking out cog and swift on Twister, and then scping/rsyncing the entire tree to the remote system.
Retrieving and Compiling the Source Code
(Keystrokes from the Swift documentation)
$ cd SwiftSupport # (base of the swift installation, usually under Matthew's scratch directory)
$ svn co https://cogkit.svn.sourceforge.net/svnroot/cogkit/trunk/current/src/cog
$ cd cog/modules
$ svn co https://svn.ci.uchicago.edu/svn/vdl2/trunk swift
$ cd swift
$ ant dist
$ cd SwiftSupport # (base of the swift installation, usually under Matthew's scratch directory)
$ ln -s path swift
(e.g., swift -> /users/ncar/mattheww/SwiftSupport/cog/modules/swift/dist/swift-svn)
Source Code edits
Pre-compilation
- Edit the coaster provider to have longer timeouts:
diff cog/modules/provider-coaster/resources/worker.pl cog/modules/provider-coaster/resources/worker.bak 126,127c126 < #my $IDLETIMEOUT = ( $#ARGV <= 2 ) ? (4 * 60) : $ARGV[3]; < my $IDLETIMEOUT = (60*60*24); --- > my $IDLETIMEOUT = ( $#ARGV <= 2 ) ? (4 * 60) : $ARGV[3];
Dynamic/runtime
- Edit the coaster provider to include debugging information when run in-process with Swift:
cog/modules/provider-coaster/resources/log4j.properties
cog/modules/swift/dist/swift-svn/etclog4j.logger.org.globus.cog.abstraction.coaster=INFO log4j.logger.org.globus.cog.abstraction.impl.common.task.TaskImpl=DEBUG
System-Specific Notes
Twister
Package |
Status/Version |
Comment |
---|---|---|
Subversion |
System Installed |
Subversion available by default and works properly |
Java |
Custom Installed |
See environment variables below |
export JAVA_HOME=/ptmp/mattheww/SwiftSupport/jdk1.6.0_21
export PATH=/ptmp/mattheww/SwiftSupport/jdk1.6.0_21/bin/:$PATH
export PATH=/fs/local/bin:$PATH # to get Apache ant
Swift support files are installed in /ptmp/mattheww/SwiftSupport
ReSET Data-Intensive Prototype
Package |
Status/Version |
Comment |
---|---|---|
Subversion |
||
Java |
Custom Installed |
See environment variables below (shared with Frost installation!) |
NetCDF |
Custom netcdf-3.6.3 |
/ptmp/mattheww/SwiftSupport/netcdf-3.6.3-icc |
NCO |
Custom nco-3.9.9 |
/ptmp/mattheww/SwiftSupport/nco-3.9.9-icc |
export JAVA_HOME=/ptmp/mattheww/SwiftSupport/jdk1.6.0_21
export PATH=/ptmp/mattheww/SwiftSupport/jdk1.6.0_21/bin/:$PATH
export PATH=/ptmp/mattheww/SwiftSupport/netcdf-3.6.3-icc/bin:$PATH
export PATH=/ptmp/mattheww/SwiftSupport/nco-3.9.9-icc/bin:$PATH
Dash
Package |
Status/Version |
Comment |
---|---|---|
Subversion |
Not Available |
Subversion does not support https; copying tree from Twister |
Java |
Complicated |
Works on compute nodes, but can't compile on head node |
Swift support files are installed in /users/ncar/mattheww/SwiftSupport
Nautilus
Package |
Status/Version |
Comment |
---|---|---|
Subversion |
System Installed |
Subversion available by default and works properly |
Java |
System Installed |
module load java && module load java-jdk |
NCO |
System Installed |
*module load nco |
Swift support files are installed in /gpfs/medusa/matthew/SwiftSupport