Child pages
  • - Development Notes
Skip to end of metadata
Go to start of metadata


Definitions / Acronyms / Initialisms

  • CS - Campaign Store



User Stories

Simple Copy

  • User wants to copy directories of data every day to the CS
    • from /ldm3_d2/grib/GFS002/%Y%m%d to /RAPDMG/grib/GFS002

Simple Tar

  • User wants to copy a directory of data into a tar file 

  • put in tar file DATEYYYYMMDD.nldn.tar
  • include cdDirTar to put relative paths into the tar file

Globs in Source

  • User wants to copy several directories of data into a tar file
  • <source>/ldm1_d2/nids/raw/nids/*/BREF1/DATEYYYYMMDD</source>

  • Do we want to support this?  Maybe make user use python to build multiple entries into the archive items dictionary?

Multiple entries into the same tar file

  • User gives several archive items with the same destination tar file 
  • Items are added to the tar file before sending to the CS.


Minimum Requirements

  • Support user stories - Simple Copy, Simple Tar, Multiple entries into the same tar file
  • Email user status of gat run

Stretch Goals

  • Support user stories - Globs in Source
  • Allow user to define expected file sizes, # of files, etc. and include results in email.
  • skip underscore files
  • set mode on uploads to CR
  • accept meta-data and make available for data discovery to other users.
  • zip files
  • encrypt files


Proof of Functionality - Connect and transfer via globus

  • Uses globus-cli to connect to the CS and transfer a file.

Add ConfigMaster Support

  • Use ConfigMaster config file to define a set of archive items
  • Iterate over them to send to the CS

Support the "Simple Copy" user story

  • No labels