Data Analysis Services Group - June 2011

News and Accomplishments

xxx

VAPOR Project

Project information is available at: http://www.vapor.ucar.edu

TG GIG PY6 Award:

Yannick continued to try and resolve a bug in the PIOVDC code that prevents its use with slice-based decompositions of 3D domains. Progress has been slow due to the difficulty of debugging MPI code and challenges in getting schedules to mesh with PIO developers.

John parallelized the forward wavelet transform filter bank used by VDC type 2 data. The threaded implementation achieved a 7x speedup on an 8-core system (storm0). More importantly, the forward transform performance is now comparable to the inverse transform performance, and is also in line with VDC type 1 data conversion time.

XD Vis Award:

Kendall started work on updating the VisIt plug-in for VAPOR Data Collection reader. Since the original plug-in was developed, new versions of both VAPOR and VisIt have been released, the former having a number of changes made to the data reader API.

Yannick completed development work to support loadable OpenGL shaders in VAPOR. Users of vaporgui will now be able to  modify, or create new, shader programs for texture based volume rendering and ray casting. Helpful shader error diagnostics were also added.

John revised the specification for describing attributes (e.g. geometric transformations) of 3D model files to provide greater flexibility for users, and be more consistent with other VAPOR metadata. John worked with Kenny Gruchalla to develop an XML parser for the new model description language. Kenny Gruchalla implemented the remainder of the specification and has checked in his code for the 2.1 VAPOR release.

John worked with Ben Brown and collaborators at Utah to produce an animation of Ben's stellar MHD simulation data for the DOE's SciDAC Visualization Night, to be held in Denver in July. The imagery for the animation was rendered with VAPOR using Utah's GLURay ray tracer. The animation is available from http://vis.ucar.edu/~clyne/SciDAC2011/2011_StellarMag2_small.mov

Yannick continued prototype work on depth peeling.

Development:

Alan fixed a bug in the Python/WRF scripts that was causing problems in the calculation of potential vorticity.

Craig Ruff figured out the problem we were having with fonts in
Linux.  The problem was due to conflicts between Qt and the vnc X
server.  Craig found that we can provide an environment variable
with Qt4.7.3 to workaround the problem.

John started to fix bugs in preparation for the 2.1 release: the path to Python startup script is now correctly generated on Mac platforms when vaporgui is invoked from the command line.

A release plan for 2.1 has been finalized: the targeted release date has been set for end of September.

Kendall started working on updating the wrf2vdf and wrfvdfcreate applications to handle conversion to VDC2 data sets.

Outreach and Consulting:

Sherrie Fredrick set up wrf vapor data for Arlene Laing’s Africa run.

Alan met several times with Mel Shapiro on the ERICA storm simulation data and made several animations such as shown at http://vis.ucar.edu/~alan/shapiro/ericabwctt.mov . We are now compiling and converting a final simulation run from Ryan Maue.

Alan has been working with Hsiao-ming Hsu on unsteady flow integration in preparation for an analysis of a polution event in Germany.  An animation of backwards particle trajectories of this event is athttp://vis.ucar.edu/~alan/hsu/11diag.mov

Alan completed and presented a VAPOR poster at the WRF workshop at CG.  There was a lot of interest, and several people expressed a desire to use VAPOR in their work.

We (Alan, Karamjeet, Kendall and Yannick) gave a VAPOR tutorial on June 24th at the Wrf Workshop.  We had a full room (2 people per computer).  The attendees showed lots of interest and the class went very smooth.

KISTI Proposal:

DASG received news that KISTI had received their federal funding and was ready to resume contract negotiations with NCAR. In anticipation of an eventual award we continued preliminary work on KISTI project deliverables.

Karamjeet, DASG SCIparCS intern, has put together a set of Python scripts to convert
POP data into vapor.  These use CDO to create a weights file that
is then used to convert the data to lat/lon.  PyNIO and PyNGL are also used in creating needed variables before ncdf2vdf performs the conversion.  Karamjeet presented this work to Frank Bryan and we got some valuable feedback to use in putting together our plans for ocean data visualization.

John sub-classed the RegularGrid abstract data type developed in May to provide a Layered grid data representation (a rectilinear grid where the coordinates of one dimension are a function of i-j-k grid indeces, such as used by WRF). After testing the new LayeredGrid class John then began integrating the new data representation into vaporgui. Among other things the new gridded data representation will enable vaporgui to support missing values, such as those produced by land masses in global ocean circulation model data.

Community service and professional development:

Alan attended the GPU course presented by Rory Kelly.

Documentation:

Kendall has been working on adding documentation, updated search, and navigation functionality to the web site.

After finalizing the layout of the drupal based VAPOR documentation John consolidated VAPOR's multiple data preparation documents and re-authored them in Drupal. The conversion process was quite tedious: the browser-based wysiwyg editing tools for Drupal are extremely simplistic (lacking essential features such as spell checking) and buggy.

Admin:

John authored the quarterly ARRA report for the XD Vis award.

John authored the VAPOR section of the FY12 POP

Software Research Projects

Data Analysis & Visualization Lab Projects

File System Space Management Project

  • Restarted work on FMU development and documentation with accounting data gathering and reporting functionally as the first goal.

Accounting & Statistics Project

  • xxx

Security & Administration Projects

  • Tracked the People DB 2.0 changes.  Will wait for People DB 2 and LDAP to stabilize before attempting to use it as an authoritative source for authorization information.
  • Updated the KROLE management utilities to support the group style role principal and to obtain account and group information from the new LDAP schema.

System Monitoring Project

  • Added GPU monitoring to ganglia, tracking GPU load, memory usage, fan speed, and temperature of our storm systems.

CISL Projects

GLADE Project

  • xxx

Lustre Project

  • xxx

Data Transfer Services Project

  • xxx

GridFTP/HPSS Interface

  • xxx

TeraGrid Project

  • xxx

Lynx Project

  • xxx

Batch Systems & Scheduler Project

  • xxx

NWSC Planning

DASG staff continue evaluation of vendor bids submitted in response to the CISL RFP:

  • John and Pam presented to the TET a revised analysis of the DAV and CFDS components of the vendor bids after receiving responses to clarification questions.
  • Pam and John checked references on one of the vendor bids

Production Visualization Services & Consulting

  • xxx

Publications, Papers & Presentations

  • xxx

System Support

Data Analysis & Visualization Clusters

  • Figured out a work around to get VAPOR to display nice fonts while being run inside a VNC session.  The Qt 4 toolkit font rendering really wants
    to use the Xrender extension, which the older VNC X server code does not support.
  • Compiled and installed versions of Mpich2 for each pair of compilers.
  • Helped Paul Goodman configure a new GLADE client on a virtualized VETS system.
  • Installed the GDAL, ipython, libSVM,  packages on all systems.
  • Updated the "R" software and installed the fields, maps, geoR, and ncdf4 packages.
  • Updated NCL to version 6.0 on all systems.
  • Updated the get_user_info.pl script on Blizzard to attempt to use the new LDAP format.

GLADE Storage Cluster

  • Upgraded the firmware for the internal RAID controller on iceberg (/glade/home backup data mover) to prevent sporadic system lockups. This appears to have fixed the issue.
  • Upgraded the software on glacier (the DataDomain DD670 which is the /glade/home and DAV server backup repository).  Started testing to see if the new IPMI support will be usable to enable remote power control and console access.
  • Created the cola1045, colu1032, csu1329, ucar1057, udel1011, uhaw1053, ucol1413, and EOL project spaces.
  • Replaced several failed drives and a failed fan module in the DDN 9900 storage system.
  • Finished work on the "gladequotas" utility, after adding the ability for CSG to run it with sudo to display home/user quotas for a specified user.
  • Moved waccm project space into the ACD allocation.
  • Removed the glade011, dasg020, and dasg022 project spaces.
  • Updated the GLADE documentation with some clarifications and info about the gladequotas command.

TeraGrid Cluster

  • xxx

Data Transfer Cluster

  • xxx

Other

  • xxx
  • No labels