Data Analysis Services Group - July 2013
News and Accomplishments
Jin-Hee Yuk, an ocean modeler from KISTI, arrived from Seoul for a two month visit with the VAPOR team.
VAPOR Project
Project information is available at: http://www.vapor.ucar.edu
TG GIG PY6 Award:
XD Vis Award:
KISTI Award:
Jin-Hee Yuk, an ocean modeler from KISTI, arrived from Seoul for a two month visit with the VAPOR team.
Scott continued efforts to prototype a translation GUI for conversion from foreign data (notably MOM, ROM, and POP) into a VAPOR VDC. After several initial GUI designs were implemented and presented a final "wizard" organization was finalized. Scott is now implementing the GUI back-end with the goal of having a working prototype in early August. Through the process Scott has spent several weeks coming up to speed with the sometimes poorly documented data conversion APIs, and the utterly undocumented VAPOR build system.
2.x Development:
The number of bugs in 2.2.2 has unfortunately reached a critical mass and the decision to put out another patch release, 2.2.4, has been made. All critical bugs have been fixed and regression testing has commenced. Notable problems in 2.2.2 included:
- Alan worked with technical support from IDL, resolving the bug we encountered with running IDL on Windows with VAPOR.
- John fixed a problem with the Mac and Linux installers that was presenting access to the Unidata "udunits" unit conversion data base.
- Data sets containing missing values whose locations varied from one variable to another, such as found in POP, were not handled correctly.
- Derived variables weren't accessible in vaporgui when loaded into a new session.
- Workarounds had to be found for a number of problems related to compression of coordinate variables.
3.0 Development:
Planning for 3.0 continued. John and Alan both produced several revisions to APIs for the ControlExecutive class and VAPOR Data Collection file format. Numerous use cases were also developed to help vet the APIs. John has started prototyping the VDC API.
Administrative:
Education and Outreach:
John provided a letter of support for SongLak Kang's (TTU) NSF CAREER proposal. If funded the VAPOR team has agreed to provide a tutorial at a summer workshop.
Alan helped out with the annual WRF tutorial.
John gave a VisLab tour to Dave Kenny, the CEO of the Weather Channel.
Website:
Meetings on a CISL-wide consolidated software web site continued with VETS, IMAGe and Steve Gienosky.
Software Research Projects
Feature Tracking:
Climate data compression:
SCIparCS Internship
- Mohammad Abouali, a SiParcs student working with Alan and Rick Brownrigg, completed most of his planned effort, producing an NCL library that can produce KML to visualize WRF, MPAS, and other model data in Google Earth. Mohammed presented his work to a group from NOAA, as well as scientists from WRF and WRF-CHEM. Mohammed was unable to stay at NCAR for his final 3 week due to immigration regulations; however his work is almost complete and we still expect the completion of his work to be incorporated into the next NCL release.
- Rick Brownrigg and Alan presented Mohammad’s GE work to a WRF and MPAS-A audience, who expressed a strong interest in using it as soon as it is released in NCL.
Production Visualization Services & Consulting
- In a collaboration with NCSA and Cray, we have been visualizing WRF simulation of Hurricane Sandy on a 500m horizontal grid with 150 vertical layers at half-hour intervals. This resulted in 49 TB of data, with resolution and size unprecedented (among WRF simulations) This data was converted to a VAPOR VDC. Exploiting VAPOR multiresolution techniques, we produced several animations that enabled improved analysis of the dynamics of Sandy. Mel Shapiro is presenting this work at various conferences this summer (e.g. an AMS Mesoscale conference where he is the featured speaker) and expects to use it at a special Sandy workshop at AMS 2014. Mel is using this as a basis for additional funding based on the value of petascale computation in weather prediction. Numerous versions of the Hurricane Sandy visualizations may be seen here: https://www.vapor.ucar.edu/gallery/hurricane-sandy
- Pete Johnsen, Mel Shapiro, Tom Galarneau, Alan Norton, and Mark Straka submitted an article "Petascale WRF Simulation of Hurricane Sandy: Deployment of NCSA's Cray XE6 Blue Waters" that was accepted by Supercomputing 2013.
- Perry Domingo re-edited the Sandy simulation results into a video clip that we submitted to the Supercomputing 2013 Visualization Showcase.
- Mel Shapiro is preparing a presentation at an AMS Mesoscale Workshop where he is the featured speaker. He is working with Alan to prepare visualizations that illustrate various principles of understanding weather dynamics in severe storm systems.
ASD Support
Publications, Papers & Presentations
Systems Projects
Data Services
Accounting & Statistics
Security & Administration
System Monitoring
System Support
ML - Data Analysis & Visualization Clusters
GLADE Storage Cluster
Data Transfer Cluster
Experimental Clusters
Test Clusters
Storage Usage Statistics
dasg:NWSC+GLADE+Usage+Report
Other