Data Analysis Services Group - May 2015
News and Accomplishments
VAPOR Project
Project information is available at: http://www.vapor.ucar.edu
WASP Award:
The 3.0 source tree was restructured to segregate the WASP code from the rest of VAPoR.
John also added support for integer (CDF 5/3) wavelet transforms. The integer transforms have the property that they are perfectly invertible (no loss due to floating point round-off).
KISTI Award:
John researched consumer stereo display technologies in the hope of purchasing a system to facilitate development of stereo support in vaporgui.
Scott continued working on KISTI deliverables:
- In April, I experimented with the Swift parallel scripting language. While I was able to successfully run batch jobs on Geyser, I found it difficult to debug. Sheri Mickelson from the TDD group has worked with Swift, and was happy to share her experiences with it. She too found that debugging was a problem and that Swift’s error reporting was not very explanatory. Arguably, Swift’s most valuable feature is its advanced ability to work on nonlocal data. Since our parallel data conversion scripts will be working on local data, this feature is not useful for us. This drove me to use Python for our parallel script. I also met with Ben Matthews about the HPC Futures Lab, and whether or not we could use it to test KISTI’s parallel scripts. The cluster can currently run Slurm and LSF. JinHee gave me information of an expert on KISTI’s current scheduler, the Sun Grid Engine. I got KISTI’s expert in touch with Irfan, who is now in the process of getting a similar scheduler to the Sun Grid Engine running on the HPCF Lab. Evidently, the Sun Grid Engine is fairly antiquated and getting the same license as KISTI’s would be difficult. However Irfan has been assuring that we can get a functional equivalent.
Last month I created a Big Blue Marble (BBM) image for KISTI that had transparency applied to oceanic regions of the map for visualizing their ocean models. This was done by selecting regions by color and applying an alpha channel to them in a photo editor. This approach was not effective due to the variation in the oceanmap color, and would require excessive time and attention to generate properly. The alternative approach that was implemented this month was to use land and bathymetry data from an organization called GEBCO, and mask out the BBM pixels based on that.
2.X Development:
We released VAPOR 2.4.2. This was primarily a bug-fix release that addressed a couple of important bugs. However, the addition of images with transparent ocean, and a corresponding improvement in our transparency rendering, was also included This was a features requested by KISTI to use VAPOR in their planned visualization contest for this year. Fixed in 2.4.2 were:
- An installer problem that impacted all Ubuntu systems
- An installer problem on the Mac that prevented use with IDL.
Scott added keyboard commands to the VAPOR master branch. The scene can now be translated with the up, down, left, and right keys. It can also be moved forward/backward with +/.
3.X Development:
Alan continues to make progress with VAPOR 3.0. Alan continues to refactor the 3.0 GUI so as to simplify as much as possible the effort of adding new GUI elements. He has updated the documentation and has been organizing a set of instructions for GUI developers.
John continued to fix bugs in the new DataMgr.
Scott debugged a problem that Alan is experiencing with the TextRenderer in VAPOR3.0, that would cause the application to crash. Scott found that we are not currently calling any GL initialization commands. After calling glewInit(), the program would not crash. However the text is still not appearing after being enabled, and Alan believes that there are other initialization commands that may need to be applied.
Administrative:
Annual reviews were completed.
Education and Outreach:
Visitors:
We have been working with Niklas Roeber (visiting from DKRZ) on visualizing Icon and MPAS data in vapor. He has already succeeded visualizing several of these data sets by implementing his own DataMgr. John has been guiding Niklas on the steps necessary to integrate an ICON data reader into vapor 2.x. The data reader would resample the data on-the-fly to a regular grid.
Two SCIparCS interns also arrived: Shreya Mitapalli and Dongliang Chu. Shreya is working on evaluating several compression parameters in the VAPoR Data Collection, while Dongliang is re-factoring the vaporgui raycaster to support curvilinear grids.
Alan has been helping Shreya Mittapalli with her Windows/Python/VAPOR environment. This was a valuable exercise because we learned several shortcomings of using VAPOR with Python on Windows. We will need to fix these in the next (2.5) release this year. We eventually decided to have Shreya do her work on Linux (geyser) and it now works well for her
Software Research Projects
Feature Tracking:
Climate data compression:
Production Visualization Services & Consulting
Scott submitted Peter Sullivan’s LES movie to the XSEDE 2015 conference. Followed up with him to see if he still had data backed up from October and fortunately he does. He has strongly encouraged me to continue working with it. He also told me that the video got a positive review from Chris Davis, the new MMM Director.
- Alan continues to work with Rich Rotunno (from MMM) on visualization of his weather models. Rich invited an Italian collaborator, Mario Miglietta to NCAR for a few weeks, to perform an analysis of a supercell that occurred in northern Italy last year, modeled in WRF. He used VAPOR to understand the wind currents. Mario will be presenting this work at a conference in Vienna this summer.
ASD Support
- xxx
Publications, Papers & Presentations
- xxx
Systems Projects
Data Services
- xxx
Accounting & Statistics
- xxx
Security & Administration
- xxx
System Monitoring
- xxx
System Support
ML - Data Analysis & Visualization Clusters
- xxx
GLADE Storage Cluster
- xxx
Data Transfer Cluster
- xxx
Experimental Clusters
- xxx
Test Clusters
Storage Usage Statistics
Other
- xxx