A quiet day with clear skies, light haze, and warmer still with a high in the low 50’s. IOP 6 concluded today at noon.
ISS1 – All systems still running nominally. Profiler status this morning: @ 1022 100m: 4.38A @ 50V & 1.833A @ 32V, and @1030 150m rass: 3.53A @ 50V & 1.878 A @ 32 V.
ISS2 – No issues again today with all green on the Nagios.
ISFS – Nothing to report on the tower side today.
Soundings – Launched nominal sounding at 4:16 pm today without any issues. There was a sonde that would not connect with the MW41 at today’s 1015 launch. I retried the sonde this afternoon with no luck. The sonde is on the table to the right of the system with the failed sonde from the other day. We depleting another helium cylinder at this afternoon’s sounding. There are 2 full cylinders remaining.
Clear skies today with light haze, and warmer with a high in the mid-40’s. IOP 6, the “Quiescent IOP” started at noon today.
ISS1 – No issues at the ISS1 site with all systems still running nominally. Profiler status this morning: @ 0926 100m: 4.64A @ 50V & 1.837A @ 32V, and @0930 150m rass: 3.87A @ 50V & 1.887 A @ 32 V. While walking around the antenna array during the RASS, the speakers were noticeably louder than they have been in the past. However, the volume level did drop after a minute or so back to the level heard every day since my arrival. I checked the audio connections from the PC to the amplifier and the speaker wire connections and found no issues. I will keep an eye... er, and ear on it and see if the level stays stable.
ISS2 – No issues here today.
ISFS – Spent the morning and early afternoon visiting all 11 remote sites with Jacquie to install Sebastian’s fine-wire thermocouples that connect to the EC100s’ temperature input. We were trying to get the thermocouples installed at all 11 sites before the start of the IOP at noon. We managed to get them all installed, and checked that they were functional by logging into the DSMs, by 1 pm. I checked the wiring for the sonic at the Lake Creek site and may have resolved the intermitted data issue by adjusting the wires coming out of the base of the sonic such that the wires apply pressure to the sonic, possibly fixing an intermittent broken wire/solder joint issue. Jacquie monitoring the data to see if the problem persists.
Soundings – Launched soundings at 2:03 and the 4:15 pm today without any issues with Jacquie’s husband assisting on the earlier launch.
There is one sonde that the students could not get to pass the ground check. I tried twice to get the sonde working, but it is broken as the ground station is not even able to detect it when placed in the cradle. The sonde is in its box, sitting on the sounding computer table, to the right of the system. I would suggest hanging on to it for future demonstration and/or outreach events. During the second sounding, we moved to the 3rd to the last helium cylinder, i.e. we have just under 3 full cylinders left.
Mostly clear skies today, but with heavy haze in the valley today, and a little warmer reaching 50F. Hosted an outreach function for an hour with ~60 junior high school students this morning (picture on ISFS blog entry for today) that went very well. An IOP has been called for tomorrow (Saturday) starting at noon until noon on Sunday, so we will be launching sondes at 2-2:30 and at 4:15.
ISS1 – All systems still running nominally. Profiler status this morning: @ 0928 100m: 4.71A @ 50V & 1.841A @ 32V, and @0930 150m rass: 3.90A @ 50V & 1.888 A @ 32 V. Profiler SNR was notably stronger/higher today, possibly do to the heavy haze (???). I will keep an eye on the amount of snow under the array in preparation for tear down and may stop the profiler briefly during a non‑IOP time to remove what snow/ice I can from under the antenna array to help with the melting.
ISS2 – All systems functioning nominally again today.
ISFS – After the outreach event, I spent the day with Jacquie lowering the 30m towers at the Deer Creek and Provo River supersites in order to replace broken thermocouples. We said hello to some cows while at the Provo River supersite.
Soundings – Launched the 4:15 sounding today with no issues.
Mostly overcast skies with high cirrus clouds and very hazy today. IOP 5 finished up today with no issues.
ISS1 – All systems still running nominally. Profiler status this morning: @ 0953 100m: 4.71A @ 50V & 1.848A @ 32V, and @1000 150m rass: 3.983A @ 50V & 1.889 A @ 32 V. I noticed in xplotmapr’s Diagnostics that the SNR and Signal histograms are showing that Channel 0 (red) of the profiler has slightly better performance than the other two channels. I am attributing this to fact that the Channel 0 antenna panel gets more early sunlight that melts and evaporates the frost off of the antenna faster than the other two antenna panels, which are partially shaded by the speaker surrounds in the morning sun, allowing the Channel 0 antenna to transmit/receive slightly better.
ISS2 – After which, all systems functioning nominally. Some minor problems reloading the Nagios page. Gary has advised that the link found on new Firefox tabs was from IDEAL and out of date and has been removed. Browsing directly to localhost/nagios, per Isabel’s instruction, solved the issue.
ISFS – No issues for me today. Tomorrow we will lower the DCS and PRS towers to replace faulty thermocouple temperature sensors.
Soundings – Launched the 4:15 sounding today with the help of a ~12 U o U graduate students that were visiting the site.
Overcast skies early in the morning, which cleared by late morning, giving way to mostly cloudy skies by the afternoon with some haze. IOP 5 started at noon today, which the PIs are calling a "Moisture surge" IOP. ~12 U of U students will be visiting sites tomorrow afternoon with PI Sebastian. The outreach event scheduled for Friday with ~60 junior high students has been moved to the ISS1 site.
ISS1 – All systems still running nominally. Profiler status this morning: @ 0936 100m: 4.70A @ 50V & 1.842A @ 32V, and @0933 150m rass: 3.94A @ 50V & 1.891 A @ 32 V. It is interesting that the driver amplifier current increases slightly when running in the 150m mode while the HPA current decreases (as expected because of lower duty cycle).
ISS2 – Isabel cleared out the extra logins and addressed the concerns about fs_/ and fs_/boot directories issue on iss2. Bill had me reset the cell modem used with the Purple aerosol detector, though I also power cycled the Cradlepoint after thinking that was the modem. The modem has also been relocated to the back of the base trailer, just to the left of the soil sample oven, to improve the WiFi connection with the Purple aerosol sensor and/or the modem's cell connection. After which, all systems functioning nominally.
ISFS – Went with Jacquie to the Lake Creek tomorrow to swap out the EC100 and take a third soil sample for that site. Changing out the EC100 did not solve the intermittent issue with the sonic not reporting. Both EC100s showed the sonic LED blinking from green to red, so the issue is with the anemometer. During the soil sample collection, the corer was dented in by a rock that hit just outside the edge.
Soundings – Soundings were launched at 2:01 and 4:15 today. At the 2:01 sounding, another balloon had a pinhole leak, so a second balloon was used. We emptied one helium container with the first launch and the tank we connected to for the second launch was also nearly depleted though not marked with orange tape.
Clear skies but more haze in the valley today. Warm again, high in the mid-40s. I met a couple of U of U researcher today that have the trailer in the far corner of the Deer Creek supersite and learned that they are studying aerosols with their trailer. The air intakes are unique in that they use stainless steel asparagus pots over their air inlets.
ISS1 – Jacquie noted that the /data drive on the profiler computer was over 90% full and sent a message to Bill, who responded that this reading does not properly reflect the state of the /data drive, so no worries. Otherwise all systems still running nominally with profiler status: @ 1116 100m: 4.62A @ 50V & 1.846A @ 32V, and @ 1130 150m rass: 3.82A @ 50V & 1.892 A @ 32 V
ISS2 – Bill forwarded concerns about fs_/ and fs_/boot directories becoming full to Isabel. Nagios also flagging a warning about the number of logins being high, 22 out of 30. Bill suspects that there may be a lot of ghost processes/logins that have been abandoned. The number of users dropped to 17 in the afternoon and is no longer flagged as a warning.
ISFS – Systems running with nothing to report. I spent the afternoon working with Jacquie in the base trailer. Planning on travelling to Lake Creek tomorrow to swap out the EC100 to see if it fixes an issue with the sonic cutting out.
Soundings – Sounding was conducted by myself today as Jacquie was conducting the EOL seminar series this afternoon. No problems or issues to report. On the CFACT student slack channel, Gannet Hallar (from U of U?) posted that they are adding a 6:15 am sounding launch to the daily schedule starting this Friday that will be covered by the students.
Warmer today with clear and sunny in the morning, giving way to haze in the afternoon.
I have arranged for an outreach function for ~60 junior high school students scheduled for this Friday at the Deer Creek Supersite. They are only visiting for an hour, so the plan is to discuss what we are doing, show them the instruments on the short tower, the tether sonde, and finish the hour off with an extra balloon sounding at ~1045.
At last Friday’s outreach event, I offered to give my Introduction to CubeSats presentation at the local high school to the science teacher. That offer has been accepted and I will be giving the presentation next Tuesday at 1:00 pm at the Wasatch High School West Campus building for an hour.
ISS1 – all systems still running nominally with profiler status: @ 0920 100m: 5.07A @ 50V & 1.849A @ 32V, and @ 0930 150m rass: 4.12A @ 50V & 1.894 A @ 32 V
ISS2 – Nagios indicates warnings for fs_/ and fs_/boot directories are 86% and 90% full. Not sure if this is related to the lidar issue or not. The lidar is still running at lower resolution, with no signs of frost on lens, and internal conditions reported as 15.1C/14% RH by the software.
ISFS – Spent the day with Matt as he brought Jacquie up to speed. We took soil samples at the South Pivot and the Provo River Supersite, completing the second round of soil samples. While at the Provo River Supersite, we lowered the 30m tower and replaced a malfunctioning thermocouple at the 17m height.
Soundings – Sounding was conducted without any issues, launched by Jacquie.
Warmer today with same clear skies and some very light haze. Jaquie arrived this evening. We will spend tomorrow showing her the ropes before Matt leaves on Tuesday.
ISS1 – all systems still running nominally with profiler status: @ 1020 100m: 4.69A @ 50V & 1.844A @ 32V, and @ 1030 150m rass: 3.89A @ 50V & 1.892 A @ 32 V
ISS2 – No issues on Nagios. Bill will be deciding if/when a factory reset of the lidar will occur after he hears back from Leosphere's tech support. Still no sign of frost on the lens with the internal RH sensor showing between 13-15%.
ISFS – No issues. Helped Matt with a Leica measurement of the boom at the Memorial Hill site today with a nice view as we climbed the hill.
Soundings – There was a mild north wind at the sounding site during today's launch, but the sounding went off without a hitch.
Another clear day with some light haze throughout the day and a little warmer, teens in the morning and 41F in the late afternoon.
ISS1 – all systems still running nominally with profiler status: @ 0924 100m: 4.99A @50V & 1.843A @ 32V, and @ 0931 150m rass: 4.17A @ 50V & 1.892 A @ 32 V
ISS2 – Nagios indicated a problem with the ISS2 lidar not reporting this morning. We found that there was no power reaching the sounding trailer and it looked as though power had been out for several hours. The problem was traced back to a tripped GFI on the ISS2 outlet at the power drop. There was a fair amount of frost everywhere, so it is suspected that moisture may have gotten in somewhere. However the power cables junctions were inspected and were still sealed with tape. Power was restored and the system recovered. The lidar is running at lower resolution and no signs of frost on lens. Emails between Bill and PI’s indicate we may proceed with a factory reset of the lidar. Waiting to hear from Bill if the reset is a go as the lidar’s laptop software is still reporting that the hard drive is ~85% full.
ISFS – After fixing the sounding trailer power, Matt and I went to the Lake Creek and Center Creek sites. Matt used the Leica GPS to measure the direction of the sonic booms. Matt also trained me on how to take soil samples as we took samples at both locations.
Soundings – Since the sounding system had lost power, we had a little trouble getting the base tray to recognize/detect the sonde. We rebooted the computer and sounding system. The problem appears that one has to wait for the sounding system’s green power LED to stop blinking, indicating that the system is ready, which takes several minutes after the system is turned on, before placing the sonde on the ground station tray. A (soft) copy of the user’s manual may have been helpful as we were unsure what a blinking green light meant. We should consider having manuals available for troubleshooting by ISS staff. No issues with the sounding, except it was a few minutes late due to the above delay getting the sonde ready
As indicated in the images below, it was another "hard day at the office" as we trekked out to the Lake Creek site and took soil samples. We ‘borrowed’ the sled that we found on the outside of the helium/tethersonde trailer to carry the soil sampling and Leica hardware.
Clear day with some light haze throughout the day. Spent the first part of the morning with Matt doing an outreach activity at the Upper Provo site. Set up possible additional outreach for next Friday with a group of middle school students. Bill also contacted about an outreach opportunity with some graduate students next Thursday. It was announced on Slack that there will be no IOPs in the next 48 hours.
ISS1 – all systems running nominally again today with profiler status: @1656 100m: 4.35A @50V & 1.833A @ 32V, and @1700 150m rass: 3.65A @ 50V & 1.883 A @ 32 V
ISS2 – The lidar is running but still using lower resolution scans. Laptop software reports that the hard drive is still ~85% full. However, the lens appears to be frost free as of this afternoon.
ISFS – No issues today, all systems running. Matt took some Leica measurements this afternoon.
Soundings – The students reported that the gas regulator or meter was frozen during the overnight soundings when they switched tanks, so it appears that they estimated the volume of gas, as the spreadsheet has ’20ish’ entered for helium volume for a couple of soundings. The issue was not recorded for their last sounding. We found the gas meter had been set atop the oil heater in the trailer in order to warm it up, which worked well. System was functioning normally for the daily sounding. There was no surface wind during the sounding, so Matt and I, out of curiosity, unspooled the string before release. The balloon went nearly straight up and we were able to follow it visually until it reached 500 mPa in altitude.
Overcast early in the morning but clouds cleared by 0930. Temperature dropped to +1F overnight, according to airport weather station, but was up to +9F with mostly clear skies by 0900. An IOP has been called for an ephemeral fog starting today at noon.
ISS1 – all systems running nominally again today with profiler status: @0923 100m: 5.09A @50V & 1.846A @ 32V, and @0930 150m rass: 4.15A @ 50V & 1.892 A @ 32 V. Below is are the temperature curves for the last 7 days taken at 1641 this afternoon.
Temperatures in the amplifier enclosure are varying between +9C and +36C over the last few days. The downward trend corresponds with the colder temperatures seen. The temperatures are seen to start increasing around 1000 in the morning as the sun gets higher while cooling starts in the late afternoon, ~1600. There is an unusually narrow ‘heating’ spike seen around 1000 on 02-02; nothing to worry about, but an odd piece of data worth noting. The profiler currents were also noted for the 100m mode at 1640 this afternoon to be 4.76A @ 50V & 1.835A @ 32V. We continue to see lower currents, especially on the 50 VDC supply to the power amplifier modules, with warmer enclosure temperatures.
ISS2 – The lidar was not running this morning, however the lens had been rotated and was pointing vertical, so at least some sunlight reached the lens and may help with the frosting issue. Lidar was functional by the start of the IOP, though using lower resolution scans. The internal RH was noted to be 13% around 1100 this morning. The lens was not visible so no update on the lens frosting issue.
ISFS – No issues today - all sites operational. It appears that the AC line power was restored to the Midway site this morning (batteries charged up and Icharge current went positive). We were able to get the soil sampling T-bar tool repaired at a local welding shop as well.
Sounding – The first IOP sounding conducted without any problems. During the second sounding, after the balloon was filled, a whistling sound was heard coming from the balloon. A pin hole leak was found at the top of the balloon so the balloon was discarded, a second balloon filled, and the sounding launched on time at 1615.
Sebastian and Alexi were at the sounding site as well, prepping the tether sonde for operations.
Clear and cold (5F) in the morning, partly cloudy in the afternoon, again with a cold N wind.
ISS1 – all systems running nominally again today with profiler status: @0925 100m: 50 V 4.99 A 31 V 1.842 A, @0930 150m rass: 50 V 4.12 A, 31V 1.898 A. Temperatures in the enclosure look good, a bit on the cooler side, corresponding to the slightly higher currents (amplifiers work better when cooler).
ISS2 – Lidar not running in the morning, but partially functional by evening. The second desiccant box was baked out in the soil sample oven at 155 deg F for ~5 hours and then placed on top of the existing desiccant box inside the lidar, as shown in the image below. Note that the upper desiccant box is not secured in any manner, but just sitting atop the first desiccant box. Bill reports that the RH inside the enclosure fell from about 20% to 14% in the evening. There was noticeably less frosting on the lidar lens in the late afternoon, shown in pictures below. The reason for the decrease in the frost is not known, since the second desiccant pack had just been installed.
Also, below is an image of the interior of the lidar, looking upwards, showing how the system is constructed, as requested by Liz. The main optics structure has some seams in it that may or may not be completely hermetically sealed, i.e. water vapor may be able to get in through.
ISFS – Batteries were then replaced at the DC site, where the AC line power to the outlet has been connected. When we initially plugged the system in and turned on the power strip, the GFI (ground fault interface) at the outlet tripped. The AC voltage measured at the power box was also low when back on, measuring ~109 VAC. The system is working and will be monitored for any power issues.
Sounding – sounding conducted without any problems at 1615 local time.
An IOP has been called for an ephemeral fog starting tomorrow at noon, so we will be launching two balloons tomorrow. Additionally, the overnight low is forecasted to be -6 deg F, so it will be a cold morning tomorrow.
As noted in the daily summary, the windcube lidar has had a database issue the last couple of days. The lidar had been saving spectral data which might be useful if we ever need to reprocess the data (some papers show how reprocessing the spectra can result in extending the range of the lidar by extracting moments out of noisy spectra in a similar manner to how NIMA extends the range of the radar wind profilers). This results in much more data than normal operation, which shouldn't be a problem as long as the extra data can be cleared off the database. Unfortunately there appears to be a problem with the clearing off process not freeing up disk space after the data has been deleted. We turned off the spectral collection last week and have been deleting data but the database system hasn't freed up disk space, and the lidar stopped collecting data.
Vaisala tech support connected to the system along with Leosphere engineers in France and were able to partially unclog the system this morning. They have a process running that seems to be slowly clearing the database out, and that allowed limited measurements to resume. The clearing out process is running very slowly so I'm setting up a limited observation schedule with low resolution measurements to ensure that the lidar can run for tomorrow's IOP. We may be able to do a factory reset to restore the system, however that process can take 2 or 3 days apparently so we are delaying that until after the IOP.
Weather was about the same as it has been for the past few days. It did warm up to the 40s but some wind and clouds moved in toward the end of the day.
Today I showed John the ways to check on ISS equipment remotely and Matt started showing him some of the ways to check on ISFS as well.
We were told that the work was finished at Daniel's Canyon, so we went to go plug in the power cord. I read out the outlet first to be sure, and it turned out there was no power and no work done.
Tsoils at DCS has been very intermittent still and went out again so we replaced a PIC board.
During the sounding today we got the RH level over limit. We restarted the whole system and the issue cleared. Was a good way to show John what to do. We had to start a new helium tank and when we opened it up to finish filling the balloon I could hear and feel a leak around the fitting. I tried taking it off and putting back on and it was still there. We had also thought the last tank didn't last very long (5.5 balloons). We quickly filled the rest of the balloon so we could get it up and to look at this issue more. We tried the fitting on another tank and it leaked as well but worse, so we determined the fitting was now bad. After some proper communication with Lou we were able to find the spare fitting and John will try this out tomorrow.
Clear blue day in the morning, giving way to mostly cloudy with a cold NW wind by later afternoon.
ISS1 - system running nominally with profiler status: @0925 100m: 50V 4.87A, 31V 1.835A, @0930 150m rass: 50V 4.08A, 31V 1.889A
ISS2 - Lidar still not running. There are two issues with the system. First, there is frost on the inside of the lidar lends, which we may not be able to anything about, though Sebastian has suggested adding more desiccant near the scanner head. Will wait to hear from Bill/Vaisala on next step for this issue. There is also a software issue that is preventing the on-board computer/system from running, though Bill has been in contact with Vaisala and thinks this issue will be resolved possibly as soon as tomorrow.
Late morning was spent at the Midway site trouble shooting a power issue. The breaker for the AC line power has tripped and is not accessible. Dan has been in contact with the property owner and the man tending to the horses, but no word back yet on when AC line power will be restored.
Sounding - late yesterday we were able to locate a different high pressure connector for the helium tank on one of the spare regulators after talking with Lou. The nut of the spare connector has damaged threads, but we were able to use the nipple of the spare connector with the original nut and had no issues or noticeable leaks once attached to the helium tank. The 'bad' nipple and nut are zipped tied together and in the spare regulator tub.
Since we had time before the sounding launch, I went to reboot the sounding computer and instead shut it down. It took a call to Lou and some keen eyes to find the tiny power button on the front of the new sounding computer and get the system back up and running. Sounding was conducted with no other issues, however under a strong and cold NW wind that had developed.
Liz left mid-day today with Matt giving her a ride to the SLC airport.