Mostly clear but hazy today.
Laura and I visited iss2 and iss3 today. At iss2 I gave Laura a tour of the site and a refresher on the MW41 system. I also backed up the sounding system config for this project. At iss3 I backed up the sounding system config and got the temperature probe working again (I had to power cycle it and clear the log, which was taking too long to download).
I leave tomorrow, so this is the last blog post from me!
Very hazy today, especially on the coast. Windy and cool at Rancho Alegre.
IOP 6 went very well, all launches were successful. We launched on a modified schedule, at 10, 1, 4, and 5:30. Chris is now an expert balloon launcher. Chris also scavenged enough horseshoes for us to play a couple rounds in between soundings!
Laura arrived today, and tomorrow we'll visit iss2 and do a refresher on soundings. I'm also planning a visit to iss3 to look at the temp probe. No IOP planned for tomorrow or Saturday.
Very hazy today, getting gradually more overcast as the day went on.
IOP is on for tomorrow with modified schedule–our last launch is at 5:30, not 7. Chris and I will take care of the launches tomorrow.
All quiet for ISS today, so I went to Gaviota Peak with Chris for a maintenance visit.
Hazy and warm inland, more and more overcast as the day went on. Windy further west along the coast.
John arrived today, Lou leaves tomorrow. No IOP tomorrow but likely will have one on Thursday.
I went to iss3 to look into why we weren't getting ceilometer and all sky cam data. When I arrived at the site the ceilometer computer was on and responsive but somewhat sluggish, with something like 5 copies of each of the ceilometer programs running. Rsync server was not responsive. I tried to open task manager to kill rsync server but got this error message:
Tried restarting and shutting down the computer from the start menu, but both times after you confirmed shutdown/restart it just stayed running. Eventually power cycled via the power button. After that I started the ceilometer software and tested that rsyncs to the data manager work.
Unfortunately it looks like the ceilometer software was not saving data while the rsyncs weren't working, so we lost ~17hrs of data.
The iss3 container was full of earwigs which were getting in through a gap in the edge of the screen of the window behind the sounding computer. I taped over the screen edges which will hopefully keep them out.
Warm, calm, and some high cirrus at Rancho Alegre today. EOP 2 went well, with all balloon launches successful. Saw some deer and turkeys on an evening walk. There's a tree down across the road to the site (as of this afternoon), but it's not hard to drive around it.
No IOP or EOP expected for tomorrow or Wednesday.
After getting back to the hotel this evening noticed in nagios that the data from the ceilometer and all sky camera at iss3 stopped a couple hours ago. I can still ping the ceilometer pc, but rsyncs from it fail. I'm guessing rsync server stopped (either on its own or because the computer rebooted). It's possible the ceilometer computer is still collecting data and just not transferring it, but if the computer rebooted or hung I'm not sure that's the case. I'm planning to visit iss3 tomorrow to get it working again.
Another clear and warm day. All good at all three sites this morning when I checked nagios.
Lou went to iss3 to drop off some cleaning supplies for teardown. I spent the day with Chris going to Little Pine for a maintenance visit.
Clear today, calm in the morning and getting windier through the afternoon. Morning data check showed all good at all 3 sites, and we didn't hear about any problems from the students at iss3. Our launches at iss2 all went smoothly.
Possible EOP on Monday, but that won't be decided until tomorrow.
Another quiet day for ISS. No issues at morning data check so worked from hotel. IOP is on for tomorrow so we will be launching at iss2.
In the afternoon helped Mack go recover a dropsonde off the ridge near Romero Saddle!
A quiet day for ISS. I arrived yesterday evening, Liz left this morning. Thanks Liz!
Morning data check showed no issues, so Lou and I spent the day working on other stuff from the hotel. The potential IOP for Friday has now been pushed back to Saturday.
This morning Nagios showed that the ISS3 sounding computer was down so we suspected it's the same issues we have had with the UPS. When Lou and I got there the UPS was off so I turned it back on and everything booted up fine. We had stopped at the ACE and grabbed a digital clock to plug into the UPS to see what the time difference is the next time. There had been no indication of a power outage as the other electric clock in the trailer read the correct time.
Next we went to ISS2 since the plots had not updated for the CL51. We went to make sure the files were still saving since Nagios did not indicate any issues with data. Once we verified the files were still saving, I called Isabel to help fix the issue. She found that her work the other day created a bug in the script and she was able to get everything fixed.
I head back to Boulder tomorrow and Isabel safety made it to Santa Barbara to take my spot.
Today was warm with high clouds and some wind later on.
Last night the 4am and 7am launches did not happen due to miscommunications and LAX not approving the launches.
This morning while checking Nagios I noticed the all sky camera and CL31 at ISS3 had not reported data for a while. Lou and I spent the morning there troubleshooting. It seemed as if the computer was frozen or working extremely slowly, so we restarted it. I also did a hard restart on the CL31. We realized we both didn't quite remember how to restart the CL31 software so we contacted Gary and Isabel. Between the both of them and us physically in the trailer we were able to resolve the issues and data had started coming back in.
ISS2 and ISS1 do not have any pressing issues. There is a hard drive filling up (81%) at ISS1 for the profiler however.
Today started out clear and calm in the morning with clouds moving in in the afternoon. Hasn't been much wind at all other than a very light breeze every so often.
Today ended EOP1 and IOP4 started.
One of the students notified us that the outside light of ISS3 burned out last night, so I went and replaced it after the first sounding. Other than that all systems are running well.
Sunny with very few high scattered clouds throughout the day. There were some slight breezes throughout the day but became all calm by evening.
Today we held EOP #1 which will lead into IOP #4 tomorrow. The PI's are wanting to see the transition into sundowner wind conditions, so essentially balloons will be launched for a 48hr period with the same sounding schedule we have been doing.
Everything for ISS ran smoothly with no issues arising.
Weather: Sunny, warmer than yesterday, and intermittently breezy.
ISS on Nagios showed a healthy network. The webplots have been updating.
The only issue was the the Lidar Leosphere visualizer froze and I had to exit the software. I still have not been able to bring it back online. However, it looks like data is being saved and webplots are being updated.
Nothing on the Leosphere, even after re-start
Output from the 449 MHz profiler at ISS1 at the end of the day.
Spent much of the day with Matt at S5 (Exxon) who's tower had completely blown down. Refer to Matt's blog for today.
Downed S5 tower.
Today is my last day. Liz arrived to take over ISS ops. Thanks Liz!
Until teardown ...
Weather: Sunny, clear skies all day. Calm-ish winds in the AM transitioning to westerly winds > 10 mph in the PM.
IOP3 was successful. All ISS radiosonde profiles within the 24 hr cycle made it beyond the tropopause (~200 hPa) before bursting.
A rather quiet ISS day. Bill went to ISS1 in the morning to check a few things before departing for a well-deserved break. Thank you Bill.
Nagios is mostly green except for a few orange warnings for ISS1 lidar cfradial/man netcdfs (I was told to ignore) and ISS3 allsky and webcam jpgs.
However, we can see the hourly allsky and webcam images on the webplots so we know these jpegs are being saved.
ISS3 webcam taking a photo of the 2000 UTC launch.
The afternoon was spent accompanying Matt on ISFS site visits. Refer to his blog post for today.
At the end of the day, we dropped off soil samples to cook back at ISFS base and I checked the Lidar data streaming on the DM at the ISS1 trailer.
Output from the 449 MHz profiler.