Sunny and hazy (less hazy on the north side of the mountains); winds started to pickup in the afternoon. All sites looked good and were reporting all data, so did not visit any ISFS sites. Went with Isabel to ISS 2 to launch balloons for the day for IOP-6.
Laura Joined us today, and Isabel leaves us tomorow, so Isabel and Laura went to ISS2 and ISS3, and Isabel gave Laura a refresher on balloon launching.
I took care of some stuff at base before heading off to check on the sites along the coast (S1, S3 and S5)
S5 (Exxon): had some general looseness of all stakes, I added a two more stakes to the tripod, and then added rocks to all three pads. The generaly upwind leg already had two 70lb sand bags that Matt had installed. The NR01 strap was a little loose, and I tightened it. I also removed two dry weed stalks that were almost touching the NR01 (these were anomilious weeds, where eveything else was about 2-3 feet high). I was able to break the weed stalk off without disturbing anything else. Forgot the water at S5, but looked over NR01 windows and they looked good.
S3: Cleaned the NR01 windows, and tighted the rachet strap two notches (it had no tension). Guy wires and everything else looked good.
NR01 was really dirty, and the white tape cover seems to have a large open flap.
S1: Cleaned the NR01, and checked the guys. Snugged up a few lock nuts, and everything else looked and felt good.
As a check of the ec150 sanity, I've produced the following plot from 30min of data from yesterday:
These are power spectra from the EC150 at s16 (red) and s11 (green) and from H2O derived from the TRHs at these sites (blue and cyan, respectively). At low frequencies, I would say that the agreement between the blue and red lines are similar to the agreement of the green and cyan lines, indicating that the gain of the EC150 water vapor channel (and thus hopefully the co2 channel) is probably okay. For flux calculations, biases – even large ones – don't matter. Also, as Chris has pointed out, co2 from this project isn't a high priority.
Thus, I'd say that swapping the EC150 here would be the next step, but a low priority.
P.S. Below is the R code I used to generate this plot:
spspec = function(x) specsmooth(pspectra(fftw(x)))
dpar(lenday=1)
dpar.now()
dpar.prev()
dpar(lenmin=30)
iod = prep(c("h2o.s16","h2o.s11"))
ec150 = readts(iod)
close(iod)
iod = prep(c("T.2m.s16","RH.2m.s16","T.2m.s11","RH.2m.s11"))
trh = readts(iod)
close(iod)
Rm = 0.4617
trh[trh[,c("RH.2m.s16"]>100,c("RH.2m.s16"] = 100
trh[trh[,c("RH.2m.s11"]>100,c("RH.2m.s11"] = 100
Pv = dat("satvp", TC = trh[,"T.2m.s16"]) * trh[,"RH.2m.s16"]
h2o = Pv/Rm/(trh[,"T.2m.s16"] + 273.15)
Pv = dat("satvp", TC = trh[,"T.2m.s11"]) * trh[,"RH.2m.s11"]
h2o = Cbind(h2o, Pv/Rm/(trh[,"T.2m.s11"] + 273.15))
plot(spspec(ec150))
lines(spspec(h2o[,1]),col=4)
lines(spspec(h2o[,2]),col=5)
legend(locator(1),c("H2O.s16","H2O.s11"),col=4:5,lty=c(1,1))
Weather day was warm with minimal winds, very hazy with some light clouds in the afternoon.
All data from sites looked good. Decided to go to S12 (Gaviota), to get a second soil sample, and check out the site. The road up to the site is still good, with some expertly cleared brush (Thanks Steve), still has plenty of steep spots, tight turns and occasional hikers.
At site:
Cleaned the NR01 windows.
Tightened the inner guy wires (only slightly loose, and outer guys seemed good).
Took soil sample.
Verified boom angle.
Leila had mentioned that the winds at this site looked different from most of the other sites. Everything looked good and the boom angle is within a few degrees of NW, (matching what was previously written). I think the local topography for this site is odd. Attached is a picture looking due West, and you can see that wind can kind of funnel up from the valley toward the site, and being on the ridge can also create other weird effects.
Below is a picture of S12 looking due West
This morning’s data check found no issues, other than the intermittent data from S11 (with poor cell reception).
Finish the soil for S16, and realized that we forgot to measure the boom angle for S16. The boom angle will have to be measured during the next service visit, or the last soil sample.
Unfortunately, the EC150 for S16 is still reading maxed CO2 - after the replacement of the scrubber bottles.
At Base, finished inputting calibration coefficients into TRHs. We now have 7 spare TRH probes, all are in the truck console.
Looked over TRH bodies, and tried to determine what was wrong with the spare unit that was in the truck (ended up not having 12V on the wires to the Fan). Moved this unit to the bad pile. We now only have 2 spare TRH bodies, both are in the truck with spares.
Tony has been fixing some of the TRH bodies. If we have anymore TRH body failures in the next few days, Tony will fedex a few to us. Otherwise he will pass a few TRH bodies to Jacquie for her to bring out as spares.
Data check showed all stations were operational and reporting data except for the NR01 from S9 (soils were still reporting data, but NR01 was reporting NAs).
Cleanup up the pod to access the spare NR01s.
Drove up to S9. All cables to the NR01 and mote looked good. Did pull out and reseat cable to NR01, wetness sensor, and mote.
Connected to DSM with console. Decided to reboot DSM, however nidas did not restart on boot. (not sure what caused this.
Eventually nidas was started manually, and all sensors started reporting including the NR01.
Also cleaned the windows on the NR01, and tightened one inner guy wire.
Also stopped by S10 to check it out.
Cleaned windows on NR01.
All guy wires tight.
Everything else looked good at S10
Weather was calm, warm and sunny, high around 80. A slight bit of haze on the horizon.
Isabel and I decided to take the white truck up to Little Pine (S16). We needed to get a soil ample and replace scrubber bottles on the EC150. The road up to little pine was in great shape (much better than during setup). The road had been recently graded ( about 2-3 weeks prior based on conversations with mt. bikers). The white truck should be fine for tear down unless there are other significant rainfall events. Based on amount of dirt biker and mt. biker traffic, I would suggest teardown on a weekday.
S16 (Little Pine):
- Lowered Tower and replaced scrubber bottles on the EC150
- Tried to replace TRH (due to 4/22 dropouts), but replacement did not work, so kept old one
- Tightened all guy wires both inner and outer
- Cleaned NR01
- NR01 ratchet strap tail was loose and blowing into sample volume, cleaned this up & retied.
- Pounded in one slightly loose stake
- Took soil sample
Weather was calm, warm, sunny, severe clear with nary a cloud in the sky.
S7 (Reagan ranch):
S7 was still showing no data, so checked in with both Micah and Brent, and received permission to enter the site. Micah was able to open gate once I contacted him (24) from the gate.
Tried to connect to the R Pi with console cable, and with wifi; neither worked. Both Red and green LED on the Pi were solid on (Green is normally blinking). Data had stopped recording to the raw data file at 6:27 on April 22, and we had missed ~36 hours of data. Rebooted several times, and verified data was being recorded, also notified Gary about S7 Pi lockup.
- Reboot Pi Several time
- Guy wires all felt tight
- Cleaned NR01
- NR01 ratchet strap tail was loose and blowing into sample volume, cleaned this up retied.
S4:
S4 was also showing all NAs. Went to site. I was able to log in with the console, and files were still being saved to the USB stick. Rebooted to get the modem back on line.
- Rebooted Pi
- removed unterminated ribbon cable from DSM
- Added black tape over DSM power button to reduce visibility (in highly visible location)
- Guy wires all felt tight
- Cleaned NR01
- NR01 strap itself was loose, ratcheted strap about 4 clicks to get more tension.
Went back to base and completed weighing of 3 soil samples
The weather today remained unsettled, with scattered showers and clouds over parts of the SY mountains throughout the day. Chris and I spent the day hitting sites 13, 14, and 15 for soils and boom angles, while also cleaning radiometers and re-tensioning some guy wires at 14 and 15. I also showed Chris some of the tools we use to monitor the data and network status.
Other small things of note-
- S14: We noticed that the CSAT boom at S14 is completely upside down. Perhaps this is explained in a previous blog post that I missed, but I thought it was noteworthy.
- S7: Many sites have been intermittently offline again today, but S7 has been completely offline since ~11:22 pm last night. I suspect it’s a cell network issue because s6 and 8 were both flaky as well. Chris and I also had spotty coverage at times today.
- S16: The TRH wasn’t reporting all morning and was also out for a short period last night. Rebooting the DSM did not seem to help, but it eventually came back on its own in the afternoon. Could moisture be getting into the connection?
- S10: Had to restart Nidas twice to bring back up the power monitor. Also Qsoil was not reporting all day, but a port power cycle brought it back in the late afternoon.
Chris arrived today and Liz left back to Boulder. We didn’t get together much, but thanks for the company Liz!
The weather today was unsettled, with periods of low cumulus. Deeper cumulus persisted over the SY mountains throughout the day, making visiting any of the ridge sites potentially perilous. There were no new data or sensor issues this morning and I had meetings, so Chris and I decided not to visit any sites today. Chris spent the afternoon configuring some of the spare sensors that he brought from Boulder and I recorded yesterday’s dried soil samples. I also purchased a replacement 9V battery for the scale in the base trailer. We still need to collect a 2nd round of soils from 12, 13, 14, 15, 16, and 17 - as well as measuring boom angles at the same (except 12).
There were a lot of short outages across the domain today, which I suspect to be due to the weather. I noted that S1, S6, S8, S10, S11, S12, and S13 were all down on Nagios at various times of the day, but all have come back since.
Today was cool and mostly clear, although the mountains in the eastern part of the domain were cloud-topped through the morning. I had a busy day today running around to various sites for soil samples and other tasks, summarized below.
- S10: Restarted nidas brought back up the power monitor stream.
- S11: Spent the morning in the clouds while collecting a soil sample and troubleshooting the Victron, which has been out for about a week. It turns out the cable was unplugged (probably due to wind). I also swapped out a new cable because the connector seemed bent. After restarting nidas, the power monitor came back and has been reporting data since. I cleaned the radiometer around 11 am, and added zip ties to the guy wires. Finally I measured the boom angle at 316m.
- S9: Drove past intending to check out and possibly replace the TP01, but someone was parked in the pull out, so I decided to not stop.
- S8: Took a soil sample and cleaned the radiometer around 2:50 pm. This site takes forever to get to.
- S7: Visited the site to take a soil sample and measure the boom angle (311m). I cleaned the radiometer around 3:45 pm.
- S6: Took a soil sample and cleaned the radiometer around 4:20 pm.
Today was mostly clear with periods of high clouds and quite warm temperatures. Some sites saw fairly strong winds throughout the day, while it was very windy in the afternoon and evening in town. This morning all sensors were operating nominally, other than the few known issues (S16 IRGA, S11 Victron, S9 TP01). Today I collected soil samples, measured boom angles, and cleaned radiometers at S2 and S4. Tomorrow I will continue doing round 2 of soil samples.
Today we transitioned from EOP 1 to IOP 4. The ISFS network was in good shape, so it was another light day. Gary remotely rebooted S18, which resolved the usb disk issue. I visited S5 to troubleshoot the sonic and generally check out the station. A simple power cycle on the port seemed to do the trick and we have been getting reasonable data since. I also re-tensioned some of the guy wires, so the tower is much more plumb than we left it. Before I left I remembered to measured the boom angle, which was 136 magnetic. Later in the day I collected a soil sample at s1 and measured the boom angle at S3 at 136 magnetic.
Everything else behaved pretty well today, but Steve did notice that the TP01 at S9 is reporting likely bad data. I will consider replacing it if I make it to S9 for soils in the next few days.
Today was EOP 1, which is planned to continue directly into IOP 4 tomorrow, leaving me without support from the ISS crew. Fortunately all ISFS systems are functioning reasonably well and today was a light day. Some ongoing issues are summarized below.
- S5: CSAT ldiag remains at 1 since we raised the tower. I will look for another spare in the Pod tomorrow, as I suspect we need to swap the sonic to fix this issue. I may visit the site tomorrow to visually inspect things.
- S11: No Vbatt.
- S16: Bad CO2.
- S18: According to Nagios the raw_data has not been rsync’d in >7 hrs. It also seems like the usb disk is not being found on the DSM - confirmed by nothing returned by lsu command. It seems like another visit to s18 is in order to reboot the DSM.
The weather today started mostly sunny, quickly changed to overcast with drizzle, before clearing and becoming breezy in the afternoon. All sites except s17 measured some very light rain today.
During the morning overcast and drizzle, Liz and I set out to finish repairs at s5. We dressed cables on the EC150 boom, again inspected the tower, and raised the mast before noon. We had one 70 pound sandbag on the northwest tripod leg while raising and the leg barely budged, but added a 2nd sandbag to further secure it and could add a 3rd as well if necessary. We also had to re-tension all guy wires and added zip ties to help keep them secure. At this point the tower is up and reporting data, however since then there have been no CO2, H2O, or CSAT winds coming in. I hope this is due to the weather, since the ldiag is 1 and it was precipitating when we left the site.
After breaking for lunch, we visited s18 to troubleshoot the NR01 that stopped reporting a few days ago. Everything looked visibly normal, so we reseated and loosened the NR01 cable to no avail. However, a power cycle of the mote brought the radiation back up. This time we saw about a dozen goats - so the streak of seeing new animals at every visit continues.
I had noticed that s4 was offline and unreachable since sometime this afternoon, so we paid the site a visit at the end of the day. I could not connect via usb cable, so I connected to the WiFi instead. Data was coming in and looked normal, save for nothing coming from the NR01. A couple of mote power cycles seemed to do the trick, but the site was still off the network when we left. It is now back up, as of 10 pm and the NR01 is reporting.
A few other notes:
- S16: TRH seems to have been reporting NA’s during the middle of the day, but is now alright.
- A scattering of NA’s across several variables at multiple sites, likely due to the weather.
- Jacquie left this morning and I'd like to thank her for all the help during the first week of my ops shift. Thanks Jacquie!