Blog

10/9/13, twh

The trailer is rocking today; Spd.10m.base = 15 m/s, gusts to 30 m/s.
Wack-a-Mote continues...
Planned to make service visit to FLR today, but too windy.
Yesterday replaced TRH.25m.near, raised near sawhorse to 2.02 m, reseated radio in rad.far mote, etc.

T/RH: Ifan 20m.rim (10/9 8:00), 25m.near (10/8 15:00) = 0, but data looks okay. 
          RH.25m.rim replaced and now matches profile,  but RH.50m.near and RH.35m.rim(?) high
          T.20m.rim fits into profile at high wind speeds today
P: ok
csat u,v: ok
csat ldiag: ok
csat w, tc: ok
kh2o: FLR flaky; FLR fluxes noisy but believable
motes: FAR ok 
           NEAR ok
           FLR rad just restarted around 10 am.
Wetness: ok
radiation: Rpile.out.far flat-lined since 10/7 09:00
Tsoil: Tsoil.3.1cm.near has offset since 10/7 12:15
Gsoil: ok
Qsoil: FLR, FAR < 0
Cvsoil: ok
2D sonic: ok, FLR wind direction not yet corrected

I installed the spare solar panel array and charge controller at the German Scintillometer on the south rim today. We did not utilize the cooler for battery containment, we used one of their existing boxes. The equipment that belongs to ISFS at this location is as follows:

1 set of solar panels on a frame

1 charge controller, mounted inside the battery box that belongs to the Germans

4 rebar stakes at the feet of the solar panels

1 solar panel connector cable

Thanks,

Tim

Tom, Gordon arrived 4:19

Replaced dessicant in 4x radiometers.

Inspected mote radios with "dental" mirror. Radio in radiation mote (ID4) was tilted a bit out of its contacts on the board. Removed the mote and took it back to the base for maintenance. Did not have a spare mote that is programmed to talk to base mote #17.

Radiation mote battery has not been charging. Removed cover on rad mote battery box. One of the connectors was loose. Saw 16V on the PV side of the little circuit board.

Departed at 4:50

Tom, Gordon arrived 2:45 local

Replaced trh transducer at 25m, due to anomalously high RH. Transducer was #43. Replaced with #25.

Checked base mote. Antenna cable connection inside white box was a bit loose, was able to tighten 3/4 turn by fingers.

Plugged ethernet cable from nearup into surge protector inside near DSM. It was going straight into the switch box.

Removed GPS jumper on soil mote. All antennas tight on motes.

Raised rad stand to 2.02 meters. Pyranometer bubble is in the circle. Pyrgeometer bubble high to south.

Left rad stand at 4:02. Departed site at 4:06

Just for fun, the plot below attempts to display all of the winds (one-hour averages) during the morning 00-06 Oct 06.  In all of these profiles, there is a jet in the middle of both the NEAR and RIM profiles.  I've shown winds from the other towers as an appropriately-colored rectangle (two in the case of the 2 slope sites).  (I tried giving all the data to the image function, but a color mess resulted.)

I've put the code (<mostly> in R) for this in the METCRAXII project R directory.

Normally we do not use data at wind directions of +/- 45 degrees from 180 (in sonic coordinates), in order to eliminate winds blowing through the tower and into the back of the sonic.  However, because of the topography at RIM, I used data at +/- 60 about the nominal axis of the gap, SSW or 202.5 deg.  The tilt plots are done with respect to the sonic u axis, which was pointed to 253 degrees.  Thus in instrument coordinates, the acceptable data is from -10 deg to 110 deg.

Also at RIM, there is a systematic descent of air into the crater, i.e. a vertical flow normal to the "surface".   I find more realistic lean angles by setting w.off = 0 and use the latter results in the RIM cal_files.

10/8/13
T/RH: ok, high RH.25m.near
P: ok
csat u,v: ok
csat ldiag: ok
csat w, tc: ok
kh2o: Looks more believable today.
Wetness: ok
radiation: sensors ok, motes intermittent
NEAR SPN1 mote down 10/8 10:00
Tsoil: sensors ok, motes intermittent
NEAR Tsoil.3.1cm down
FAR soil mote down 10/8 09:25 FlR soil mote down 10/8 08:15
Gsoil: ok
Qsoil: FLR, FAR < 0
Cvsoil: Vpile off scale, Gordon changed NIDAS limits
2D sonic: ok, FLR wind direction not yet corrected

TP01:

We've seen the Lambasoil/Csoil problem before.  I'm convinced that something is wrong in the signed math code in the TP01's PIC.  If Vpile.off goes even slightly negative, it is reported as NaN by NIDAS.  This behavior wouldn't rule out a NIDAS/mote parsing error, however the TP01's derived value for Lambdasoil (that depends on Vpile.off) also comes back as NaN, even though it should always be a positive number.  Ideally, new code would be written, loaded onto a laptop, and downloaded to the sensors at each site.  Otherwise, we can write code that sets Vpile.off always to 0 (a good assumption) and recompute Lambdasoil.  To do this, we need to know the serial number of each TP01, since this calculation involves sensor-specific calibration coefficients.  (This was the reason we had the PIC do the calculation in the first place.)

I'll talk to SRS about this code.

Single-soilT probe:

Chris and I discussed his installation of these probes (he did at FLR and FAR).  He oriented them horizontally, nominally at the level of the second? picklefork tine (1.9cm).  However, there have been HUGE soilT vertical gradients -- 15 degrees between 0.6 and 4.4cm is common in the middle of the day -- so even a small vertical (or even horizontal?) difference in position could cause the differences of up to several degrees that we are seeing.

I don't know how Kurt and Gordon positioned this sensor at NEAR.

I don't believe any action is required at this point.

The data show that the single probe appears to be at 0.6cm at flr, 2.5cm (or averaging) at near, and 4.4cm at far.

Technician from Sunstate arrived at 1 pm to service the generator. He did not bring a separate generator with him, and so power was shut down to the trailer for about 30 minutes for the service.

I did not do anything to shutdown the gaus or the data manager, but left them running on their UPS.

I turned off some monitors, and unmounted and powered off /media/backup and /media/usbdisk, but left the laptops running.

The Sunstate tech noticed a break in a return fuel hose, which was slowly leaking diesel and fixed it.

While it was down I refilled both diesel tanks.

Our UPS battery is shot. When the UPS is turned on, it beeps incessantly until all loads were removed from the battery side. Tim Lim plans to buy a replacement for his unit which he will let us use.

Otherwise all our systems came back, the WIFI antennas associated, and the DSMs reattached on their own.

The only issue was the nameserver on flux. Clients could not resolve names. I did the following, which fixed it:

sudo systemctl restart named.service
sudo systemctl restart dhcpd.service

The GAUS data manager (rack mount Dell server) powered back up on its own. I powered up the Dell tower (iss2).

The service interval is 300 hours (12.5 days), so the next service would be around Monday, Oct 20.

10/9/13, twh

Inventory of mote status

rad.far:

started 9/30 10:25   ID16
stopped 10/2 12:15   Tom shorted power input
started 10/2 16:20   Installed ID4, with repaired power cable
stopped 10/4 11:25
started 10/5 15:50    Replaced missing antenna
stopped 10/6 18:50
started 10/6 21:45   outage = 2:55
stopped 10/8 16:40   Removed to reseat antenna
started 10/8 17:50   Replaced
tx power 10/8: pl=4
xb reset change 10/10 9:00 xr=2hours, gps=off
xb-hb crontab 10/14 running every 4hours
still up 10/16 09:00

soil.far:

started 9/29 11:40   ID22
stopped 10/8 09:30
started 10/8 12:25   outage = 2:55
tx power 10/8: pl=4
xb reset change 10/10 9:00 xr=2hours, gps=off
xb-hb crontab 10/14 running every 4hours
stopped 10/16 13:45
started 10/16 15:40   outage = 1:55

rad.near

started 9/29 17:45   ID8
stopped 10/2 03:30
started 10/2 06:25   outage = 2:55
stopped 10/6 06:45
started 10/6 12:45   outage = 3:00
10/6 16:00 steady yellow light on mote, recycled power, but data coming in already
stopped 10/8 15:30
started 10/8 18:15   outage = 2:45
tx power 10/8: pl=4
stopped 10/9 12:20
started 10/9 15:15   outage = 2:45
xb reset change 10/10 9:00 xr=off, gps=off
xb-hb crontab 10/14 running every 4hours
stopped 10/15 01:35
started 10/15 05:20   outage = 3:45
still up 10/16 09:00

soil.near

started 9/30 11:20 ID3
stopped 9/30 20:30
started 9/30 23:25   outage = 2:55
stopped 10/1 02:30
started 10/1 05:25   outage = 2:55
stopped 10/3 17:30
started 10/3 19:20   outage = 2:55
stopped 10/6 01:25
started 10/6 04:20   outage = 2:55
tx power 10/8: pl=3
xb reset change 10/10 9:00 xr=hourly, gps=off
xb-hb crontab 10/14 running every 4hours
xb reset change 10/14 12:00 xr=off, gps=off
still up 10/16 09:00

SPN1.near

started 9/30 16:05   ID18 fried at installation, replaced with ID10
stopped 10/1 04:10
started 10/1 07:05   outage = 2:55
stopped 10/2 07:10
started 10/2 10:05   outage = 2:55
stopped 10/5 21:45
started 10/6 00:40   outage = 2:55
10/6 16:00 steady yellow light on mote, recycled power, but data coming in already
stopped 10/8 04:00
started 10/8 06:55   outage = 2:55
tx power 10/8 pl=4
stopped 10/8 10:00
started 10/8 12:55   outage = 2:55
stopped 10/8 19:00
started 10/8 20:30   outage = 1:30
xb reset change 10/10 9:00 xr=off, gps=off
xb-hb crontab 10/14 running every 4hours
stopped 10/14 05:25   up for 5+ days
started 10/14 09:15   outage = 3:50
still up 10/16 09:00

rad.flr

started 9/30 14:40 ID19
stopped 9/30 21:25
started 9/30 22:45   outage = 1:20
stopped 10/1 08:55
started 10/1 10:50   outage = 1:55
stopped 10/2 18:20
started 10/2 21:15   outage = 2:55
stopped 10/3 20:55
started 10/4 15:10 Replaced with mote ID17; radio of ID19 had worked loose
stopped 10/5 03:15
started 10/5 21:00   outage = 17:45
stopped 10/7 04:20
started 10/8 02:00   outage = 21:40
tx power 10/8 pl=4
stopped 10/9 06:40
started 10/9 09:35   outage = 2:55
stopped 10/9 12:40
started 10/9 15:35   outage = 2:55
stopped 10/10 00:50
started 10/10 10:40   outage = 9:50
stopped 10/10 12:05   then intermittently off and on
xb reset change 10/10 9:00 xr=hourly, gps=off
started 10/10 17:40
stopped 10/12 19:55
started 10/13 16:05   outage = 20:10
stopped 10/14 01:05
started 10/14 05:25   outage = 4:20
xb-hb crontab 10/14 running every 4hours
xb reset change 10/14 12:00 xr=12hours, gps=off
stopped 10/14 21:30
started 10/15 01:00   outage = 3:30
stopped 10/15 08:15
started 10/15 09:05   outage = 00:50
stopped 10/15 15:15
started 10/15 16:15   outage = 1:00
stopped 10/15 21:20
started 10/15 22:25   outage = 1:05
stopped 10/16 00:10
started 10/16 01:10   outage = 1:00
stopped 10/16 03:20
started 10/16 04:30   outage = 1:00
stopped 10/16 06:15
started 10/16 07:15   outage = 1:00
hardwired serial cable about 14:00 10/16

soil.flr

started 9/30 12:40 ID2
stopped 9/30 15:35
started 9/30 17:25   outage = 1:50
stopped 9/30 20:00
started 9/30 22:55   outage = 2:55
stopped 10/1 03:00
started 10/1 05:30   outage = 2:30
stopped 10/1 10:20
started 10/1 15:10   outage = 2:50
stopped 10/3 05:20
started 10/4 02:20   outage = 21:00
stopped 10/5 03:30
started 10/5 06:35   outage = 3:05
stopped 10/6 09:25
started 10/7 01:05   outage = 3:40
stopped 10/8 08:20
started 10/9 04:00   outage = 19:40
tx power 10/11 pl=3
stopped 10/10 15:50
xb reset change 10/10 9:00 xr=hourly, gps=off
started 10/11 04:40   outage = 11:50
stopped 10/11 11:15
started 10/11 13:50   outage = 2:35
stopped 10/12 14:20
started 10/13 07:20   outage = 14:00
stopped 10/14 00:05
started 10/14 01:00
stopped 10/14 09:55
started 10/14 11:55   outage = 2:00
xb-hb crontab 10/14 running every 4hours
xb reset change 10/14 12:00 xr=12hours, gps=off
stopped 10/14 17:10
started 10/14 21:00   outage = 3:50
stopped 10/15 01:30
started 10/15 05:00   outage = 3:30
stopped 10/15 09:35
started 10/15 13:00   outage = 3:25
stopped 10/15 21:20
started 10/15 22:25   outage = 1:05
stopped 10/15 05:10
started 10/15 05:10   outage = 1:00
hardwired serial cable about 14:00 10/16

SPN1.flr

started 10/1 10:50 ID1
stopped 10/1 15:20
started 10/2 03:55   outage = 12:35
stopped 10/4 01:45
started 10/4 16:05   outage = 14:20
stopped 10/5 05:45
started 10/6 00:40   outage = 18:55
stopped 10/7 05:25
started 10/8 03:35   outage = 22:10
tx power 10/8 pl=4
stopped 10/10 00:40
started 10/10 08:30   outage = 7:50
stopped 10/10 09:45   then intermittently off and on
xb reset change 10/10 9:00 xr=hourly, gps=off
started 10/11 13:50
stopped 10/12 07:55
started 10/13 06:10   outage = 22:15
stopped 10/14 05:30
started 10/14 09:00   outage = 3:30
xb-hb crontab 10/14 running every 4hours
xb reset change 10/14 12:00 xr=12hours, gps=off
stopped 10/15 01:45
started 10/15 05:15   outage = 3:30
stopped 10/15 09:10
started 10/15 13:00   outage = 3:50
stopped 10/15 21:20
started 10/15 22:25   outage = 1:05
stopped 10/16 00:10
started 10/16 01:10   outage = 1:00
stopped 10/16 11:45
started 10/16 13:55   outage = 2:10
hardwired serial cable about 14:00 10/16

10/7/13

T/RH: ok, high mr.25m.near; high T.20m.rim is real
P: ok
csat u,v: ok
csat ldiag: ok
csat w, tc: ok
kh2o: Sebastian checked FLR connectors 18:15, 10/6, , so we now have a signal but the variance is much higher than near and far.
Wetness: currently missing at FLR; mote is likely down
radiation: sensors ok, motes intermittent
FLR rad mote went down 10/7 04:00
NEAR rad mote restarted 10/6 12:40 
FAR rad mote down 10/6 18:50 - 21:45
SPN1.flr mote down 10/7 05:20
Tsoil: sensors ok, motes intermittent
NEAR soil mote down 10/6 01:25 - 04:20
FLR soil mote down 10/6 09:00 to 10/7 01:10
Gsoil: ok
Qsoil: FLR, FAR < 0
Cvsoil: intermittent, but this is calculated, not measured
2D sonic: ok, FLR wind direction not yet corrected

Norbert (German research group) has a scintillometer above the rim tower site powered by two 100 amp deep cycle gell cell batteries, but has no charging system other than hauling a generator up to the remote site. He has asked if the spare (backup) panels are available to be placed up near their site to facilitate charging. Norbert has been made aware that these are ISFS spares, and if any need arises, he must immediately return these to the trailer for ISFS use. Tom and Gordon have agreed to the loan, and I will help Norbert relocate and install these on Tuesday. He is also going to borrow a spare cooler and charge controller. Please be sure to retrieve these at end of project!

At the end of IOP #01, I went to investigate poor repeater performance at the rim, and found the main battery pack voltage at 8.5V (!) I removed all repeater gear and box, relocating them near the base of the rim tower battery bank and solar panels. I replaced the now assumed bad battery with another good battery from the MGAUS truck, (checking before and after voltages) and placed this in parallel with the the western most (closest to tower) battery box. This battery and its load are placed in a blue rubbermaid box. The immediate results were improved repeater performance, which will hopefully be more useful in the coming weeks.

Gordon and I have discussed increased monitoring of battery voltages, and comparing them to previous days to insure that this does no harm to the power bank at the tower. This is a minor load increase, but also a 33% increase in storage capacity, hopefully able to store excess PV energy collected during the day. Please pay increased attention to the morning battery voltages to insure this is the case. If anyone needs to remove the repeater, simply open the cooler and remove the clamps to the battery leads.

Just wanted to write down some thoughts on the FLR and FAR soil probe installations.

FLR:

Soil was very loose.  Tried to get probes in their respectable places, but cannot guarantee with such loose soil (sand).  I did put in single probe horizontal and did not realize the angle of install on old single probe temperature.  Very windy down there with loose soil I was worried about erosion and exposure of probes at the end of this project.

FAR:

All seemed okay on install.  More stable sand but was still loose.  Single temperature probe was installed horizontally. 

 This is at start of project.

RIM TOWER

NE (in to out)

300,440,440,400

W

425,475,500,460

S

290,470,470,460