Blog

This done at about 0820 by Gordon (with Steve looking on).  The new sensor took about 10min to cool down, but now looks much better.

The old TRH housing was a bit loose.  I'll take it apart now to check.  Gordon noted that the fan had been running when he removed it.

P.S. It appears that the fan was wired backwards.  When tested, air was blowing down from the plate.  Also noticed that the black/red wires from the fan were opposite those from another spare in the trailer.  Have marked this sensor to be fixed in Boulder (since it isn't obvious to me how to cleanly rewire the small black connector on the end of these wires).

On flux, dsm_server should always be running.

If you make a cal file change, do the following:

1. dsm_server_restart
2. statsproc.sh
3. shutdown cockpit
4. cockpit.sh

The statsproc.sh script starts 3 statsproc processes. One creates the normal 5 minute, boom normal data. Another creates the 5 minute tilt corrected winds, The third generates 30 second means of speeds, directions, temperatures and pressure for the IOP plots.

To rerun all the covars from the beginning of the project, do:

batch
covar_redo
ctrl-D

That script will run statsproc on the merged data up to 0Z of the current day. It then runs statsproc on today's real-time raw data. Therefore if the network has been congested, the netcdf statistics for today may have gaps.

FYI: the dsm_server process also provides a feed of surface met data over a socket to the GAUS system in the trailer.

Info

See these entries on the SEW wiki:

• http://wiki.eol.ucar.edu/sew/ISFS/R
• http://wiki.eol.ucar.edu/sew/R (needs cleaning up)

IOP Plots

On flux, in $ISFF/projects/METCRAXII/ISFF/R is "iop_plots.R", "dataset.R" and a makefile. If you make modifications or add an R file, do a "make" in that directory, then quit/restart R. There is no "synchronize()". You would have to do a "detach(...)", followed by an "attach(...)", so right now it is easiest to quit/restart.

I have not yet created a project.init.R. Would also have to add to $HOME/.Rprofile, to run it automatically.

To run the iop plots, run function "plot.loop()". It reads from dataset("field-30sec") and creates 9 graphics windows.

There is also a script "iop_plots.sh" which should run R and create those plots. There should be a shortcut icon on the desktop called "Rplots" which should run that script.

Updating our R packages

My intention is to put our packages on http://www.eol.ucar.edu/software/R, but currently the web server does not allow downloads from that location. In the meantime, the packages are at http://www.eol.ucar.edu/isf/projects/METCRAXII/isfs/R. I've made a script "update_R" in the project script directory which will fetch and install the packages on Linux.

Clayton and I were in the crater, starting down at 11:15 and returning to the visitor center about 3.  Tasks done:

FLR tower:

- Swapped serializer, noted voltage returned to 1.68V (rather than 0.05V seen before we set out).  

I've plotted kh2o and tc time series for both far and flr.  tc and kh2o are well correlated in both cases.  Also in both cases kh2o is noisier than tc, mostly noticeable when the variance is low.  I conclude that this serializer is working acceptably well.

- Removed FET jumper on PWR mote serial port.  Verified that we could still talk to it.

SSW1:

- Battery was at 3.3V, even with full sun.  Replaced with spare battery module that we had brought down.  Gordon reported that it was back up.

SW:

- Battery was at 12.48.  Just fine.

WSW:

- Battery at 14.56, slowly dropped when solar panel disconnected, which seemed normal.

W:

- Battery was at 11.28.  Didn't change when charger removed, so guessed that battery wasn't charging.  Opened up box and noticed diode on battery side of charger.  This <seemed> okay, but pulled it out and tried again.  Voltage jumped to 11.68 and continued to slowly climb (in full sun).  Declared it fixed.

Enjoyed our tumbleweed swim.  Enjoyed less our game of boulder rolling.

10/14/13

Summary:
SPO arrived yesterday, will replace floor serializer today.
Wack-A-Mote on crater floor
Rpile.out.near out again, but may be recovering
IOP 3 expected Wednesday evening, 4 & 5 perhaps Friday & Sunday.

T/RH: T.20m.rim often a little high, but also sometimes in line with others.  When does this occur?  At night??
          RH.40m.near and RH.35m.rim perhaps a bit high.
P: ok
csat u,v: ok
csat ldiag: ldiag.near > 0.01 for 3 times for lower dsm (dropped samples sent in real time?)
csat w, tc: ok
kh2o: FLR still flaky; FLR fluxes noisy but believable
motes: FAR ok for past 3 days
           NEAR SPN1 out 10/14 05:25 - 09:15
           FLR all off and on; SPN1 currently missing
Wetness: a bit of dew or frost this morning
radiation: Rpile.out.far flat-lined after 10/13, about 16:45
Tsoil: Tsoil.4.4cm.near has recovered
         Tsoil.2.5cm.flr & far (linear avg) does not fit profile
Gsoil: ok
Qsoil: ok
Cvsoil: ok
2D sonic: Dir.10m.flr at times appears to be off by 180 deg.

Clayton and I refilled the ISFS base trailer generator on Sunday the 13th at approximately 4pm. The lower tank was approximately 60% full, so it could have been pushed to Monday, but it was good training and orientation for Clayton. The new refill by date is now Wednesday at the earliest, but Thursday would be a good bet.

inadyn is a process that runs on the rimup DSM periodically to update the dynamic dns entry for isfs1.dyndns.org. The idea is that it will set the dns entry to the address of the verizon modem on the cradlepoint router. However, the default gateway on that DSM was set to 192.168.0.1, which is the router for the hughes satellite.

As a result, the IP address for isfs1.dyndns.org was being set to the WAN IP address of the Hughes directway from time to time.

I changed the default gateway on that DSM to 192.168.0.5, which is the cradlepoint router.

Also, nearup was also running inadyn to update the DYNDNS entry, which it shouldn't.

So now, the address for isfs1.dyndns.org should now consistently be the Verizon modem address.

At flr, near and far, crontab will now run rserial to send a "hb" followed by a CRNL to the base mote every 4 hours. at 00:00 UTC, 04:00 UTC, etc:

0 */4 * * * echo "hb"$'\r'$'\n' | rserial /dev/ttyS1 localhost > /dev/null 2>&1

The base motes are on ttyS2 at flr, ttyS1 at near and ttyS7 at far.

10/13/13
Summary:
Second IOP on the night of October 11-12

10/11: 
Replaced floor kh2o
Nothing obviously wrong with P.nne.flr, but Sebastian swapped with P.ssw1.flr
Replaced battery at P.sw.flr
Checked battery voltages at all pressure motes on floor.
Finished replacing dessicant in all radiometers

10/12
Measured guy wire tensions.
Recycling power fixed TRH.20m.rim and Rpile.out.far
Gordon diagnosed and fixed problem with power on the serial ports for the
power-monitering motes
ETC.

10/13
FLR kh2o still dropping out.  Is the serializer having problems?

T/RH: T.20m.rim often a little high, but also sometimes in line with others.  When does this occur?
          RH.40m.near and RH.35m.rim perhaps a bit high.
P: ok
csat u,v: ok
csat ldiag: some ldiag > 0 morning of Oct 12
csat w, tc: ok
kh2o: FLR still flaky; FLR fluxes noisy but believable
motes: FAR ok for past 3 days
           NEAR ok for past 3 days
           FLR all off and on; rad currently missing
Wetness: dew or frost night of October 11-12 (IOP)
radiation: ok
Tsoil: Tsoil.4.4cm.near has offset since 10/12 10:50
         Tsoil.2.5cm.flr & far (linear avg) does not fit profile
Gsoil: ok
Qsoil: ok
Cvsoil: ok
2D sonic: Dir.10m.flr at times appears to be off by 180 deg.

10/12/13

14:10 Visited NEAR to measure guy wire tensions and remove power jumpers on serial ports for power motes.

16:50 Tested FAR pyrgeometer by plugging into radiation mote.  Worked okay.

10/12/13 14:38-14:45

Added a third stake to the soil mote 'mast'.

Removed serial FET plus power jumpers on serial ports for power monitor motes.

Removed down-facing pyrgeometer which was flat-lined for Rpile.

Opened the pyrgeometer in the lab and checked the thermopile voltage and continuity of the connectors.  All looked okay.

17:18 After testing pyrgeometer at NEAR, reinstalled at FAR.  Chances are we could have 'fixed' the pyrgeometer simply with by switching the power off and on.

10/12, twh

Gordon and I visited the RIM station to measure guy wire tensions, add strap to hold the door closed to the upper mote, and to replace the 20m TRH.  

The TRH was fixed by removing jumpers unnecessarily providing power to the power monitors.  See related blog post.

10/12/13 Gordon and Tom measured guy wire tensions.

RIM about 13:00, very low wind speeds

NEAR about 14:10, still low wind speeds

Measured from low to high

10/21/13 Steve S&O; NEAR about 12:00, RIM about 12:30

Marked in orange are the only 2 that changed significantly.

From Sebastian:

IOP 2, 11 October 2013
16:10 MST removed sensor from ssw1
16:25 MST connected ssw1 sensor to nne
16:43 MST connected nne sensor to ssw1

Went to rim-tower to replace the TRH at 20m since it has been reading way off scale, and we have not been able to reset it with ctrl-R or "eio 15 0/1". See https://wiki.ucar.edu/x/6RtaE and https://wiki.ucar.edu/x/FRdaE

It is connected to port 15. The power monitors for rim and rimup are in ports 14 and 16. I wondered if somehow the power monitors are "back" powering the interface board through their serial cables. To test this, I removed the jumpers labeled FET and +V for ports 14 and 16. The power motes kept coming in, so, as expected, they are getting their power from the battery. After removing those jumpers, I can now power cycle the TRH with eio, and reset it with ctrl-R. After that it looks like the TRH is reporting good data, so didn't replace it.

Prior to removing the jumpers, at some point the rim dsm crashed as I've seen once or twice before on this project, reporting "too much work for irq 4", which is the irq of the second serial board. I believe this happens after an "eio 15 0", which might exacerbate the "back" power problem. See https://wiki.ucar.edu/x/jwtaE

Since this seemed to help rim, I then removed the power monitor jumpers at near, far and base. All those motes continued to report.

Post-Project Update, JohnM. 15Nov13.

I cannot reproduce the reported 'back-trh-power' problem coming from the power monitor motes.   I did a test using the rim dsm, a trh plugged into ttyS15 and the same 2 power monitor motes connected into ttyS14/16 and with all the jumpers re-installed, although there were no other sensors installed and the exact same cables cannot be determined.   Nevertheless this is the same sub-configuration that Gordon had with 2 separate battery boxes for pwrmote ID8 and ID9 and one running the dsm.  I tried eio 15 0/1 with eio 14/16 0/1, etc. and everything worked as advertized.    I really didn't think the problem reported could have been due to the power monitor motes because as he mentioned, they're powered independently by the batteries themselves and the interface cable has only gnd,tx,rx on it: i.e. no power.    I suppose there is a possibility that there was a bad cable but i'm doubtful of that too because it is hard to see how either a bad cable could have been working, which it was, or how it could have back-fed power to port 15 forcing it's way through the power control fets specific to each port.   What exactly happened is a mystery.   Could there have been an issue in the Diamond board that caused either both, or the wrong digital i/o line to be pulled?   Was the dsm restarted or 'tickled' or was the power shutdown while the jumpers were changed and the trh worked on?   Could one of the fet jumpers for the trh have been loose or intermittent in cold/warm and possibly contact re-established?    Etc.,etc.   I don't know but feel it's safe to say the power monitor motes cannot back-feed power through their serial port to a dsm.   And if in doubt they can also be setup for wireless (bluetooth, xbee or wifi) data delivery.