Two merged archive files have been found with fairly big blocks of backwards samples.

The statsproc process logs the following after every 100 backwards samples, and many were seen in the logs of processing the PCAPS data.

2013-01-01,22:45:17|WARNING|NearestResampler: sample id 2,102 backwards by 2325.2 sec at 2011 01 26 22:21:05.469201

I re-sorted the data with a 1 hour sorter in nidsmerge:

nidsmerge -s "2011 01 29 16:00" -e "2011 01 30 00:00" -i isfs_20110129_160000.dat.bz2 -r 3600 -o $s/isfs_%Y%m%d_%H%M%S.dat.bz2 -l $((8*3600))

The files have been replaced on /scr/isfs and on the HPSS.

Here are data_stats listing of the old and new files.

data_stats isfs_20110126_160000.dat.bz2
2013-01-02,09:30:18\|NOTICE\|parsing: /net/isf/isff/projects/PCAPS/ISFF/config/pcaps.xml
Exception: EOFException: isfs_20110126_160000.dat.bz2: open: EOF
sensor                   dsm sampid    nsamps \|------\- start \-----\--\|  \|-\----\- end \-----\|    rate    minMaxDT(sec) minMaxLen
isfs1:/dev/ttyS2           1     20     30083 2011 01 26 16:00:00.138  01 26 23:59:59.726    1.04-2332.729 890.893   19   19
isfs1:/var/tmp/gps_pty0    1     30     60658 2011 01 26 16:00:00.500  01 26 23:59:59.708    2.11-2333.150 890.859   51   73
isfs1:/dev/ttyS4           1     40     30604 2011 01 26 16:00:00.559  01 26 23:59:59.606    1.06-2332.339 890.471   18   18
isfs1:/dev/ttyS5           1     50     27235 2011 01 26 16:00:00.201  01 26 23:59:59.538    0.95-2332.128 891.041   21   39
isfs1:/dev/ttyS1           1    100   1804701 2011 01 26 16:00:00.004  01 26 23:59:59.995   62.66-2333.789 890.018   14   14
isfs1:/dev/ttyS6           1  32768     18054 2011 01 26 16:00:02.016  01 26 23:59:59.688    0.63-2332.560 892.475   21   80
isfs2:/dev/ttyS2           2     20     30082 2011 01 26 16:00:00.403  01 26 23:59:59.191    1.04-2333.743 890.905   20   20
isfs2:/var/tmp/gps_pty0    2     30     60157 2011 01 26 16:00:00.505  01 26 23:59:59.742    2.09-2334.145 890.857   72   73
isfs2:/dev/ttyS5           2     50     30729 2011 01 26 16:00:00.105  01 26 23:59:59.569    1.07-2333.530 891.714   29   31
isfs2:/dev/ttyS1           2    100   1804650 2011 01 26 16:00:00.006  01 26 23:59:59.989   62.66-2334.522 890.388   14   14
isfs2:/dev/ttyS6           2  32768     18062 2011 01 26 16:00:01.207  01 26 23:59:58.369    0.63-2330.950 894.940    6  116
isfs3:/dev/ttyS2           3     20     30084 2011 01 26 16:00:00.059  01 26 23:59:59.855    1.04-2333.907 890.009   20   20
isfs3:/var/tmp/gps_pty0    3     30     60160 2011 01 26 16:00:00.501  01 26 23:59:59.730    2.09-2333.186 890.829   72   73
isfs3:/dev/ttyS5           3     50     30730 2011 01 26 16:00:00.602  01 26 23:59:59.258    1.07-2333.224 890.619   25   30
isfs3:/dev/ttyS8           3     80      6011 2011 01 26 16:00:03.430  01 26 23:59:55.246    0.21-2330.504 895.250    9    9
isfs3:/dev/ttyS1           3    100   1804820 2011 01 26 16:00:00.004  01 26 23:59:59.994   62.67-2334.070 889.825   14   14
isfs3:/dev/ttyS6           3  32768     18062 2011 01 26 16:00:03.144  01 26 23:59:58.299    0.63-2330.892 894.098    7  106
isfs4:/dev/ttyS2           4     20     30088 2011 01 26 16:00:00.189  01 26 23:59:59.753    1.04-2333.458 891.704   20   20
isfs4:/var/tmp/gps_pty0    4     30     60656 2011 01 26 16:00:00.497  01 26 23:59:59.752    2.11-2333.119 890.881   51   73
isfs4:/dev/ttyS5           4     50     30744 2011 01 26 16:00:00.379  01 26 23:59:59.972    1.07-2332.118 891.174   29   30
isfs4:/dev/ttyS8           4     80      5998 2011 01 26 16:00:04.099  01 26 23:59:59.640    0.21-2329.348 896.573    9    9
isfs4:/dev/ttyS1           4    100   1804616 2011 01 26 16:00:00.014  01 26 23:59:59.987   62.66-2333.565 890.692   14   14
isfs4:/dev/ttyS6           4  32768     18170 2011 01 26 16:00:00.547  01 26 23:59:59.591    0.63-2330.193 894.345   13   80
isfs5:/dev/ttyS2           5     20     30082 2011 01 26 16:00:00.317  01 26 23:59:59.481    1.04-2332.861 890.942   19   19
isfs5:/var/tmp/gps_pty0    5     30     60659 2011 01 26 16:00:00.486  01 26 23:59:59.700    2.11-2333.165 890.869   51   73
isfs5:/dev/ttyS5           5     50     30858 2011 01 26 16:00:00.008  01 26 23:59:59.130    1.07-2332.543 891.871   27   30
isfs5:/dev/ttyS1           5    100   1804680 2011 01 26 16:00:00.011  01 26 23:59:59.995   62.66-2333.643 890.550   14   14
isfs5:/dev/ttyS6           5  32768     24082 2011 01 26 16:00:00.057  01 26 23:59:59.848    0.84-2331.946 891.964   22   80
isfs6:/dev/ttyS2           6     20     30077 2011 01 26 16:00:00.794  01 26 23:59:59.165    1.04-2331.884 891.950   19   19
isfs6:/var/tmp/gps_pty0    6     30     60657 2011 01 26 16:00:00.512  01 26 23:59:59.755    2.11-2333.168 890.929   51   73
isfs6:/dev/ttyS4           6     40     30601 2011 01 26 16:00:00.442  01 26 23:59:59.043    1.06-2332.396 891.479   18   18
isfs6:/dev/ttyS5           6     50     30813 2011 01 26 16:00:00.794  01 26 23:59:59.555    1.07-2332.146 891.269   29   30
isfs6:/dev/ttyS8           6     80      6023 2011 01 26 16:00:01.303  01 26 23:59:58.299    0.21-2326.959 893.833    9    9
isfs6:/dev/ttyS1           6    100   1804581 2011 01 26 16:00:00.004  01 26 23:59:59.985   62.66-2333.460 890.460   14   14
isfs6:/dev/ttyS6           6  32768     24069 2011 01 26 16:00:00.322  01 26 23:59:59.868    0.84-2333.189 891.923   22   80
isfs7:/dev/ttyS2           7     20     30081 2011 01 26 16:00:00.874  01 26 23:59:59.581    1.04-2332.898 890.961   19   19
isfs7:/var/tmp/gps_pty0    7     30     60660 2011 01 26 16:00:00.513  01 26 23:59:59.724    2.11-2333.152 890.841   51   73
isfs7:/dev/ttyS5           7     50     30870 2011 01 26 16:00:00.224  01 26 23:59:59.083    1.07-2332.989 890.713   29   30
isfs7:/dev/ttyS1           7    100   1804759 2011 01 26 16:00:00.012  01 26 23:59:59.988   62.67-2333.731 890.255   14   14
isfs7:/dev/ttyS6           7  32768     18060 2011 01 26 16:00:01.147  01 26 23:59:57.571    0.63-2330.000 891.367    7  112
data_stats /scr/tmp/maclean/isfs_20110126_160000.dat.bz2 
2013-01-02,10:37:32|NOTICE|parsing: /net/isf/isff/projects/PCAPS/ISFF/config/pcaps.xml
Exception: EOFException: /scr/tmp/maclean/isfs_20110126_160000.dat.bz2: open: EOF
sensor                   dsm sampid    nsamps |------- start -------|  |------ end -----|    rate  minMaxDT(sec) minMaxLen
isfs1:/dev/ttyS2           1     20     28804 2011 01 26 16:00:00.138  01 26 23:59:59.726    1.00  0.724   1.277   19   19
isfs1:/var/tmp/gps_pty0    1     30     58080 2011 01 26 16:00:00.500  01 26 23:59:59.708    2.02  0.033   1.299   51   73
isfs1:/dev/ttyS4           1     40     29302 2011 01 26 16:00:00.559  01 26 23:59:59.606    1.02  0.431   1.529   18   18
isfs1:/dev/ttyS5           1     50     25927 2011 01 26 16:00:00.201  01 26 23:59:59.538    0.90  0.029 103.568   21   39
isfs1:/dev/ttyS1           1    100   1727994 2011 01 26 16:00:00.004  01 26 23:59:59.995   60.00  0.000   0.134   14   14
isfs1:/dev/ttyS6           1  32768     17287 2011 01 26 16:00:02.016  01 26 23:59:59.688    0.60  0.007   3.473   21   80
isfs2:/dev/ttyS2           2     20     28803 2011 01 26 16:00:00.403  01 26 23:59:59.191    1.00  0.020   2.335   20   20
isfs2:/var/tmp/gps_pty0    2     30     57600 2011 01 26 16:00:00.505  01 26 23:59:59.742    2.00  0.014   2.254   72   73
isfs2:/dev/ttyS5           2     50     29424 2011 01 26 16:00:00.105  01 26 23:59:59.569    1.02  0.029   2.104   29   31
isfs2:/dev/ttyS1           2    100   1728020 2011 01 26 16:00:00.006  01 26 23:59:59.989   60.00  0.000   0.277   14   14
isfs2:/dev/ttyS6           2  32768     17292 2011 01 26 16:00:01.207  01 26 23:59:58.369    0.60  0.004   4.133    6  116
isfs3:/dev/ttyS2           3     20     28802 2011 01 26 16:00:00.059  01 26 23:59:59.855    1.00  0.003   1.979   20   20
isfs3:/var/tmp/gps_pty0    3     30     57600 2011 01 26 16:00:00.501  01 26 23:59:59.730    2.00  0.029   1.627   72   73
isfs3:/dev/ttyS5           3     50     29422 2011 01 26 16:00:00.602  01 26 23:59:59.258    1.02  0.494   1.466   25   30
isfs3:/dev/ttyS8           3     80      5755 2011 01 26 16:00:03.430  01 26 23:59:55.246    0.20  4.041   5.959    9    9
isfs3:/dev/ttyS1           3    100   1728005 2011 01 26 16:00:00.004  01 26 23:59:59.994   60.00  0.000   0.164   14   14
isfs3:/dev/ttyS6           3  32768     17290 2011 01 26 16:00:03.144  01 26 23:59:58.299    0.60  0.001   4.864    7  106
isfs4:/dev/ttyS2           4     20     28809 2011 01 26 16:00:00.189  01 26 23:59:59.753    1.00  0.020   2.763   20   20
isfs4:/var/tmp/gps_pty0    4     30     58080 2011 01 26 16:00:00.497  01 26 23:59:59.752    2.02  0.067   2.303   51   73
isfs4:/dev/ttyS5           4     50     29438 2011 01 26 16:00:00.379  01 26 23:59:59.972    1.02  0.030   2.823   29   30
isfs4:/dev/ttyS8           4     80      5745 2011 01 26 16:00:04.099  01 26 23:59:59.640    0.20  4.989   5.033    9    9
isfs4:/dev/ttyS1           4    100   1727993 2011 01 26 16:00:00.014  01 26 23:59:59.987   60.00  0.000   0.269   14   14
isfs4:/dev/ttyS6           4  32768     17403 2011 01 26 16:00:00.547  01 26 23:59:59.591    0.60  0.003   5.149   13   80
isfs5:/dev/ttyS2           5     20     28802 2011 01 26 16:00:00.317  01 26 23:59:59.481    1.00  0.049   1.952   19   19
isfs5:/var/tmp/gps_pty0    5     30     58080 2011 01 26 16:00:00.486  01 26 23:59:59.700    2.02  0.132   1.531   51   73
isfs5:/dev/ttyS5           5     50     29547 2011 01 26 16:00:00.008  01 26 23:59:59.130    1.03  0.360   1.591   27   30
isfs5:/dev/ttyS1           5    100   1727995 2011 01 26 16:00:00.011  01 26 23:59:59.995   60.00  0.000   0.153   14   14
isfs5:/dev/ttyS6           5  32768     23061 2011 01 26 16:00:00.057  01 26 23:59:59.848    0.80  0.004   3.022   22   80
isfs6:/dev/ttyS2           6     20     28801 2011 01 26 16:00:00.794  01 26 23:59:59.165    1.00  0.019   2.175   19   19
isfs6:/var/tmp/gps_pty0    6     30     58080 2011 01 26 16:00:00.512  01 26 23:59:59.755    2.02  0.029   1.625   51   73
isfs6:/dev/ttyS4           6     40     29301 2011 01 26 16:00:00.442  01 26 23:59:59.043    1.02  0.018   2.367   18   18
isfs6:/dev/ttyS5           6     50     29505 2011 01 26 16:00:00.794  01 26 23:59:59.555    1.02  0.002   2.223   29   30
isfs6:/dev/ttyS8           6     80      5768 2011 01 26 16:00:01.303  01 26 23:59:58.299    0.20  4.381   5.605    9    9
isfs6:/dev/ttyS1           6    100   1727971 2011 01 26 16:00:00.004  01 26 23:59:59.985   60.00  0.000   0.272   14   14
isfs6:/dev/ttyS6           6  32768     23046 2011 01 26 16:00:00.322  01 26 23:59:59.868    0.80  0.001   3.838   22   80
isfs7:/dev/ttyS2           7     20     28801 2011 01 26 16:00:00.874  01 26 23:59:59.581    1.00  0.452   1.548   19   19
isfs7:/var/tmp/gps_pty0    7     30     58080 2011 01 26 16:00:00.513  01 26 23:59:59.724    2.02  0.023   1.666   51   73
isfs7:/dev/ttyS5           7     50     29555 2011 01 26 16:00:00.224  01 26 23:59:59.083    1.03  0.225   1.716   29   30
isfs7:/dev/ttyS1           7    100   1727986 2011 01 26 16:00:00.012  01 26 23:59:59.988   60.00  0.000   0.157   14   14
isfs7:/dev/ttyS6           7  32768     17292 2011 01 26 16:00:01.147  01 26 23:59:57.571    0.60  0.000   4.963    7  112
data_stats isfs_20110129_160000.dat.bz2 
2013-01-02,10:42:50|NOTICE|parsing: /net/isf/isff/projects/PCAPS/ISFF/config/pcaps.xml
Exception: EOFException: isfs_20110129_160000.dat.bz2: open: EOF
sensor                   dsm sampid    nsamps |------- start -------|  |------ end -----|    rate     minMaxDT(sec) minMaxLen
isfs1:/dev/ttyS2           1     20     29000 2011 01 29 16:00:00.073  01 29 23:59:59.629    1.01-1741.788 1547.804   19   19
isfs1:/var/tmp/gps_pty0    1     30     58475 2011 01 29 16:00:00.503  01 29 23:59:59.719    2.03-1743.099 1548.802   51   73
isfs1:/dev/ttyS4           1     40     29501 2011 01 29 16:00:00.473  01 29 23:59:59.487    1.02-1741.648 1548.014   18   18
isfs1:/dev/ttyS5           1     50     29685 2011 01 29 16:00:00.033  01 29 23:59:59.150    1.03-1742.440 1549.053   29   30
isfs1:/dev/ttyS1           1    100   1739733 2011 01 29 16:00:00.011  01 29 23:59:59.990   60.41-1743.284 1547.603   14   14
isfs1:/dev/ttyS6           1  32768     17405 2011 01 29 16:00:03.399  01 29 23:59:59.859    0.60-1741.478 1553.678    7  123
isfs2:/dev/ttyS2           2     20     28999 2011 01 29 16:00:00.451  01 29 23:59:59.251    1.01-1742.803 1548.830   20   20
isfs2:/var/tmp/gps_pty0    2     30     57992 2011 01 29 16:00:00.520  01 29 23:59:59.721    2.01-1743.136 1548.758   72   73
isfs2:/dev/ttyS5           2     50     29624 2011 01 29 16:00:00.833  01 29 23:59:59.512    1.03-1742.302 1548.496   29   30
isfs2:/dev/ttyS1           2    100   1739768 2011 01 29 16:00:00.010  01 29 23:59:59.993   60.41-1743.354 1547.614   14   14
isfs2:/dev/ttyS6           2  32768     17410 2011 01 29 16:00:01.412  01 29 23:59:59.279    0.60-1739.993 1549.991   21   79
isfs3:/dev/ttyS2           3     20     28997 2011 01 29 16:00:00.393  01 29 23:59:59.236    1.01-1742.872 1548.937   20   20
isfs3:/var/tmp/gps_pty0    3     30     57992 2011 01 29 16:00:00.525  01 29 23:59:59.715    2.01-1743.127 1548.869   72   73
isfs3:/dev/ttyS5           3     50     29624 2011 01 29 16:00:00.176  01 29 23:59:59.515    1.03-1742.966 1549.202   29   30
isfs3:/dev/ttyS8           3     80      5796 2011 01 29 16:00:03.156  01 29 23:59:56.543    0.20-1734.665 1549.710    9    9
isfs3:/dev/ttyS1           3    100   1739768 2011 01 29 16:00:00.003  01 29 23:59:59.995   60.41-1743.641 1547.626   14   14
isfs3:/dev/ttyS6           3  32768     17406 2011 01 29 16:00:00.915  01 29 23:59:58.465    0.60-1739.996 1549.990    7  106
isfs4:/dev/ttyS2           4     20     29003 2011 01 29 16:00:00.931  01 29 23:59:59.669    1.01-1742.759 1548.489   20   20
isfs4:/var/tmp/gps_pty0    4     30     58471 2011 01 29 16:00:00.512  01 29 23:59:59.739    2.03-1742.107 1548.797   51   73
isfs4:/dev/ttyS5           4     50     29639 2011 01 29 16:00:00.615  01 29 23:59:59.469    1.03-1742.215 1548.529   29   30
isfs4:/dev/ttyS8           4     80      5787 2011 01 29 16:00:00.624  01 29 23:59:55.437    0.20-1737.845 1552.578    9    9
isfs4:/dev/ttyS1           4    100   1739651 2011 01 29 16:00:00.014  01 29 23:59:59.990   60.40-1742.863 1547.598   14   14
isfs4:/dev/ttyS6           4  32768     17524 2011 01 29 16:00:03.872  01 29 23:59:59.529    0.61-1740.727 1554.277   13   82
isfs5:/dev/ttyS2           5     20     28997 2011 01 29 16:00:00.397  01 29 23:59:59.554    1.01-1741.895 1548.918   19   19
isfs5:/var/tmp/gps_pty0    5     30     58473 2011 01 29 16:00:00.508  01 29 23:59:59.704    2.03-1742.166 1548.915   51   73
isfs5:/dev/ttyS5           5     50     29533 2011 01 29 16:00:00.740  01 29 23:59:59.526    1.03-1742.235 1548.008   28   29
isfs5:/dev/ttyS1           5    100   1739719 2011 01 29 16:00:00.011  01 29 23:59:59.984   60.41-1742.993 1547.616   14   14
isfs5:/dev/ttyS6           5  32768     23212 2011 01 29 16:00:01.708  01 29 23:59:58.034    0.81-1739.995 1549.525   22   80
isfs6:/dev/ttyS2           6     20     28997 2011 01 29 16:00:00.089  01 29 23:59:59.448    1.01-1741.898 1548.919   19   19
isfs6:/var/tmp/gps_pty0    6     30     58473 2011 01 29 16:00:00.518  01 29 23:59:59.698    2.03-1742.108 1548.810   51   73
isfs6:/dev/ttyS4           6     40     29499 2011 01 29 16:00:00.497  01 29 23:59:59.106    1.02-1741.667 1549.023   18   18
isfs6:/dev/ttyS5           6     50     29704 2011 01 29 16:00:00.109  01 29 23:59:59.117    1.03-1741.878 1548.562   29   30
isfs6:/dev/ttyS8           6     80      5806 2011 01 29 16:00:04.617  01 29 23:59:55.740    0.20-1737.703 1552.949    9    9
isfs6:/dev/ttyS1           6    100   1739682 2011 01 29 16:00:00.011  01 29 23:59:59.982   60.41-1742.753 1547.620   14   14
isfs6:/dev/ttyS6           6  32768     23199 2011 01 29 16:00:00.698  01 29 23:59:58.787    0.81-1739.997 1549.902   22   80
isfs7:/dev/ttyS2           7     20     28997 2011 01 29 16:00:00.321  01 29 23:59:59.024    1.01-1741.924 1547.932   19   19
isfs7:/var/tmp/gps_pty0    7     30     58475 2011 01 29 16:00:00.522  01 29 23:59:59.713    2.03-1743.147 1548.820   51   73
isfs7:/dev/ttyS5           7     50     29757 2011 01 29 16:00:00.533  01 29 23:59:59.975    1.03-1742.586 1548.651   29   30
isfs7:/dev/ttyS1           7    100   1739746 2011 01 29 16:00:00.007  01 29 23:59:59.998   60.41-1743.534 1547.602   14   14
isfs7:/dev/ttyS6           7  32768     17405 2011 01 29 16:00:02.423  01 29 23:59:58.715    0.60-1740.449 1554.782   22   87
data_stats $s/isfs_20110129_160000.dat.bz2
2013-01-02,10:55:26|NOTICE|parsing: /net/isf/isff/projects/PCAPS/ISFF/config/pcaps.xml
Exception: EOFException: /scr/tmp/maclean/isfs_20110129_160000.dat.bz2: open: EOF
sensor                   dsm sampid    nsamps |------- start -------|  |------ end -----|    rate minMaxDT(sec) minMaxLen
isfs1:/dev/ttyS2           1     20     28804 2011 01 29 16:00:00.073  01 29 23:59:59.629    1.00  0.577  1.403   19   19
isfs1:/var/tmp/gps_pty0    1     30     58080 2011 01 29 16:00:00.503  01 29 23:59:59.719    2.02  0.106  1.111   51   73
isfs1:/dev/ttyS4           1     40     29302 2011 01 29 16:00:00.473  01 29 23:59:59.487    1.02  0.873  1.087   18   18
isfs1:/dev/ttyS5           1     50     29485 2011 01 29 16:00:00.033  01 29 23:59:59.150    1.02  0.603  1.348   29   30
isfs1:/dev/ttyS1           1    100   1727990 2011 01 29 16:00:00.011  01 29 23:59:59.990   60.00  0.000  0.145   14   14
isfs1:/dev/ttyS6           1  32768     17288 2011 01 29 16:00:03.399  01 29 23:59:59.859    0.60  0.000  4.197    7  123
isfs2:/dev/ttyS2           2     20     28803 2011 01 29 16:00:00.451  01 29 23:59:59.251    1.00  0.020  2.555   20   20
isfs2:/var/tmp/gps_pty0    2     30     57600 2011 01 29 16:00:00.520  01 29 23:59:59.721    2.00  0.132  2.021   72   73
isfs2:/dev/ttyS5           2     50     29424 2011 01 29 16:00:00.833  01 29 23:59:59.512    1.02  0.029  2.303   29   30
isfs2:/dev/ttyS1           2    100   1728021 2011 01 29 16:00:00.010  01 29 23:59:59.993   60.00  0.000  0.278   14   14
isfs2:/dev/ttyS6           2  32768     17293 2011 01 29 16:00:01.412  01 29 23:59:59.279    0.60  0.005  4.471   21   79
isfs3:/dev/ttyS2           3     20     28801 2011 01 29 16:00:00.393  01 29 23:59:59.236    1.00  0.568  1.413   20   20
isfs3:/var/tmp/gps_pty0    3     30     57600 2011 01 29 16:00:00.525  01 29 23:59:59.715    2.00  0.059  1.448   72   73
isfs3:/dev/ttyS5           3     50     29424 2011 01 29 16:00:00.176  01 29 23:59:59.515    1.02  0.116  1.843   29   30
isfs3:/dev/ttyS8           3     80      5757 2011 01 29 16:00:03.156  01 29 23:59:56.543    0.20  4.746  5.265    9    9
isfs3:/dev/ttyS1           3    100   1728005 2011 01 29 16:00:00.003  01 29 23:59:59.995   60.00  0.000  0.146   14   14
isfs3:/dev/ttyS6           3  32768     17289 2011 01 29 16:00:00.915  01 29 23:59:58.465    0.60  0.005  3.229    7  106
isfs4:/dev/ttyS2           4     20     28808 2011 01 29 16:00:00.931  01 29 23:59:59.669    1.00  0.020  3.039   20   20
isfs4:/var/tmp/gps_pty0    4     30     58080 2011 01 29 16:00:00.512  01 29 23:59:59.739    2.02  0.022  2.280   51   73
isfs4:/dev/ttyS5           4     50     29440 2011 01 29 16:00:00.615  01 29 23:59:59.469    1.02  0.029  2.211   29   30
isfs4:/dev/ttyS8           4     80      5748 2011 01 29 16:00:00.624  01 29 23:59:55.437    0.20  4.197  5.804    9    9
isfs4:/dev/ttyS1           4    100   1727993 2011 01 29 16:00:00.014  01 29 23:59:59.990   60.00  0.000  0.285   14   14
isfs4:/dev/ttyS6           4  32768     17406 2011 01 29 16:00:03.872  01 29 23:59:59.529    0.60  0.003  4.496   13   82
isfs5:/dev/ttyS2           5     20     28802 2011 01 29 16:00:00.397  01 29 23:59:59.554    1.00  0.617  1.383   19   19
isfs5:/var/tmp/gps_pty0    5     30     58080 2011 01 29 16:00:00.508  01 29 23:59:59.704    2.02  0.138  1.184   51   73
isfs5:/dev/ttyS5           5     50     29333 2011 01 29 16:00:00.740  01 29 23:59:59.526    1.02  0.146  1.824   28   29
isfs5:/dev/ttyS1           5    100   1727994 2011 01 29 16:00:00.011  01 29 23:59:59.984   60.00  0.001  0.155   14   14
isfs5:/dev/ttyS6           5  32768     23056 2011 01 29 16:00:01.708  01 29 23:59:58.034    0.80  0.001  4.276   22   80
isfs6:/dev/ttyS2           6     20     28802 2011 01 29 16:00:00.089  01 29 23:59:59.448    1.00  0.365  1.635   19   19
isfs6:/var/tmp/gps_pty0    6     30     58080 2011 01 29 16:00:00.518  01 29 23:59:59.698    2.02  0.129  1.574   51   73
isfs6:/dev/ttyS4           6     40     29301 2011 01 29 16:00:00.497  01 29 23:59:59.106    1.02  0.001  1.969   18   18
isfs6:/dev/ttyS5           6     50     29504 2011 01 29 16:00:00.109  01 29 23:59:59.117    1.02  0.154  1.796   29   30
isfs6:/dev/ttyS8           6     80      5767 2011 01 29 16:00:04.617  01 29 23:59:55.740    0.20  4.426  5.561    9    9
isfs6:/dev/ttyS1           6    100   1727972 2011 01 29 16:00:00.011  01 29 23:59:59.982   60.00  0.000  0.165   14   14
isfs6:/dev/ttyS6           6  32768     23043 2011 01 29 16:00:00.698  01 29 23:59:58.787    0.80  0.005  4.236   22   80
isfs7:/dev/ttyS2           7     20     28801 2011 01 29 16:00:00.321  01 29 23:59:59.024    1.00  0.006  2.144   19   19
isfs7:/var/tmp/gps_pty0    7     30     58080 2011 01 29 16:00:00.522  01 29 23:59:59.713    2.02  0.015  1.349   51   73
isfs7:/dev/ttyS5           7     50     29556 2011 01 29 16:00:00.533  01 29 23:59:59.975    1.03  0.029  2.034   29   30
isfs7:/dev/ttyS1           7    100   1727988 2011 01 29 16:00:00.007  01 29 23:59:59.998   60.00  0.000  0.195   14   14
isfs7:/dev/ttyS6           7  32768     17288 2011 01 29 16:00:02.423  01 29 23:59:58.715    0.60  0.000  5.747   22   87

We did not have the opportunity to calibrate the pyrgeometers prior to the start of PCAPS, and therefore used old thermopile calibration factors for the Kipp & Zonen CG4 and Eppley PIR pyrgeometers.  These were subsequently recalibrated in the NOAA black body on March 8 & 10, 2011.  The following table lists multiplicative correction factors for Rpile outputs during the PCAPS field program.

Sensor

Location

Position

Pre-ops

Post-ops

Correction

k*

 

 

 

uV-m^2/W

uV-m^2/W

pre/post

 

PIR 031980

Playa (1)

in

3.58

3.58

1.000

4.0

PIR 031976

Playa

out

3.93

3.74

1.051

3.5

PIR 031975

ABC (2)

in

3.97

3.98

0.997

3.5

PIR 031978

ABC

out

3.50

3.53

0.992

4.0

CG4 050824

Highland (3)

in

14.09

13.81

1.020

NA

CG4 040740

Highland

out

7.85

7.76

1.012

NA

CG4 030675

West Valley (4)

in

9.97

9.93

1.004

NA

CG4 040741

West Valley

out

8.82

8.76

1.007

NA

PIR 031979

East Slope (5)

in

3.99

3.81

1.047

4.5

PIR 031981

East Slope

out

3.69

3.66

1.008

3.5

CG4 100225

West Slope (6)

in

11.93

11.28

1.058

NA

CG4 100226

West Slope

out

11.51

10.81

1.065

NA

CG4 050823

Riverton (7)

in

9.32

9.21

1.012

NA

CG4 030676

Riverton

out

11.27

11.21

1.005

NA

*per Ellsworth Dutton, blackbody values for k have been increased by 0.3 and then rounded up to nearest 0.5

editing TP01 data

When the TP01 is power cycled, the values for the first 300? seconds (while it is taking its first reading) are not valid.  In the present raw_data, Vheat and Tau63 are set to missing values during this period, but not Lambdasoil (which reads as 0).  These spikes to zero are averaged into the 5min means of Lambdasoil, which introduce spikes into our Cvsoil values and eventually Gsfc.  Ordinarily, this <shouldn't> be much of a problem, but it appears that there were periods during PCAPS when these systems were power cycled quite often (e.g. every 5 hours on 21 Jan).  Presumably these were mote or power issues -- I haven't yet checked.

Regardless, it would be best to set all TP01 values to NA during these start-up periods.

TP01 analysis

UPDATE to the below discussion:

After discussions with Tom, I've realized that my entire philosophy of deriving soil moisture in frozen conditions from Cv is flawed.  In mixed conditions, Cv = Cv_liq * Qsoil_liq + Cv_ice * Qsoil_ice + Cv_dry, where Cv_liq = 4.81 and Cv_ice = 2.11.  Supposedly, from the plots below, I can compute Cv_dry by looking at the relation between Cv and Qsoil_liq when Qsoil_ice is zero (Tsoil well above freezing).  IF it was known that Qsoil_liq was zero (Tsoil well below freezing), this Cv relation could be used.  However, in mixed conditions, I don't know how much moisture is liquid or ice, so I can't independently compute soil water content (in any phase or combined).  Thus, I have 3 situations:

1. Soil not frozen: TP01 gives redundant Qsoil_liq to EC-5, once calibrated (using the EC-5).  Thus, not really independent.

2. Soil totally frozen: TP01 gives Qsoil_ice, once calibrated in totally warm conditions using the EC-5.  This is a unique measurement, but requires a long-term project.

3. Mixed soil conditions: TP01, combined with EC-5, gives Qsoil_ice, once calibrated in totally warm conditions using the EC-5.  As this procedure has several steps, there would be a large uncertainty in the result!

-------------------------------------

I've been trying to figure out why the Qsoil/Csoil comparison has looked lousy at most sites.  What I've found:

1. I now have better values for the soil densities than my first cut at looking at the data (which only used the first set of gravimetric measurements).  I'm now using the mean of all measured values at 3-6cm.  (There sometimes is a lot of variation between these measurements.)

2. The soil motes apparently had a "watchdog timeout" that reset power on the soil sensors rather frequently.  I believe that this occurred at all sites sometime, and, when bad, happens every ~4hours:50minutes.  Each time, the TP01 reported values of 0 until it was able to take another reading.  These 0 values were averaged in to each TP01 variable, generating lots of spikes.  A range check on the TP01 Vheat and Tau63 values has now been added to eliminate these zeros.  The covars are being rerun now.

(Note that gaps of one sample (5 s) every 5-minutes are normal, as John's code uses this time to sample the mote's power voltage.)

3. Even with all this, the comparison is good at only about 1/2 of the sites.  ("Good" means that the change of Cvsoil with Qsoil follows what I would expect.  "Bad" means that Cvsoil changes more than I would expect.)  I can trace the bad values to Vpile.on being too low at many of the sites.  (asoil values seem reasonable at all sites and Vpile.off and Vheat -- the only other variables in the Lambdasoil calculation -- are okay.)  This is true at most of these sites even during the last half of January, when there was a warm period that might have thawed the soil.  Possible reasons for Vpile.on being low are:

a. Inadequate contact between the probe and soil (the most likely culprite, despite my best efforts)

b. Voltage drop on Vheat (unlikely, since the 4-wire probe should measure this)

c. Electrical contact problem on Vpile (unlikely, but should be checked)

d. Heat going into latent heat of fusion (melting mixed-phase moisture), rather than sensible heat (can partly check by looking at warm periods -- a new idea for me).  [The results of this check are not very convincing -- see below plot.]

4. Horizontal lines on the plot below are caused by rapid changes of measured qsoil (which is really liquid qsoil) that, in the one case I examined in detail, was caused by rapid warming (and thus melting) of the soil.  The portion of this change that occurs in the 3 hours that the TP01 is reporting replicate values appears as a horizontal line.  I'll update this plot with data values averaged to 3hours to eliminate this artifact, but there still will be the question of whether the qsoil value is valid.

By the way, the warming case I found is 23 Jan, near noon, at ABC.

This is closer than I've been before with this comparison, but I still have a long way to go.

Attached is my current comparison plot for each of the 9 soil sites after filtering the statistics for:

- good Tau63 and Vheat values

- rerun of covars over first half of the data

- removing qsoil values when tsoil<0C

Points in blue are for the first month of the project, when probes may have been settling (actually, mostly not too bad).  Arguably, these values may be ignored.  Points in green are for tsoil>5C, when soil should not have ice.  The black line is the expected variation of Cvsoil with Qsoil (4.184 J/kgK for 100% soil moisture by mass) with no dry soil heat capacity.  Actual values should fall along the green line, which is parallel to the black line and has been fitted by assuming dry soil heat capacities of  1.3,1.0,0.9,0.2,0.6,0.2,0.6,1.8,0.2 J/kg K, respectively, for each of the sites.

In the "even newer" version, I've manually eliminated periods where I think there might be a chance of frozen soil moisture (that would give me a false qsoil measurement).  This eliminates about 3/4 of the data, but greatly reduces the scatter.

Even newer plot:

Updated plot:

Old plot:

Site coordinates

Here are the site coordinates as recorded by Gordon in the netcdf files:

"We looked at the GPS output from each station, when there was satellite lock
and the lat and long were constant, and put them as non-time series variables
in the netcdf."

Site

Latitude

Longitude

Playa(1)

40.7786

-112.1031

ABC(2)

40.7377

-111.9132

Highland(3)

40.6969

-111.839

W Valley(4)

40.681

-111.9949

E Slope(5)

40.6135

-111.8039

W Slope(6)

40.5938

-112.0621

Riverton(7)

40.5136

-111.9737

More TP01 testing

From the Sept/Oct 2010 tests, I found that new, sensor-specific, calibrations of E.lambda and Tau63_ref from the agar test produced results with less scatter.  Thus, the plan is to use these for data processing.  Since I will need to rely on these values, it seems like a good idea to do agar post-cals on these probes.  In addition, I might as well double-check the probe resistances, so Chris is now making a test cable that I'll use with the HP5006 plug-in 4-wire meter.  With these revised calibrations, I'll go back and look at the PCAPS data when the soil wasn't frozen.  Finally, we'll rerun some tests in the FLAB yard.  I'm not planning a new set of keg tests.

Resistance values measured 15 Apr 2011 using 4-wire meter:

ID

Sensor S/N

R_measured (Ohms)

R_manufacturer (Ohms)

Difference (Ohms)

1

200589

16.14

16.1

0.04

2

200590

15.40

15.4

0.00

3

200236

14.79

14.8

-0.01

4

200591

16.70

16.7

0.00

5

200239

16.37

16.2

0.17

6

200592

15.53

15.5

0.03

7

200241

16.27

16.2

0.07

8

200242

16.10

15.9

0.20

9

200243

15.76

15.6

0.16

10

200234

14.72

14.7

0.02

11

200238

15.50

15.4

0.10

 

200235

sliced

14.8

 

 

200240

lightning fried

15.0

 

All are within 0.2 ohms of manufacturer values.  Except for one at 0.01 ohms, our values are higher.  <Might> be connectors, but 4-wire measurement should compensate.

Agar results (first row Sep 2010, second row Apr 2011).  Lambda values use original Hukseflux coefficients in this table:

ID

Sensor

Vheat (uV)

Vpile.on (uV)

Vpile.off (uV)

V63 (uV)

Tau63 (0.01s)

Lambda (mW/m K)

 

Expected

1200000

 

0

 

1900

600

1

200589

1236927

375

0

139

1943

666

 

 

1236416

375

0

139

1921

665

2

200590

1194264

314

0

116

1940

668

 

 

1194652

314

0

116

1985

669

3

200236

1238958

313

0

116

1877

578

 

 

1237230

313

0

116

1892

576

4

200391

1240328

313

0

116

2073

686

 

 

1239126

313

0

115

2042

684

5

200239

1239719

470

0

174

1965

591

 

 

1238975

470

0

173

1972

590

6

200592

1237441

407

0

150

2125

655

 

 

1237288

407

0

150

2170

655

7

200241

1241515

376

0

139

1892

596

 

 

1239874

376

0

139

1867

595

8

200242

1236806

407

0

150

2089

600

 

 

1236332

407

0

150

2105

600

9

200243

1236372

438

0

162

1746

598

 

 

1236324

438

0

162

1755

598

10

200234

1237511

281

0

104

1679

603

 

 

1237304

281

0

104

1673

603

11

200238

1238329

439

0

162

1773

586

 

 

1236740

439

0

162

1792

584

All of these have very close agreement between the Sept 2010 and Apr 2011 values.  Tau63 differences are 0-0.5s, Lambda differences are 0-2.  Thus, best approach is to take the average of these values to generate new Tau63_ref and E.lambda values.

Agar recipe

Thought I'd start by scribbling down the Agar recipe:

Add 7g agar flakes to 1.75l water in 2l beaker.

Place on 1000W burner of 2-burner hotplate.

Rig up a shroud and lid around beaker to reduce heat loss.  I borrowed a chunk of pipe and PVC from the shop this time (see photo).  The spot I used in the cage had a draft, so I used the blue foam to block it.

It took about 20min on high to get to a boil.  I stir every couple of minutes using a glass rod Chris found to the left of the cal lab sink.

Reduce heat to medium-high and simmer with lid off (packet instructions said for 5min, TP01 said 20min, I did 15min this time).  I noted that the volume reduced to 1.6l after all this time, which is still fine.

Let cool until milky white (several hours).

Tsoil check

March 14, 2011

TESTED GOOD:  TS016, TS017, TS015, TS013(cleaned connector), TS005, TS011.

BAD:  TS002, TS004, TS003.

TS012 tested good but had #4 sensor out.

Post Krypton cal

The kryptons were checked in the chamber at 10C. The density range went from 1 to 9 g/m^3.

sensor

error range (g/m^3)

1525

   1.52 to 3.00

1389

   1.01 to 1.56

1390

   0.65 to 0.81

1393

   -0.21 to 0.02

1394

  2.49 to 3.43

1133

   1.88 to 2.47

1258

   1.95 to 2.32

At the start of the project we did not have the new calibration coefficients for the KZ CM21 and EPPLEY PSP shortwave radiation sensors.

Therefore a correction factor must be applied to the data.

sensor

location

pistion

cal factor

EPP 16945

Playa

in

1.040632

EPP 17877

Playa

out

1.033604

EPP 17414

ABC

in

1.009423

EPP 15108

ABC

out

1.024915

KZ 940185

Highland

in

0.993432

KZ 970378

Highland

out

0.987761

KZ 970380

West Valley

in

1.002629

KZ 940181

West Valley

out

0.993238

EPP 16107

East Slope

in

1.023545

EPP 16714

East Slope

out

1.033453

EPP 26224*

East Slope

out

1.013483

KZ 970379

West Slope

in

0.988474

KZ 940187

West Slope

out

0.993795

KZ 970377

Riverton

in

0.992930

KZ 940186

Riverton

out

0.999155

* removed Rsw.out at East Slope ~18:30 Dec 4; replaced 14:20 Dec 6

Post cal TRH

Post cal results of temperature, temperature range 15C to -20C

Sensor

mean (error)

sd (error)   

min to max

017

.019

.014

-.036 to .001

023

.017

.012

-.004 to .030

019

.016

.014

-.001 to .032

013

.000

.006

-.007 to .010

025

-.008

.011

-.034 to .002

024

.002

.008

-.011 to .013

009

-.035

.007

-.044 to -.022

028

-.018

.007

-.028 to -.004

027

-.011

.016

-.027 to .02

Post cal of RH, two runs from 10% to 90% at 10C and -4C, Gortex still on

Sensor

mean (error)

sd (Error)

min to max

017

-3.3

0.9

-4.8 to -1.4

023

-1.7

1.2

-3.8 to -0.4

019

-8.6

1.2

-11.1 to -6.3

013

0.1

0.5

-0.7 to 0.8

025

0.7

1.44

-2.2 to 2.2

024

1.1

1.7

-1.1 to 3.8

009

-0.6

0.6

-1.1 to 0.9

028

0.6

1.9

-3.6 to 2.6

027

1.2

1.6

-2.1 to 2.6

Location of SHT sensors through out the project. Some sensors replaced during the project were sent back to Boulder for repairs then the sensor was deployed again

                                              DATE of CHANGE

STATION

11/28

12/07

12/12

12/13

1/15

11/16

1/17

1/18

1/27

1/28

2/6

2/7

1

13

 

9

26

 

 

21

 

13

 

 

13

2

15

21

 

 

 

 

24

 

 

 

 

24

3

20

 

 

 

 

 

19

23

 

 

 

23

4

19

 

 

 

 

25

 

 

 

 

 

25

5

18

 

 

 

 

 

 

 

 

9

17

17

6

23

24

 

 

 

 

27

 

 

 

 

27

7

17

 

 

 

28

 

 

 

 

 

 

28

A post cal of the barometers was completed today. Each barometer was run over the range of 800 to 900 mb.

The average error was .16 mb, error span was -.21 to .26 mb, sd was .06

Post Serial Numbers

Sensor

1
PLAYA

2
ABC

3
HIGHLAND

4
WEST VALLEY

5
EAST
SLOPE

6
WEST
SLOPE

7
RIVERTON

CSAT3

0537

0800

0855

853

0671

1119

0539

Krypton

1395 / 1390 (1/2/11)

1393

1525

1258

1133

1389

1394

Barometer

B8
U4110003

B7
U4110001

B3
NCAR003

B10
491125

B4
S0610001

B5
S0610002

B9
U411004

TRH

 

 

 

 

 

 

 

Propvane

00318

----------------

----------------

---------------------

--------------------

???

-------------------

Rad. IN (pyn/pyg)

16945 /
31980

17414 /
31975

940185 /
050824

970380 /
030675

16107 /
31979

970379 /
100225

970377 /
050823

Rad. OUT (pyn/pyg)

17877 /
31976

15108 /
31978

970378 /
040740

940181 /
040741

26224 /
31981

940187 /
100226

940186 /
030676

LiCor

PY7121(DIR)
PY99291(DIFF)

----------------

----------------

---------------------

-------------------

---------------

PY10835(DIR)
PY10836(DIFF)

Tsoil

TS017

TS016

SING(026)

TS002/TS015

TS023
TS004
TS012

TS013
SING(022)
TS003
TS011

TS005

Qsoil

ECHO007(017)

ECHO010(020)

ECHO005(015)

ECHO009(019)

ECHO004(014)
ECHO006(016)

  MISSING
ECHO001(011)

ECHO002(012)

Gsoil

H993563(007)

H013390(009)

H983211(005)

H023009(002)

H013391(004)
87034(010)

H993561(003)
H993560(006)

H983213(008)

TPO1

TP025(200590)

TP006(200592)

TP005(200239)

TP007(200241)

TP009(200243)
TP008(200242)

TP010(200234)
TP004(200391) [200591?  spo 4/18/11]

TP001(200589)

Rain Gauge

---------------------

---------------------

???????????

????????????

-------------------

????????????

--------------------

Motes

B,7,19

F,4,5

D,22,6

15,15,8

13,12,11,20

11,10,9,21

17,16,18

Vbatt

31

32

38

34

35

36

37

Viper
Sunshine
Modem/
Router

16


M-8(09105212383)/
R-1(MM090194300014)

1


M-2(09105211885)/
R-2(MM090168701586)

14


M-3(09105211890)/
R-3(MM090168700567)

12


M-4(09105211882)/
R-4(MM090139201501)



M-5(09105212400)/
R-5(MM091468703640)

6


M-6(09105211884)/
R-6(MM090139202886)

3
SPN1-A465


M-7(09105212393)/
R-7(MM090139202713)


I've just plotted Cvsoil from the entire project to see if I could determine a time when the TP01 (as a surrogate for all soil sensors) had "healed" into the soil.  I see a definite step at about 7 Dec at 1, 5, 6, and to some extent 6.aux and 7.  This was a few days into the first PCAPS warming period with temperatures above freezing.  Given what comes later (other steps in Jan), it isn't obvious that the TP01 had completely settled in (perhaps ever).  Thus, the most definite conclusion I can make is that at least half of the sensors took at least 5 weeks to settle in.

Rain gauges, Sites 3 and 4

February 7

~13:30 removed rain gauge, W Valley

14:30 removed rain gauge, Hiland

Precip started 15:00

Status, February 7, 2011

Status evaluated from 36 hour plots on Feb 7 8:40-9:20 MST
On site: Semmer and Horst, Chris and Kurt

Weather: Overcast, light winds

Summary: some soil issues at W slope
        
Todo: nada

Vdsm: ok, > 12.2V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.2 V  E Slope main lowest

P: ok
Tbaro: ok
T: ok
RH: ok
Q: ok
Rainr (3,4,6): ok, no tips
Spd (1,6): ok
Dir (1,6): ok

csat diag: ok, activity on 2,4,5,6,7 around 14:00
spd: ok, light winds < 4 m/s
dir: ok, NW at W Slope since 6 am
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok, 0.15
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: noise at playa 1.9 cm
Qsoil:  w slope main bad
Gsoil: W slope looks too small
Cvsoil: ok

Feb 6 Eslope5, TRH

10:30 - 10:35

Overcast

replaced TRH009 with TRH017. New TRH does not have Gortex cover

Status, Feb 6, 2011

Status evaluated from 36 hour plots on Feb 6 9:00-9:30 MST
On site: Semmer and Horst

Weather: Overcast, light winds

Summary: some soil issues
        
Todo: replace TRh at E slope

Vdsm: ok, > 12.2V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.2 V  E Slope main lowest

P: ok
Tbaro: ok
T: E Slope has gone bad again, ~3:00 am
RH: ditto, warm, dry air at W slope at ~6:30am
Q: ditto, 3.5 g/kg
Rainr (3,4,6): ok, no tips
Spd (1,6): ok
Dir (1,6): ok

csat diag: ok, activity on 2,4,5,6,7 around 14:00
spd: ok, light winds < 4 m/s
dir: ok, NW at W Slope since 6 am
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, 1,3,4 at 0.5-0.8 V
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok, 0.15
Rsw.dfs (1,7): ok, SPN1 > Licor
Rsw.global (1,7): ok, ditto

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: noise at playa 1.9 cm
Qsoil:  w slope main bad
Gsoil: ok, ABC and Wslope show strong negative, -45 w/m2, during daytime
Cvsoil: ok

Sonic configuration files

Station 1

nSN0537 26apr10 rev 4.0t &=1 AC=1 AF=050 AH=1 AO=00300 ar=0 AQ= 60 BR=0 BX=0\r\n CF=1 C0o= 0 0 0 C0b= 0 0 0 CA=1 CD=0 cs=37195 DC=4 dl=015 DM=c DR=03465\r\n duty=087 DT=16240 ET= 60 FA=00050 FL=007 FX=038 GN=555a GO=00000 HA=0\r\n HG=01560 HH=02700 KT=0 LG=00832 LH=00100 MA=-020 MS=-010 MX=0 ND=1 NI=2\r\n ns=00203 OR=1 os=0 PD=2 RA=00020 RC=0 RF=00900 RH=015 RI=1 RS=1 RX=002 SD=0\r\n SL=035 SR=1 ss=1 T0123=1000 TD=a TF=02600 02600 02600 TK=1 TO= 0 0 0 TP=t\r\n ts=i X=0 WM=o WR=006 XD=d xp=2 XX=00875 ZZ=0\

Station 2

SN0800 11oct10 rev 4.0t &=0 AC=1 AF=050 AH=1 AO=00300 ar=0 AQ= 60 BR=0 BX=0\r\n CF=1 C0o= 0 0 0 C0b= 0 0 0 CA=1 CD=0 cs=07392 DC=4 dl=015 DM=c DR=03465\r\n duty=086 DT=16240 ET= 60 FA=00050 FL=007 FX=038 GN=545a GO=00000 HA=0\r\n HG=01560 HH=02700 KT=0 LG=00832 LH=00100 MA=-020 MS=-010 MX=0 ND=1 NI=2\r\n ns=00203 OR=1 os=0 PD=2 RA=00020 RC=0 RF=00900 RH=015 RI=1 RS=1 RX=002 SD=0\r\n SL=035 SR=1 ss=1 T0123=1000 TD=a TF=02600 02\r600 02600 TK=1 TO= 0 0 0 TP=t\r\n ts=i X=0 WM=o WR=006 XD=d xp=2 XX=00875 ZZ=0

Station 3

SN0855 01oct10 rev 4.0t &=0 AC=1 AF=050 AH=1 AO=00300 ar=0 AQ= 60 BR=0 BX=0\r\n CF=1 C0o= 0 0 0 C0b= 0 0 0 CA=1 CD=0 cs=07585 DC=4 dl=015 DM=c DR=03465\r\n duty=087 DT=16240 ET= 60 FA=00050 FL=007 FX=038 GN=465a GO=00000 HA=0\r\n HG=01560 HH=02700 KT=0 LG=00832 LH=00100 MA=-020 MS=-010 MX=0 ND=1 NI=2\r\n ns=00203 OR=1 os=0 PD=2 RA=00020 RC=0 RF=00900 RH=015 RI=1 RS=1 RX=002 SD=0\r\n SL=035 SR=1 ss=1 T0123=1000 TD=a TF=02600 02600 02600 TK=1 TO= 0 0 0 TP=t\r\n ts=i X=0 WM=o WR=006 XD=d xp=2 XX=00875 ZZ=0\

Station 4

SN0853 30sep10 rev 4.0t &=0 AC=1 AF=050 AH=1 AO=00300 ar=0 AQ= 60 BR=0 BX=0\r\n CF=1 C0o= 0 0 0 C0b= 0 0 0 CA=1 CD=0 cs=55688 DC=4 dl=015 DM=c DR=03465\r\n duty=087 DT=16240 ET= 60 FA=00050 FL=007 FX=038 GN=556a GO=00000 HA=0\r\n HG=01560 HH=02700 KT=0 LG=00832 LH=00100 MA=-020 MS=-010 MX=0 ND=1 NI=2\r\n ns=00203 OR=1 os=0 PD=2 RA=00020 RC=0 RF=00900 RH=015 RI=1 RS=1 RX=002 SD=0\r\n SL=035 SR=1 ss=1 T0123=1000 TD=a TF=02600 02600 02600 TK=1 TO= 0 0 0 TP=t\r\n ts=i X=0 WM=o WR=006 XD=d xp=2 XX=00875 ZZ=0\

Station 5

SN0671 29sep10 rev 4.0t &=0 AC=1 AF=050 AH=1 AO=00300 ar=0 AQ= 60 BR=0 BX=0\r\n CF=1 C0o= 0 0 0 C0b= 0 0 0 CA=1 CD=0 cs=06650 DC=4 dl=015 DM=c DR=03465\r\n duty=087 DT=16240 ET= 60 FA=00050 FL=007 FX=038 GN=565a GO=00000 HA=0\r\n HG=01560 HH=02700 KT=0 LG=00832 LH=00100 MA=-020 MS=-010 MX=0 ND=1 NI=2\r\n ns=00203 OR=1 os=0 PD=2 RA=00020 RC=0 RF=00900 RH=015 RI=1 RS=1 RX=002 SD=0\r\n SL=035 SR=1 ss=1 T0123=1000 TD=a TF=02600 02600 02600 TK=1 TO= 0 0 0 TP=t\r\n ts=i X=0 WM=o WR=006 XD=d xp=2 XX=00875 ZZ=0

Station 6

SN1119 05oct10 rev 4.0t &=0 AC=1 AF=050 AH=1 AO=00300 ar=0 AQ= 60 BR=0 BX=0\r\n CF=1 C0o= 0 0 0 C0b= 0 0 0 CA=1 CD=0 cs=27140 DC=4 dl=015 DM=c DR=03465\r\n duty=087 DT=16240 ET= 60 FA=00050 FL=007 FX=038 GN=666a GO=00000 HA=0\r\n HG=01560 HH=02700 KT=0 LG=00832 LH=00100 MA=-020 MS=-010 MX=0 ND=1 NI=2\r\n ns=00203 OR=1 os=0 PD=2 RA=00020 RC=0 RF=00900 RH=015 RI=1 RS=1 RX=002 SD=0\r\n SL=035 SR=1 ss=1 T0123=1000 TD=a TF=02600 02600 02600 TK=1 TO= 0 0 0 TP=t\r\n ts=i X=0 WM=o WR=006 XD=d xp=2 XX=00875 ZZ=0

Station 7

SN0539 30apr10 rev 4.0t &=0 AC=1 AF=050 AH=1 AO=00300 ar=0 AQ= 60 BR=0 BX=0\r\n CF=1 C0o= 0 0 0 C0b= 0 0 0 CA=1 CD=0 cs=12284 DC=4 dl=015 DM=c DR=03465\r\n duty=088 DT=16240 ET= 60 FA=00050 FL=007 FX=038 GN=545a GO=00000 HA=0\r\n HG=01560 HH=02700 KT=0 LG=00832 LH=00100 MA=-020 MS=-010 MX=0 ND=1 NI=2\r\n ns=00203 OR=1 os=0 PD=2 RA=00020 RC=0 RF=00900 RH=015 RI=1 RS=1 RX=002 SD=0\r\n SL=035 SR=1 ss=1 T0123=1000 TD=a TF=02600 02600 02600 TK=1 TO= 0 0 0 TP=t\r\n ts=i X=0 WM=o WR=006 XD=d xp=2 XX=00875 ZZ=0

Feb 5 Eslope5, TRH

10:00 - 10:30

   Overcast

SHT went back yesterday. Pulled sensor and found corrosion on the sensor pins. used the special contact goop and re-installeed. Working ok.

10:21 - radiometer domes cleaned

Status, February 5, 2011

Status evaluated from 36 hour plots on Feb 5 9:00-9:30 MST
On site: Semmer and Horst

Weather: Overcast, light winds

Summary: E Slope TRH bad; some soil issues
        
Todo: Replace E Slope TRH

Vdsm: ok, > 12.2V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.3 V  E Slope main lowest

P: ok
Tbaro: ok
T: E Slope bad since noon yesterday
RH: ditto, warm, dry air at W Slope since 6 am
Q: ditto, 3.8 g/kg
Rainr (3,4,6): ok, no tips
Spd (1,6): ok
Dir (1,6): ok

csat diag: ok
spd: ok, highest at W Slope since 6 am
dir: ok, NW at W Slope since 6 am
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok, negative at W Slope since 6 am

kh2oV: ok, 1,3,4 at 0.5-0.6 V
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok, positive at W Slope since 6 am

Rsw.in: ok
Rsw.out: ok
albedo: ok, ~0.15
Rsw.dfs (1,7): ok, SPN1 > Licor
Rsw.global (1,7): ok, ditto

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: noise at playa 1.9 cm; E Slope 4.4 cm recently recovered
Qsoil: playa drying to 35 vol%; w slope main bad
Gsoil: ok
Cvsoil: ok

10:30 cleaned radiometers

Steve tried, unsuccessfully, to reprogram the router

Update:  14:15-14:40

After testing the cables in the base came back to 2 to reprogram router.This time it was successful

It did a long time after the router reboot to be recognized as isfs2.

Status, February 4, 2011

Status evaluated from 36 hour plots on Feb 4 8:30-9:30 MST
On site: Semmer and Horst

Weather: Overcast, light winds

Summary: Successfully reconfigured the shadowbands on Wednesday.
         Some issues with Tsoil and Qsoil
         
Todo: Prepare for teardown

Vdsm: ok, > 12.2V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.2 V  E Slope main lowest

P: ok
Tbaro: ok
T: ok, low = -15 degC at W Slope`
RH: ok
Q: ok, 2 g/kg
Rainr (3,4,6): ok,tips at highland, seem erroneous
Spd (1,6): ok
Dir (1,6): ok

csat diag: ok
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, all > 1 V
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: noise at playa 1.9 cm; bad data at E Slope 4.4 cm
Qsoil: playa finally drying to 30 vol%, w slope main bad
Gsoil: ok
Cvsoil: playa increased to 7e8 yesterday; W slope aux to 2.5e7

14:30 - 14:50

clear and cold

Re-adjustment of shadowband

Radoimeters cleaned at 14:45

12:20 - 13:00

clear, cold, and light winds

After a conversation with Sebast1an, we found out the proper way to setup the shadowband.

(1) set the angle of the ring support arm to the latitude of the site not the sun's elevation.

(2) adjust the position of the shadow ring such that a shadow is cast on the center of the Licor (the ring is moved up or down on the support arm.

Tom completed the re-alignment at 12:47. This included aligning the unit to south

12:35 .Radiometer domes were cleaned

15:30 - 15:40

revisit because diffuse reading low. Connector was loose, reset and all is well.

Sensor was unshaded for a short time

Status, February 2, 2011

Status evaluated from 36 hour plots on Feb 2 08:30-09:00 MST
On site: Semmer and Horst

Weather: Clear and cold with light winds

Summary: Questionable precip tips
         3 hrs missing rad data at Riverton
         All Tsoil frozen
         4.4 cm Tsoil bad at E Slope
         Qsoil bad at W Slope main
         
Todo: work on shadowband at riverton and playa

Vdsm: ok, > 12.2V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.2 V  E Slope main lowest

P: ok
Tbaro: ok
T: ok, low = -18 degC at W Slope`
RH: ok
Q: ok, mostly < 1 gm/kg
Rainr (3,4,6): ok, tips last night at all 3 sites; seems erroneous
Spd (1,6): ok, down to 3 m/s at w Slope, less at Playa
Dir (1,6): ok, currently N

csat diag: ok
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, 1-3 V
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok, 3 hrs missing at Riverton
Rpile.in: ok, ditto
Tcase.in: ok, ditto
Tdome.in (1,2,5): ok

Rlw.out: ok, 3 hrs missing at Riverton
Rpile.out: ok, ditto
Tcase.out: ok, ditto
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: 4.4 cm at Eslope has gone walkabout
Qsoil: ok, W slope main has gone walkabout
Gsoil: ok
Cvsoil: ok

14:30 - 15:00

Clear and cold with winds

This was an afternoon adjustment of the shadowband. It appears the latest change made with the mounting of the Licor

sensor did not help. Tom found he had to make a major elevation change to shadow the sensor. He left the setting at 32 degrees.

Kurt is sending out one of the other shadowband units so we can do some comparisons

Status, Feb 1, 2011

Status evaluated from 36 hour plots on Feb 1 15:15-15:50 MST
On site: Semmer and Horst

Weather: Clear and cold with high winds

Summary: 3 hrs missing soil data at E Slope aux: 1/31 22:00 -> 2/1 01:00
         
Todo: work on shadowband at riverton and playa

Vdsm: ok, > 12.2V
Vmote.rad: ok, > 12.6 V
Vmote.soil: ok, > 12.2 V  E Slope main lowest

P: ok
Tbaro: ok
T: ok, low = -12 degC at W Slope`
RH: ok
Q: ok, 0.75 g/kg
Rainr (3,4,6): ok,tips this morning a Wslope and riverton
Spd (1,6): ok
Dir (1,6): ok, currently NE

csat diag: ok
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, all > 1 V this afternoon
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok, spread in readings at each pyranometer this afternoon

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok, (smaller) spread in readings at each pyranometer this afternoon

soil.aux at E & W Slope (5,6)
Tsoil: ok, but spikes at 1.9 cm at Playa and 3 hrs missing data at E Slope aux
Qsoil: ok, up to 38 vol% at Playa; 3 hrs missing data at E Slope aux
Gsoil: ok, 3 hr missing data at E Slope aux
Cvsoil: ok, missing data at E Slope aux

12:30 - 13:15

clear and cold with high winds

Re-adjusted the shadowband so it is like the unit at station 7. Elevation angle before any changes was 31 degrees. Elevation angle set to 34 degrees based on shadow cast on sensor.

Program said angle should have been 32.2 degrees.

8:30 - 9:00

Clear and very cold with high winds

Re-installed shadowband with modifications. We still suffer from sunshine when the unit is set to the correct elevation angle.

Tom lowered the licor in the mounting block such that the top of the sensor is in line with the pivot axis of the block.

Now we are getting shading at the correct angle.

14:00 - 14:30

partly cloudy and cold

Tom wanted to see if the shadowband was doing its job at sunset. To make a long story short, Tom determined that the location of the sensor was not in the center of the circular ring made by the shadowband. So we removed the unit and I will make mods to it for redeployment tomorrow.

Riverton shadow band

We went to Riverton on 9/31 to check on the shadow band, arriving about 1545 and leaving about 1615.

We discovered that the licor window is not in the center of the shadow band because of the way

it is mounted.  We removed the shadowband arm to fix this back at the Residence Inn.

Brushed snow off radiometers at sites 1 and 3 between 10:30 and 11:30, Jan 31.

Status, Jan 31, 2011

Status evaluated from 36 hour plots on Jan 31 10:00-10:30 MST
On site: Semmer and Horst, Cohn visiting

Weather: cloudy this morning, light winds, light snow last night

Summary: Took soil sample at Playa yesterday
         
Todo:

Vdsm: ok, > 12.1 V
Vmote.rad: ok, > 12.4 V
Vmote.soil: ok, > 12.2 V  E Slope main lowest

P: ok
Tbaro: ok
T: ok
RH: ok
Q: 3.6-4.1 g/kg
Rainr (3,4,6): ok, precip at all sites after 9 pm 1/30
Spd (1,6): ok
Dir (1,6): ok, currently NW

csat diag: ok, spiking last night at 1:3,7 presumable due to snow
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, all < 1 V this morning
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok
Qsoil: ok, up to 38 vol% at Playa
Gsoil: ok
Cvsoil: ok, W Valley continues to have spikes at ~ 5 hr intervals

10:40 - 10:55

fog  conditions; the soil sensors are under standing water.

soil sample taken. even though standing water the soil was not that soggy.

Radiometer domes cleaned

Status, Jan 30, 2011

Status evaluated from 36 hour plots on Jan 30 08:45-9:15 MST
On site: Semmer and Horst

Weather: fog this morning, light winds

Summary: shadow band at Riverton produced rabbit ear yesterday evening
         
Todo: Soil sample at 1

Vdsm: ok, > 12.3 V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.2 V  E Slope lowest

P: ok
Tbaro: ok
T: ok
RH: ok
Q: 3.5-4 g/kg
Rainr (3,4,6): ok, no tips
Spd (1,6): ok,
Dir (1,6): ok

csat diag: ok, spike at W Slope around noon yesterday
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, most < 1 V this morning
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok
Qsoil: ok, up to 38 vol% at Playa
Gsoil: ok
Cvsoil: ok, W Valley continues to have spikes at ~ 5 hr intervals

14:20 - 14:40

Clear with haze

Took a soil sample in the rocky soil.

Dessicant replaced in the radiometers, incoming long and outgoing short.

12:47 - 13:00

clear with haze

Tom did not like his rabbit ears so we re-adjusted the shadowband. to an angle of 32.5 degrees. The ring was also moved along the arm to better center the licor in ths plane of the ring.

Status Jan 29, 2011

Status evaluated from 36 hour plots on Jan 29 09:15-10:10 MST
On site: Semmer and Horst

Weather: overcast skies this morning, light winds

Summary: large disagreement between two Tsoil probes at E Slope
         shadow band at Riverton produced rabbit ear yesterday evening
         
Todo: Soil samples at 1 and 2

Vdsm: ok, > 12.3 V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.2 V  E Slope lowest

P: ok
Tbaro: ok
T: ok
RH: ok
Q: 3-4 g/kg
Rainr (3,4,6): ok, no tips
Spd (1,6): ok,
Dir (1,6): ok

csat diag: ok, spikes at E Slope around noon yesterday
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, all but 3 and 5 above 1 V
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: very large difference between profile and linear probe at E Slope
Qsoil: ok, up to 38 vol% at Playa
Gsoil: ok
Cvsoil: ok

15:10 - 15:30

Revisited station 7 to see how the shadowband is doing.

The sun was starting to creep up on the licor so Tom increased the angle of the arm which decreased the angle of the ring.

The inclinometer now reads 31 degrees.

14:15 - 14:30

Clear skies with a touch haze (air pollution)

soil sample taken, containers 2:0-3 and 2:3-6

Dessicant in radiometers ok

12:35 - 12:50

Clear with a little haze

adjust shadowband  elevation for 31.36 degrees at 12:41

Best we could do with the inclinometer was about 32 degrees.

11:10 - 11:35

clear with a touch of haze

11:15   soil sample taken, containers 1:0-3 and 1:3-6,  dessicant ok in radiometers

Worked on router to get ststaion back on the air. Gordon provided a list of commands to try. Based on the results of the commands he was able to determine why the router locked up. He will work on a fix.

Recycled power to router and got it going.

wvally4 comms

Jan 28

Comms to wvally4 went down last night, in a similar way that hiland3 dropped out the previous night. hiland3 was able to restablish comms itself via the router_check.sh script, but wvally4 has not come back online.

Steve and Tom are at the site.

11:20 am:

router LEDs:
power on
wan flashing
wlan off
usb off

modem led: solid green

ifconfig eth0 looks ok

host www.google.com
64.233.187.99

ping www.google.com works

host www.eol.ucar.edu
169.254.222.173

That 169.254. address is a "private APIPA" address, which is what some systems use if they can't find a DHCP server.
Pings to that address succeed, and so the router_check.sh script thinks everything is OK.

I'll update the router_check.sh scripts so that if the host command returns 169.254.. then it will restart the router.

Status Jan 28, 2011

Status evaluated from 36 hour plots on Jan 28 09:00-10:00 MST
On site: Semmer and Horst

Weather: overcast skies this morning, light winds

Summary: Replaced TRH at E Slope yesterday
         W Valley not reporting due to Verizon disconnect
         Same thing happened at Highland yesterday

ToDo: Take soil samples at 3 and 4
      Fix comms at W Valley
      Adjust shadow band at 7

Vdsm: ok, > 12.3 V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.3 V

P: ok
Tbaro: ok
T: E Slope replaced yesterday
RH: ditto
Q: 3-4 g/kg
Rainr (3,4,6): ok, no tips
Spd (1,6): ok,
Dir (1,6): ok, currently S

csat diag: ok, spike at 5 corresponds to adjustment of wicks
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, some apparently still wet
kh2o: ok
kh2o'kh2o': noisy
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok, note that linear sensors at 5 & 6 do not agree with soil profiles
Qsoil: ok, up to 38 vol% at Playa
Gsoil: ok
Cvsoil: ok

Clear sky except for pollution

16:00 - 17:00

16:10   Tower lowered, cleaned krypton, checked bolts,

            cleaned radiometer domes and checked dessicant

16:18   Mast back up

16:30   Took soil sample, Tupperware 3:0-3, 3:3-6

16:35   Pounded in stakes

16:54   adjustment of shadowband

14:15 - 15:15

Clear day, 50% snow cover on ground.

14:23    SHT018 replaced with SHT009

14:28    Mast lowered

14:31    cleaned krypton with water (1.6v to 2.35v), retracted sonic wicks, checkd tower bolts, soil sample taken at main site

14:50    raised mast

14:55    cleaned radiometer domes with water, soil sample at secondary site,  sample depths backwards in tupperware

13:14    pounded tower stakes in

Status, Jan 27, 2011

Status evaluated from 36 hour plots on Jan 27 10:00-11:00 MST
On site: Semmer and Horst, Militzer leaves today

Weather: clear this morning

Summary: TRH at E Slope has been flaky

Vdsm: ok, > 12.3 V
Vmote.rad: ok, > 12.5 V
Vmote.soil: ok, > 12.3 V

P: ok                                                                                           
Tbaro: ok
T: Playa fixed yesterday; E Slope had bad values 1700 - 1735 Jan 26
RH: ditto, RH at E Slope looks bad
Q: ditto
Rainr (3,4,6): ok, no tips
Spd (1,6): ok,                                                                                  
Dir (1,6): ok, currently southerly

csat diag: ok, spikes at 3,5,6
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok                                                                                        
u*: ok
sigma_w/u*: ok

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, all > 1V prior except Playa, which appears to be drying
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok, E Slope appears to still have some snow
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok, Playa appears to be drying or fog clearing
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok, note that linear sensors at 5 & 6 do not agree with soil profiles
Qsoil: ok, up to 38 vol% at Playa
Gsoil: ok
Cvsoil: spikes at ~5 hr intervals at W Valley

Jan 27
There is a plot gap of about 20 minutes last night from all stations except 1 at 5z (10 pm MST). This looks like a Verizon network interruption. If it was an EOL server problem, all stations would have the gap.

hiland3 didn't come back online after the interruption, until 15z, 8am MST this morning. I logged in and saw that the router_check.sh script was not being run from 2 to 14z in crontab, which was the plan when we were turning off the routers and modems overnight. When the script ran at 15z it worked as it should and brought the system back online. I changed hiland3's crontab back to the original settings where router_check.sh is run every hour (and net_check.sh every 15 minutes). Also logged into all other stations to verify their crontabs.

Stn2 Visit-2 26Jan11

26Jan2011 15:05-15:25 local Militzer/Semmer

Conditions: partly cloudy, low q with high humidity, calm, upper 30's

Purpose: clean kh2o

15:10-15:18 Tower Down

15:15 Kh2O cleaned with methanol and distilled.  Voltage went from 0 to 1.8 volts

Rad was cleaned earlier.   No snow now at all

Stn4 Visit 26Jan11

26Jan2011 14:10-14:40 local Militzer/Semmer

Conditions: above freezing, clear here but fog throughout valley, calm

Purpose: cleaning

14:22 Rad: 1 drop of water cleaned off Rsw.in

14:20-14:30 Mast down

14:22 Krypton cleaned.  voltage went up from 0 to 1.4+ volts

Snow Depth: 1" The snow is mostly covering the ground here all around but melting in steadily.

Stn1 Visit 26Jan11

26Jan2011 12:50-13:30 local Militzer/Semmer

Conditions: humid air, patchy fog in valley, mid 30's, snow basically gone, marshy soil

Purpose: check/fix trh, set diffuse radiometer, cleaning

13:00 Rad cleaned Rsw.in that had a couple of water droplets on it

13:06 Rad: adjusted shadow band up to correct elevation angle.   It had been at about 40deg and needed to be 31deg; set it at ~32deg for 'windage'.   This was done by placing a square on the 'ephermous ring' and an inclinometer on that for the peak sun elevation on the date/latitude.   Also adjusted the ring up a bit and leveled the sensor.   Couldn't exactly see it but believe that the previous setting was also being shaded but may have had some exposure at the beginning/end of the day due to incorrect tilt.

13:10 Rad: cleaned shadow band licor with methanol which had some bird poop on the plate and sensor head.   The pile, being white, didn't really show much evidence of that but almost certainly it had some on it as well.

13:05 TRH swapped, #13 for #21.  By the time we got here it was "working" but at ~1.8deg, 98%humidity.  After swap it was at ~2deg (ie same) but down to 78%rh

13:15 Krypton cleaned.   output went up to 1.25volts from zero.

Took a couple pictures of the standing water under the rad stand and soil mote.

Stn2 Visit 26Jan11

26Jan2011 10:00-10:15 local Militzer

Conditions: very foggy, calm

Purpose: Check radiometers, solar panels and measure snow

PV: solar panels have cleared off from the snow yesterday...no problems

Snow Depth: very little left, just a little bit but spotty around rad.   3/4" was deepest patch i could find next to rad.

10:09: Rad Rsw.in had light frost covering cleaned off with methanol.   Other sensors were clear.

Kh2O voltage zero, wet in fog.  no 2nd person to lower tower

Status, Jan 25

Status evaluated from 36 hour plots on Jan 25 09:30-11:45 MST
On site: John and Tom

Weather: snowing this morning

Summary:

Vdsm: ok, ~14.5 V yesterday
Vmote.rad: ok, > 14 V yesterday
Vmote.soil: ok, > 14 V yesterday

P: ok
Tbaro: ok
T: ok
RH: ok, 70-90 %RH
Q: ok, up to 4 g/kg
Rainr (3,4,6): ok,  snow this morning seen at all three sites
Spd (1,6): ok,
Dir (1,6): ok, currently NW

csat diag: ok, spikes at E Slope corresponding to adjustment of wicks
spd: ok
dir: ok
u'u': ok
v'v': ok
w'w': ok
u*: ok
sigma_w/u*: ok, ~1.5

tc: ok
tc'tc': ok
w'tc': ok

kh2oV: ok, all > 1V prior to morning precip
kh2o: ok
kh2o'kh2o': ok
w'kh2o': ok

Rsw.in: ok
Rsw.out: ok
albedo: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok
Rpile.in: ok
Tcase.in: ok
Tdome.in (1,2,5): ok

Rlw.out: ok
Rpile.out: ok
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok, note that linear sensors at 5 & 6 do not agree with soil profiles
Qsoil: ok, up to 38 vol% at Playa
Gsoil: ok
Cvsoil: spikes at 2,3,4; at ~5 hr intervals at W Valley

Status, Jan 24, 2011

Status evaluated from 36 hour plots on Jan 24 09:00- MST
On site: John and Tom

Wx: clear skies, light winds
Summary:

Vdsm: ok,
Vmote.rad: ok
Vmote.soil: ok

P: ok
Tbaro: ok
T: ok
RH: ok, 50-90%
Q: ok, 2.4 - 3.6 g/kg
Rainr (3,4,6): ok, 2*3 mm/hr at Highland morning of 1/23
Spd (1,6): ok, up to 6 m/s
Dir (1,6): ok, currently Sw

csat diag: ok

kh2oV: ok

Rsw.in: ok
Rsw.out: ok
Rsw.dfs (1,7): ok
Rsw.global (1,7): ok

Rlw.in: ok

Rlw.out: ok

soil.aux at E & W Slope (5,6)
Tsoil: Tsoilaux 4.4 cm at Eslope misbehaved 1/23 11:00-15:00
Qsoil: ok
Gsoil: Highland has some spikes
Cvsoil: spikes at ABC, W Valley

24Jan2011 18:00 - 18:10 local Militzer/Horst

Conditions: beginning to cloud up, after sunset

Purpose: Reset Rad Mote ID from 11 to 9

Reprogrammed Rad mote using default id=9.   Was unable to get a reliable command interaction with the motes (10,11,21) at this station.   The base mote was at power level 0 instead of 4.   Gordon reset this from Boulder using the '+++', 'atpl4', 'atwr' sequence, but we were still unable to get a good mote '#11id=9' command through to it, necessitating this change.   I had set the default id incorrectly earlier today when updating the software.

NOTE because the radiation mote id was different than expected in the xml, there will be a data gap in cockpit/web-plots.. but that can be later merged in during post processing.

NOTE: need to checkout the comms at this site a bit more thoroughly because there may be some non-optimal settings in the xbee configurations.

Stn 2 Visit 24 Jan 2011

24Jan2011 16:20-17:15 Militzer/Horst

Conditions: becoming overcast, sun going down, calm

Purpose: Cleaning, Re-Init Router, Mote software update

16:23-16:28 Update mote software.   Finger nail polish coating added to antenna bases.

16:23-16:28 Rad cleaned and downlooking rlw desiccant refreshed

16:40-17:00 Router Configuration refreshed after it had dropped back into factory defaults a couple days ago.  No data loss, but will need to fill those gaps with rsync tonight.

Stn 3 Visit 24 Jan 2011

24Jan2011 14:55-15:55 Militzer/Horst

conditions: calm, upper 30's, high overcast, moisture coming in tonight

Purpose: Cleaning, Update mote software, check ETI

14:59-15:00 Rad. cleaned with methanol and distilled.  Desiccant ok.

14:59-15:03 Mote software updated.    Put fingernail polish on antenna connectors at interface of whitebox.

15:10-15:28 Mast lowered.

15:18 Kh2O cleaned with methanol and distilled.  Went from .92 to 2.0 volts.

Sonic wicks retracted a bit because they were extending a bit too far into the measurement paths.

15:39-15:58 A few adn/aup to try to communicate with ETI, but we were using the wrong approach to try it.  Did not have the correct serial cable to tie-in directly to gauge.   NOTE still need to confirm that the TO/1 temperature compensation is on.

Note: mast bolts ok, stakes pounded in on mast and rad stand

Stn 4 Visit 24 Jan 2011

24Jan2011 13:50-14:30 local Militzer/Horst

conditions: calm, upper 30's+, clear

purpose: Cleaning

13:55-14:23 mast down

14:02 Rad cleaned with methanol.  Desiccant ok.

14:09 Kh2O cleaned with methanol.   Went up from 1.0 to 1.2 volts.  Cleaned a bit later with distilled and went up to 2.0 volts

NOTE: Mast bolts ok.   BUT south leg stakes were loose and the footpad rose up when we lowered the mast.   Ground is moist and appears that the stakes have all loosened.   We re-set one stake and pounded down the others but did not have spare stakes and wire to improve the situation substantially.

24Jan2011, 10:30-12:30 local, Militzer/Horst

Conditions: mostly calm, upper 30's, clear

Purpose: RMY swap, Soil Samples, Update Mote Code

10:40 Rad Cleaned with Methanol, did not notice desiccant condition

10:35-10:45 Mote Software Updated version 2.4

11:00-11:27 Mast Lowered.   11:08 RMY swapped.   Previous sensor seemed ok and did not hear any bearing noise.   Swapped anyway.   Zeroed and 'boom-directioned to zero' on new sensor: consistent with previous unit.   It did seem as though the new sensor's prop was rotating and we didn't noticed a reported velocity.   It did start reporting speed with numbers about .4 so we assumed it is ok.   11:11 adn and aup while setting 'ZN' and 'BD'

11:15 Kh2O cleaned.   Voltage rose from 1.25 to 1.6 volts.

NOTE: Mast bolts on lowest joint were slightly loose; tightened them up.

ETI gauge was 2/3 full so we expect it to drain soon.  

11:30-12:30 Soil Samples taken first at the main and then auxiliary locations.   Container 1,0-3 and 1,3-6 were at main.   Containers 2,0-3 and 2,3-6 were at aux.

Soil Analysis:

Location

Sample:
depth (cm)

Tray

wet wt
net (oz)

dry wt
total (oz)

pan wt
(oz)

dry wt
net (oz)

water
 (oz)

6 - main

1: 0-3

4-0

4.415

3.305

0.135

3.170

1.245

6 - main

1: 3-6

4-6

5.705

4.890

0.125

4.765

0.940

6 - aux

2: 0-3

2-0

3.420

2.715

0.135

2.580

0.840

6 - aux

2: 3-6

2-6

4.775

4.094

0.130

3.964

0.811

23 Jan: Oncley & Militzer

Purpose: Replace prop since twice has stalled for several hours overnight; new mote software?

1307: Aborted due to mud -- less than the last time we were there (primary because the mud was more frozen), but still very sloppy.  Recommed using the pickup next time.

23 Jan: Oncley&Militzer

Conditions: just a bit of snow clumped on the curb

1210: adjusted SPN-1 level (off by about a bubble -- the soil post is leaning)

1213: dry wiped radiometers

Until 1225: messed with shadowband adjustment.  Shifted the aziumth a bit to point nearly S, adjusted elevation to 60 deg (from 53), adjusted sensor level, moved collar to center shadow.

Daily status 23 Jan

Daily status 23 Jan 2011
On site: Oncley, Militzer

After a nice morning, yesterday turned cloudy with snow flurries most of the day.
This morning has mostly clear skies, with inversion starting in the valley.
Some frost on the van windshield at the Rez this morning.

Summary:
    - krypton at 3 was bad most of yesterday, but recovered overnight.
    - modem at 2 appears to have lost settings -- we'll reprogram it today
    - Shadowbands at 1&7 should be adjusted.  Sebastian will do 1 and we'll do 7.
    - Prop-vane at 6 has stalled at night for the second time.

Vdsm: ok.
Vmote.rad: ok.
Vmote.soil(aux): ok.

P: ok, rising since min at 6AM yesterday
T: ok, -7 to +6 degC
RH: ok, 45-95%.
Rainr (3,4,6): ok, hiland3 reported 1 tips yesterday (real, from snow) and 2 last night (either still from snow or false tips).
Spd (1,6): Prop stalled again at 6 last night, similar to a few days ago.  It <may> be related to freezing, but there wasn't much moisture.  It has recovered now.
Dir (1,6): See Spd.

csat diag: ok, several sites showed glitches at 4PM yesterday -- probably not real
samples.sonic: ok
spd: ok,
dir: ok, mostly N during the day yesterday
u'u': ok,
v'v': ok,
w'w': ok,
u*: ok, max 0.7 m/s
sigma_w/u*: ok

tc: ok,
tc'tc': ok
w'tc': ok, -30 to +200 W/m2.  Some sites apparently had nice conditions yesterday.

kh2oV:  ok, max 1.4--2.0 yesterday, stn 3 down yesterday, now ok
kh2o'kh2o': ok
w'kh2o': ok, up to 0.05 (120 W/m2) during day yesterday

Rsw.in: ok.  max 550-780 W/m2 yesterday!
Rsw.out: ok.
albedo: ok, 0.07-0.25, except for higher morning values with frost on incoming
Rsw.dfs (1,7): "ears" in morning and afternoon in shadowbands (but not SPN-1) show that adjustment is needed (see Summary)
Rsw.global (1,7): ok.

Rlw.in: ok, 200-320 W/m2
Rpile.in: ok,
Tcase.in: ok, 0-16 degC
Tdome.in (1,2,5): ok

Rlw.out: ok, 280-400 W/m2
Rpile.out: ok,
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok. Getting signficant diurnal swings, many into freezing at night
Gsoil: ok. All but 7 were negative yesterday
Qsoil: ok. apparent or real increases due to soil thawing
Cvsoil: ok. Jump at both sites at 6 just now, probably somehow due to water.

snow

22 Jan, 1150: Some serious snow flurries started at the University and still going 40 minutes later.  It had been nice when the daily briefing ended at 1045.

1420: Snow ended; had changed between soft hail, flurries, and some wet flakes

Daily status 22 Jan

Daily status 22 Jan 2011
On site: Oncley, Militzer

A bit of rain last night, morning of broken clouds.

Summary:
    - kryptons at 3,4,6 reading 0 due to the rain
    - 3G modem at 4 died yesterday and wasn't rebooted by dsm crontab.  Data saved locally, so no loss.
    - Shadowbands at 1&7 should be adjusted (more).  Inclination angle should be set to 60 deg.
    - Oil clean-up people left yesterday near base.
    - Tour given to John Horel's undergraduate instrument course yesterday

Vdsm: ok,
Vmote.rad: ok.
Vmote.soil(aux): ok.

P: ok, steady after falling until 11PM last night
T: ok, 0 to +7 degC (warmer)
RH: ok, 40-95%.
Rainr (3,4,6): ok, 0.8, 0.3, 1.0mm overnight, respectively
Spd (1,6): ok
Dir (1,6): ok

csat diag: ok, 6 & 7 showed simultaneous glitches yesterday, so must be physical
samples.sonic: ok,
spd: ok,
dir: ok,
u'u': ok,
v'v': ok,
w'w': ok,
u*: ok, max 0.8 m/s
sigma_w/u*: ok, a few low values at 3 (why not at 2??)

tc: ok,
tc'tc': ok
w'tc': ok, -0.01 to +0.10 m/s degC.  Was pretty nice yesterday

kh2oV:  ok, max 1.5--2.2 yesterday, stns 3,4,6 now 0 due to rain
kh2o'kh2o': ok
w'kh2o': ok, up to 0.02 (50 W/m2) during day yesterday

Rsw.in: ok. overcast yesterday morning, but cleared up in afternoon
Rsw.out: ok.
albedo: ok, 0.1-0.2
Rsw.dfs (1,7): "ears" in morning and afternoon in shadowbands (but not SPN-1) show that adjustment is needed (see Summary)
Rsw.global (1,7): ok.

Rlw.in: ok, 240-320 W/m2
Rpile.in: ok,
Tcase.in: ok, 0-16 degC
Tdome.in (1,2,5): ok

Rlw.out: ok, 310-390 W/m2
Rpile.out: ok,
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok. Only 7 is still below freezing
Gsoil: ok. Interesting abc change during the day yesterday
Qsoil: ok. apparent or real increases due to soil thawing
Cvsoil: ok.

wvally4 site visit

21 Jan: Militzer&Oncley

Purpose: Entire station lost communication at about 1:30 today (while we were giving the tour of site5).

We've seen lots of reconnects in dyndns both since midnight last night and during the day yesterday.

Steps:

Cradlepoint WAN quick blinking/no USB or WLAN LED.  Cycled power to cradlepoint.  USB now on; WAN blinking; external connection via dyndns still not up, though www.dyndns does show request; am able to ssh in with IP address; data acquisition has been running.

See in messages that router_check ran, now access via isfs4.dyndns is working.

At Gordon's suggestion, did a bunch of tests of check_router script (which should have fixed this problem by itself).  It has a long time-out of 3--5minutes, but did work as expected with the LAN cable unplugged.  We next want to run a test by unplugging the 3G modem, but ran out of power on our laptop.  We'll do this test tomorrow on the spare in the base.

P.S. The next day, we did this test on the spare, which worked exactly as expected.  Thus, this failure at wvally4 is still a mystery.

eslope5 site visit

21 Jan: Oncley&Militzer

Purpose: Class tour; reprogram motes; clean radiometers

Didn't record exact time, but everything done between 1300-1345.  Should be able to tell from mote data.

The data may record that I did an "eio 5 0/1" cycle as a demonstration.  The radiometers had been done by this time.

PS: From the data, the times of these events were:

12:57:50 rad mote restarted

13:00:15 main soil mote restarted

13:04:37 aux soil mote restarted

13:13:20 Rsw.in cleaned

13:13:40 Rlw.in cleaned

13:13:50 Rlw.out cleaned

13:13:55 Rsw.out cleaned

13:22:22 TRH eio

Daily status 21 Jan

Daily status 21 Jan 2011
On site: Oncley, Militzer

High overcast over the entire valley; an inversion developing especially to the south. 
IOP6 is in progress.

Summary: Everything OK.

Vdsm: ok, Min this AM: 12.3-12.5V. (great charging yesterday when completely clear)
Vmote.rad: ok.  Min this AM: 12.5-12.9V
Vmote.soil(aux): ok. Min this AM: 12.2-12.8.

P: ok 850 to 890 mb, minium last night, now rising
T: ok -6 to +2 degC
RH: ok 50-85%.
Rainr (3,4,6): ok, zero everywhere
Spd (1,6): ok
Dir (1,6): ok

csat diag: ok.
samples.sonic: ok
spd: ok, river7 higher than the rest (normal for this situation)
dir: ok.
u'u': ok,
v'v': ok,
w'w': ok,
u*: ok, max 0.4 m/s
sigma_w/u*: ok, odd at eslope5 at 01:30

tc: ok,
tc'tc': ok
w'tc': ok, -0.02 to +0.10 m/s degC.  Nice clear-sky case.

kh2oV:  ok, max/now 1.2--2.3
kh2o'kh2o': ok
w'kh2o': ok, up to 0.03 (75 W/m2) during day yesterday

Rsw.in: ok. Mostly clear yesterday with spikes both high and low
Rsw.out: ok.  Strange shape at wslope6, presumably due to snow melt
albedo: ok, 0.1-0.2 by the end of the day yesterday, though higher again this morning (probably frost on incoming dome)
Rsw.dfs (1,7): large values morning and evening even though shadowband at playa1 adjusted yesterday and river7 a few days ago.  Note that SPN-1 (with no adjustment capability) shows the same.
Rsw.global (1,7): ok.

Rlw.in: ok, 220-300 W/m2.  Coherence across network last night
Rpile.in: ok,
Tcase.in: ok,
Tdome.in (1,2,5): ok

Rlw.out: ok, 290-360 W/m2
Rpile.out: ok,
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok. many sites have freeze/thaw cycles
Gsoil: ok. Interesting abc change during the day yesterday
Qsoil: ok.
Cvsoil: ok.

playa1 site visit

20 Jan: Oncley & Militzer:

Purpose: Adjust shadowband; nail polish mote antennae; scout for tour

Conditions: water all over the field -- perhaps 50% cover.  Not good for a 14-person tour...  Soil clearly at field capacity, though probably is frozen at some depth.  Some of the water has an ice film (despite Tsoil reading about +4).

1634: Adjust shadowband - azimuth change from about 170 to 180 degrees; elevation change from about 35 to 53 degrees; releveled; adjust ring up a bit to center shadow.

about the same time...reprogram motes with new code; forgot to turn back on

1650: motes back alive

daily status 20 Jan

Daily status 20 Jan 2011
On site: Oncley, Militzer

Absolutely clear morning with great visibility.  There appears to be fog over the lake.
There was frost on the van windows at the Rez.
Everything's working, so we're off to Campbell Sci.

Summary:
    - Kryptons are happy
    - Batteries are happy
    - TRHs are happy
    - Motes are happy (we had to swap one yesterday that had rain damage)
    - We're happy

Vdsm: ok, Min this AM: 12.2-12.5V.
Vmote.rad: ok.  Min this AM: 12.4-12.9V
Vmote.soil(aux): ok. Min this AM: 12.2-12.8.

P: ok 850 to 875 mb, rising
T: ok -8 to +3 degC
RH: ok 60-90%.
Rainr (3,4,6): tips at hiland3 early this morning can't be real...
Spd (1,6): Prop stalled at wvally6 this morning, still hasn't recovered
Dir (1,6):

csat diag: ok.
samples.sonic: ok
spd: ok, except 5 now
dir: ok, some sustained E and W at different sites -- good drainage case?
u'u': ok,
v'v': ok,
w'w': ok,
u*: ok, max 0.6 m/s
sigma_w/u*: ok

tc: ok,
tc'tc': ok
w'tc': ok, -0.04 to +0.08 m/s degC (-40 to +80 W/m2);

kh2oV:  max/now 1.3--2.1, the last krypton recovered from rain at 5pm yesterday
kh2o'kh2o': ok
w'kh2o': ok, up to 0.03 (75 W/m2) during day yesterday

Rsw.in: ok.
Rsw.out: ok
albedo: ok, 0.1-0.3 by the end of the day yesterday.  The snow we saw at 6 must be melting (hence the mud)
Rsw.dfs (1,7): ok, but SPN-1 seems to be high.
Rsw.global (1,7): ok.

Rlw.in: ok, 200-310 W/m2.  Cold and pretty clear last night (with a nearly full moon)
Rpile.in: ok,
Tcase.in: ok,
Tdome.in (1,2,5): ok. tiny differences between sensors 1-2-3 seen during the day

Rlw.out: ok, 280-350 W/m2
Rpile.out: ok,
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok. some sites freezing up again
Gsoil: ok.
Qsoil: ok. abc2 froze up again last night
Cvsoil: ok.  strange wslope6 increase...

wslope6 site visit

19 Jan: Oncley&Militzer

Purpose: take soil sample

Conditions: Still a glaze of snow everywhere, but lots of squishy mud at the surface -- 2 out of 5 on the mucky scale.  (Not sure if it could be torn down even using the pick-up.)

1420: soil core main

1425: soil core aux

nail polish all mote antenna

wiped off clear-hard ice from Rsw.in; dessicants ok

river7 site visit

19 Jan: (a frustrated) Militzer & Oncley

Reason: Rad mote had died at 0630, but fixed itself at 1045.  Still going to check it out...

Conditions: mostly solid deck across the valley, though broken towards the western mountains.  No snow at site.

1311: ID17 probably used the new version of code to internally reset the XBee.  It hadn't had a heartbeat, so it would have used the internal timers, which was set to reset every 6 hours.  We just changed it to 3 hours (#17xr=10800).

1317: Relevel SPN-1 (about 80% bubble off) -- it appears that the post may have shifted a bit, perhaps with soils thawing?; wiped ice off of dome

1318: Wiped a bit of ice from Rsw.in dome.

nail polished both mote antennae

1318-1320: Downloaded microSD data from rad mote, just to test (later) if data were lost during the 3 hours this morning.  We think the data are just fine.

1328: reprogrammed ID16 with auto-resetting/eeprom/etc. code

wvally4 site visit

19 Jan: Oncley&Militzer

Purpose: restart radiometer mote; take soil core

Conditions: broken alto-cu; snow completely gone around site

1137: Rad mote not working; removed for diagnosis;

water came in through the XBee antenna, dribbled down the XBee and got into the real-time-clock capacitors on the back of the board, taking out the clock.  Data would have been lost.  Copied data file to Oncley's Mac.  Also found broken solder connection on GPS backup battery holder.  Mote starts okay when pressing on this holder!

1140: Soil core taken into containers 1

1153: blotted water off radiometers (though no data being taken yet)

reprogrammed spare mote to be used at this site

1234: Installed new mote; all data working and okay

added nail polish around mote antennae to avoid a recurrence of this problem.

Daily status 19 Jan

aily status 19 Jan 2011
On site: Oncley, Militzer

Rain started at 2AM last night and killed kryptons.  Sleet on campus.

Summary:
    - All kryptons now dead...again...
    - Radiometer motes at 4&7 both lost connections at 0630 this morning.

Vdsm: ok, Min this AM: 12.4-12.5V.
Vmote.rad: ok.  Min this AM: 12.5-13.0V
Vmote.soil(aux): ok. Min this AM: 12.3-12.9.

P: ok 850 to 875 mb, rising, minimum about 0100 last night.
T: ok 0 to +8 degC.  Hiland3 now fixed.
RH: ok 30-95%.
Rainr (3,4,6):  ok. Totals 9.4, 3.4, 7.1mm, respectively last night.
Spd (1,6): ok.
Dir (1,6): ok.

Sonics: 3, 5, 6 having problems in rain -- 5 especially has died

csat diag: ok. 3, 5, 6 all having problems with rain.
samples.sonic: ok, some big dropouts from 4 & 5
spd: ok, except 5 now
dir: ok,
u'u': ok, max 2 m2/s2
v'v': ok, max 3 m2/s2
w'w': ok, max 1 m2/s2
u*: ok, max 0.8 m/s
sigma_w/u*: ok

tc: ok,
tc'tc': ok max 0.5 degC
w'tc': ok, -0.1 to +0.1 m/s degC (-100 to +100 W/m2);

kh2oV:  max 1.0--2.5, but now 0 in rain.
kh2o'kh2o': daytime up to 0.03
w'kh2o': mostly up to 0.04 (100 W/m2) during day yesterday

Rsw.in: ok, max 580-670 W/m2 yesterday.
Rsw.out: ok, max 60-180 W/m2 yesterday.  (no snow!)
albedo: ok, were all <0.2 at the end of yesterday, but now 5 & 6 are >0.6 with fresh snow/sleet.
Rsw.dfs (1,7): ok, but SPN-1 seems to be high.
Rsw.global (1,7): ok.

Rlw.in: ok, 240-330 W/m2.  Coherent pattern across the network.  A few low clouds at 1AM at 3&5?
Rpile.in: ok, -120 to 0 W/m2
Tcase.in: ok, 0 to +13 degC
Tdome.in (1,2,5): ok

Rlw.out: ok, 320-380 W/m2
Rpile.out: ok, -50 to +20 W/m2
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok. Interesting stuff.  All but 7 completely thawed.
Gsoil: ok. All sites had negative fluxes yesterday -60 -- +20 W/m2, almost like "normal" conditions
Qsoil: ok. Lots of movement, upward from rain
Cvsoil: ok.  Mostly tracked Qsoil increases yesterday

River7 heat capacity

I note that recent soilT profiles show intense heating of the top soil layers during the day, followed by almost instantaneous cooling back to below freezing.  Indeed, there is a "rebound" effect where the top soil cools to below the lower soil, associated with a change of sign in the heat flux.  I believe these large changes should indicate a low heat capacity, though the Cv values are just average.  I do note that the soil density is lowest at river7, so Cp (mass-weighted) may be quite low.

hiland3 site visit

18 Jan: Militzer & Oncley

Purpose: We found an extra TRH in the bottom of our spares box!  Thus, a quick run to this site to change out the known faulty TRH (which will be sent back to Boulder tonight)

1638: old: TRH19 122.17 -1.33 16383 2\r\n

1640: new: TRH23 8.16 48.26 4792 1509\r\n

(looks much better)

Daily status 18 Jan

Daily status 16 Jan 2011
On site: Oncley, Militzer

Sprinkles today (thus warmer)

Summary:
    - WVally4 Temperature/RH sensor had been down since 2PM yesterday.  Fixed 8AM today.  No good excuses this time.
    - Since 2 TRHs have just died with getting wet, we want to try sealing everything better.  This AM we also went to WSlope6 and ABC2 to recoat/seal the SHT connectors.
    - Eslope5 Radiometers were down from 9AM - 2PM 14 Jan.  The radiometer XBee radio had to be reset.
    - All kryptons died in rain last night -- some we've let recover on their own, but we cleaned/dried 2&6.
    - Warmer temperatures should give another opportunity to take soil cores.  We'll wait another day or two.

Vdsm: ok, lowest values still at 7.   Min this AM: 12.2-12.5V. Lowest is river7.
Vmote.rad: ok.  Min this AM: 12.5-12.8V
Vmote.soil(aux): ok. Min this AM: 12.3-12.9.  lowest is eslope5 main.

P: ok 845 to 875 mb, falling.
T: ok -6 to +6 degC, Playa1 lower than the rest
RH: ok 73-102% overnight.  Playa1 highest (note the now 102%).
Rainr (3,4,6):  ok. Totals 0.8, 0.3, 0.5 mm, respectively last night.
Spd (1,6): ok.
Dir (1,6): ok.

csat diag: ok.  Many had issues last night.
samples.sonic: ok
spd: ok, River7 bad 0600-0700 this morning.
dir: ok, 4,6,7 stations south, others variable
u'u': ok, max 1.0 m2/s2
v'v': ok, max 1.2 m2/s2
w'w': ok, max 0.7 m2/s2
u*: ok, max 0.5 m/s
sigma_w/u*: 1&3 have excursions below 1

tc: ok, 5 bad in last night's rain
tc'tc': ok max 0.6 degC
w'tc': ok, -0.03 to 0.06 m/s degC;

kh2oV:  max 0.9--2.0 yesterday.  All bad overnight.  4&6 kleaned today.
kh2o'kh2o': daytime very small
w'kh2o': max << 0.01 during day

Rsw.in: ok, max 200-450 W/m2 yesterday.
Rsw.out: ok, max 130-280 W/m2 yesterday.
albedo: ok, 0.55-0.75 yesterday. Snow is going...
Rsw.dfs (1,7): max 350&365 W/m2 yesterday.  River SPN-1 somewhat higher
Rsw.global (1,7): Within 40 W/m2 of Rsw.in values -- also true for SPN-1.  (but see my logbook entry yesterday)

Rlw.in: ok, 240-340 W/m2.  Coherent pattern across the network
Rpile.in: ok, -90 to 0 W/m2
Tcase.in: ok, -6 to +16 degC
Tdome.in (1,2,5): ok

Rlw.out: ok, 280-320 W/m2
Rpile.out: ok, -70 to +5 W/m2
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok. concave profile at 2.  Frozen completely only at 7.
Gsoil: ok. 7 now negative
Qsoil: ok. Most have increased, either due to rain or thawing
Cvsoil: ok

abc2 site visit, jan 18

18 Jan: Militzer&Oncley

Purpose: clean krypton (hasn't been good for a while, despite methanol cleaning not long ago); NESW photos

Conditions: frosty, broken cirrus, great visibility

found honey bottle (without cap)

NESW photos taken

0810: mast down

0814: klean krypton with methanol 0.1 -> 1.7

0816-0818: klean krypton with distilled 0.0->1.7

0821: replace Rsw.in dessicant

0828: soil core taken

0829: mast up

0835: realized that barometer line was disconnected from barometer nipple.  reconnected

Rain at base

1647: started raining moderately heavy at base.  Great for washing out soil core sample dishes!

1701: ended

hail

17 Jan; 1603: FYI, we are in a small hail+rain storm at the 1300E interchange with I-80.  Only lentil-sized particles.  Hail stopped after a few blocks (<2min).

river7 site visit

Jan 17: Militzer&Oncley

Purpose: coat TRH; soil core

Conditions: Cirrus overcast; broken altoCu; only patches of snow left (20%?)

TRH looked good.  Added some nail polish and reinstalled.

1221: soil core taken (containers 5)

Added 3rd battery (just for giggles).  Should get a bunch of charging today.

1245: (radically) adjusted shadow band.  (Angle changed from 38->52 degrees

17 Jan: Militzer&Oncley

Purpose: coat TRH; soil core

Conditions: mostly overcast, calm, snow is melting fast here with perhaps 50% grass/soil exposed.

Radiometer dessicant okay -- no other action taken.

1110: TRH 18 removed for coating.   The probe connector had been greased previously.   There was some green corrosion on the transducer pins.   Cleaned with methanol and coated with nail polish before re-installation.

1120 MainSoil core sample taken (containers 3)

1132 AuxSoil core sample taken (containers 4)

Decided against Kh2o cleaning since yesterday it was at ~2.5v and there has been rain intermittently.   Currently at ~.3v

Decided against taking additional photos at this time because many have been taken here.

17 Jan: Oncley&Militzer

Purpose: Fix RH=0; soil; service; krypton

Conditions: Broken mid and upper-level cloud; snow about 1/4 gone

TRH had fixed itself, but serviced anyway

1010: soil core taken (containers 2)

TRH probe removed -- was wet and a bit of corrosion on SHT connector

old reading: TRH20 8.68 66.17 4861 2076\r\n (but we don't quite trust this) [pin now broken on this TRH :(]

new reading: TRH26 9.41 94.56 4914 3071\r\n (Goretex melted by methanol when we tried to clean)

newest reading: TRH19 11.56 83.88 5142 2666\r\n (when mast down)

1026: mast down

:) all bolts tight

1030: klean krypton 0 -> 0.9 (still not great...) (PS: up to 1.3 with mast up)

1032: wipe off radiometers; dessicant ok.

1045: mast up

photos taken

17 Jan: Oncley&Militzer

Purpose: coat TRH/soil sample/NESW photos/service

Conditions: calm, partly cloudy, above freezing, snow melting steadily.

0821: klean krypton (with residual water) 0 -> 1.1 V

take photos

0829: turn off TRH for fan/housing/sensor replacement.  Adding grease to connector.  Sensor removed, 26, appears to have coating on transducer pins.   The fan here was noisy but still aspirating.   Since we have new fans we swapped it out to be sure.   Very very small amount of corrosion on probe connector of housing that was here.

0830: turn on TRH new housing/sensor

0826: wipe off water (with residual water) from incoming domes, clean Rsw.dfs. Dessiccant is ok.

0835: soil core (but took 35 min to process since corer was jammed.) (containers 1)

hiland3 RH down

RH at hiland3 died within the hour.  Raw reading on this channel is now 0.  Temperature is okay.  Two "eio" power cycles and command "U"/"EXT" didn't help.  Guess we're off to the odd sites tomorrow...

TP01 values changing

With the rain today, we've started seeing large changes in values from the TP01.  As far as I can tell, there is a perfect correlation at every site with soil wetting and higher heat capacities from this event.  (This may be the first time I've seen this good a correlation.)

wind variance variations

In every daily status report, I've noted large variations in the wind variances from certain sites.  I've just poked into this a bit.  It appears that the variances become large for certain wind directions that generally are site dependent.  I've concluded that this effect is from local obstacles (though I haven't been able to completely understand S+ wind directions).  For example, at wvally4 I'm pretty sure that the highest variances are for winds over the mortuary building to the north-northeast.  River7 also has huge wind variances for the same north-northeast directions, that may also be a building, though this is not as obvious to me.

Daily status 16 Jan

Daily status 16 Jan 2011
On site: Oncley, Militzer

Sprinkles today (thus warmer)

Summary:
    - WVally4 Temperature/RH sensor had been down since 2PM yesterday.  Fixed 8AM today.  No good excuses this time.
    - Since 2 TRHs have just died with getting wet, we want to try sealing everything better.  This AM we also went to WSlope6 and ABC2 to recoat/seal the SHT connectors.
    - Eslope5 Radiometers were down from 9AM - 2PM 14 Jan.  The radiometer XBee radio had to be reset.
    - All kryptons died in rain last night -- some we've let recover on their own, but we cleaned/dried 2&6.
    - Warmer temperatures should give another opportunity to take soil cores.  We'll wait another day or two.

Vdsm: ok, lowest values still at 7.   Min this AM: 12.2-12.5V. Lowest is river7.
Vmote.rad: ok.  Min this AM: 12.5-12.8V
Vmote.soil(aux): ok. Min this AM: 12.3-12.9.  lowest is eslope5 main.

P: ok 845 to 875 mb, falling.
T: ok -6 to +6 degC, Playa1 lower than the rest
RH: ok 73-102% overnight.  Playa1 highest (note the now 102%).
Rainr (3,4,6):  ok. Totals 0.8, 0.3, 0.5 mm, respectively last night.
Spd (1,6): ok.
Dir (1,6): ok.

csat diag: ok.  Many had issues last night.
samples.sonic: ok
spd: ok, River7 bad 0600-0700 this morning.
dir: ok, 4,6,7 stations south, others variable
u'u': ok, max 1.0 m2/s2
v'v': ok, max 1.2 m2/s2
w'w': ok, max 0.7 m2/s2
u*: ok, max 0.5 m/s
sigma_w/u*: 1&3 have excursions below 1

tc: ok, 5 bad in last night's rain
tc'tc': ok max 0.6 degC
w'tc': ok, -0.03 to 0.06 m/s degC;

kh2oV:  max 0.9--2.0 yesterday.  All bad overnight.  4&6 kleaned today.
kh2o'kh2o': daytime very small
w'kh2o': max << 0.01 during day

Rsw.in: ok, max 200-450 W/m2 yesterday.
Rsw.out: ok, max 130-280 W/m2 yesterday.
albedo: ok, 0.55-0.75 yesterday. Snow is going...
Rsw.dfs (1,7): max 350&365 W/m2 yesterday.  River SPN-1 somewhat higher
Rsw.global (1,7): Within 40 W/m2 of Rsw.in values -- also true for SPN-1.  (but see my logbook entry yesterday)

Rlw.in: ok, 240-340 W/m2.  Coherent pattern across the network
Rpile.in: ok, -90 to 0 W/m2
Tcase.in: ok, -6 to +16 degC
Tdome.in (1,2,5): ok

Rlw.out: ok, 280-320 W/m2
Rpile.out: ok, -70 to +5 W/m2
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok. concave profile at 2.  Frozen completely only at 7.
Gsoil: ok. 7 now negative
Qsoil: ok. Most have increased, either due to rain or thawing
Cvsoil: ok

abc2 site visit, jan 16

16 Jan: Oncley&Militizer

Purpose: preemptive coating of TRH; krypton klean

1015: swap TRH with coated/greased sensor. 24 replaces 21

1019: mast down

1020: klean krypton, first time 1.3V;

1022: sprinkles just started

All mast bolts needed to be tightened (lowest section especially)!

1023: second kleaning 1.0V

1025: third kleaning. 1.1V

1032: mast up

1036-1040: replace dessicant in Rlw.in; wipe off incomings

forgot to take photos, but is raining.

Sunday 16Jan11 09:08-09:38 local; Militzer&Oncley

Purpose: Clean Kh2o, Check dessiccant, grease trh

0909: KH2O Cleaned.  Now reading 1.6V

0910: TRH out 24: It was working OK, however,...

0912: TRH in 27: We checked the DIN connector which was ok, but we cleaned it with ContactRenu anyway.   Sensor 24 now has nail polish on transducer pins and we greased the probe pins to hopefully avoid problems here as have been seen at other locations due to moisture and corrosion.

0913-0933: RAD: cleaned all.  Rlw.in,Rsw.out,Rlw.out had bad dessiccant, swapped; releveled (thought might have been okay)

Conditions: overcast, above freezing with snow cover melting.  Rain expected later today and tomorrow so expect kryptons to 'go away' at some point again.  Again

NESW Photos Taken

wvally4 site visit

16 Jan: Militzer, Oncley

Purpose: TRH died at 2pm yesterday.  Came back to life early this AM, but we don't believe it will stay up.  Might be possible to despike data from this period (0500-0800).

0815: removed probe 19; fan ok

0821: installed 25.   DIN connector had corrosion on it.   Cleaned with ContactRenu as best possible (seems ok) and put some grease on probe pins.   Also put nail polish on sht pins where attached to board.

0823-0832: Rad: Rlw.in,Rsw.in,Rlw.out dessiccant all shot; replaced.   Cleaned water off all  domes.

Conditions: Above freezing with snow cover on ground.   Some tufts of grass beginning to pop through snow.    Rain gauge has drained some fluid since our last visit.

Even though krypton is reading 0 (presumably water on lens), decided not to klean since more rain forecast for today and was reading 1.4V yesterday.

TRH at wvally4 down

TRH at wvally4 went nuts at about 13:45 MST, Jan 15.

rserial shows:

rs 5
TRH19 542.40 -1735.03 -99 -99\r\n
TRH19 542.40 -1735.03 -99 -99\r\n

Powering it down and up didn't help, it still showed -99 for the raw values.

eio 5 0
eio 5 1

Plots show a little bit of good data from about 21:50 MST to 22:00, then bad again.

isfs6_20110113_000000.dat is corrupt for some reason.

When one does an rsync, the following are reported on isfs6:/var/log/isfs/messages

Jan 15 23:15:55 isfs6 rsyncd[854]: rsync: read errors mapping "PCAPS/raw_data/isfs6_20110113_000000.dat" (in data): Input/output error (5) 

The rsync also results in a bunch of these errors in /var/log/isfs/kernel:

Jan 15 23:12:15 isfs6 kernel: attempt to access beyond end of device
Jan 15 23:12:15 isfs6 kernel: sda1: rw=0, want=78103088, limit=15825499

So that it doesn't get in the way of successive rsyncs, I moved the file to isfs6:/media/usbdisk/projects/PCAPS/bad

data_stats on the rsync'd file reports many bad samples, with incorrect types, ids, length, times, etc.

I was able to clear our most of the junk with these commands on the file at EOL:

sensor_extract -s 6,30 -s 6,100 -s 6,20 -s 6,50 -s 6,0x8000 -s 6,40 -s 6,80 -l 0 isfs6.dat isfs6_20110113_000000.dat 
nidsmerge -i isfs6.dat -o isfs6x.dat -s "2011 01 13 00:00"  -e "2011 01 13 12:00"
sensor_extract -s 6,30 -s 6,100 -s 6,20 -s 6,50 -s 6,0x8000 -s 6,40 -s 6,80 -l 0 isfs6xx.dat isfs6x.dat
bzip2 /scr/tmp/maclean/isfs6xx.dat -c > isfs6_20110113_000000.dat.bz2 

The real-time file for isfs6 for that day looks quite complete, so I don't think we have lost much, if anything.

We'll see if we have any other problems on the usb flash drive at wslope6. I plan to leave that file where it is in case it is covering a bad address on the flash, to avoid writing to that address again.

The glitch doesn't seem to have anything to do with a sudden power down, wslope6 has been up for 23 days.

Rsw

Comparing Rsw last night from the Kipp&Zonen, Quantum, and SPN-1 at 7, I notice that the Quantum reads very close to zero, the K&Z read an average of -1.5 W/m^2 (range -3 to 0), and the SPN-1 an average of +4 W/m^2 (range +2 to +7).  Apparently, the nighttime error is proportional to the cost of the sensor??

Daily status 15 Jan

Daily status 15 Jan 2011
On site: Oncley, Militzer

No precip last night.  Mid-level overcast.  Noticeably warmer.

Summary:
    - River7 Temperature/RH sensor had been down since 10AM 13 Jan.  Fixed 5PM 14 Jan.  We feel really bad that this was bad for so long.  Our only "excuse" is that the grey color used for River7 caused it to hide on the black axes of the WWW plots.  Also, since it reported (bad) values, cockpit didn't show it as missing...
    - Eslope5 Radiometers were down from 9AM - 2PM 14 Jan.  The radiometer XBee radio had to be reset. 
    - Only Playa1 and ABC2 kryptons read low values last night -- we'll let them recover on their own.
    - Warmer temperatures may give another opportunity to take soil cores.  We'll wait another day or two.

Vdsm: ok, lowest values still at 7.   Min this AM: 12.2-12.5V.
Vmote.rad: ok.  Min this AM: 12.5-12.8V
Vmote.soil(aux): ok. Min this AM: 12.3-12.9.  lowest is eslope main

P: ok 855 to 885 mb, falling.
T: ok -8 to +4 degC, Playa1 lower than the rest
RH: ok 75-101% overnight.  Playa1 highest (note the 101%).
Rainr (3,4,6):  ok. None.
Spd (1,6): ok.
Dir (1,6): ok.

csat diag: ok. Some issue at river7 at midnight.
samples.sonic: ok
spd: ok, River7 has strong drainage not seen at other sites (but consistent with other MesoWest stations)
dir: ok, most stations south, playa1 and wslope4 show some northerly intrusions
u'u': ok, max 1.0 m2/s2 higher variations 6&7
v'v': ok, max 1.2 m2/s2
w'w': ok, max 0.5 m2/s2
u*: ok, max 0.4 m/s
sigma_w/u*: 1&6 have the same excursions even below 1 (perhaps real?)

tc: ok
tc'tc': ok max 0.6 degC; large values at 5,6,7
w'tc': ok, -0.03 to 0.06 m/s degC; large overnight negative values at 5&6.

kh2oV:  max 1.0--2.1 yesterday.  1&2 bad overnight.  Lowest now is 0.3V@2
kh2o'kh2o': daytime very small
w'kh2o': max < 0.01 during day

Rsw.in: ok, max 450-650 W/m2 yesterday.
Rsw.out: ok, max 315-450 W/m2 yesterday.
albedo: ok, 0.7-0.8 yesterday. 6 had decreasing values during day, but back now.  All may get lower if warming continues.
Rsw.dfs (1,7): max 150&250 W/m2 yesterday.  River SPN-1 somewhat higher
Rsw.global (1,7): generally within 10 W/m2 of Rsw.in values -- also true for SPN-1.

Rlw.in: ok, 240-320 W/m2.  Coherent pattern across the network
Rpile.in: ok, -150 to 0 W/m2
Tcase.in: ok, -5 to +14 degC
Tdome.in (1,2,5): ok

Rlw.out: ok, 270-320 W/m2
Rpile.out: ok, -100 to +5 W/m2
Tcase.out: ok
Tdome.out (1,2,5): ok

soil.aux at E & W Slope (5,6)
Tsoil: ok. concave profile at 2.  Frozen (only) at 2, 7, some 5 and 6
Gsoil: ok.
Qsoil: ok. 2,5,6 partly frozen (consistent with Tsoil)
Cvsoil: ok

River7 site visit

14 Jan: Oncley & Militzer

Purpose: Fix TRH which died yesterday about 10AM (and no one caught :( )

1728: remove housing and old sensor (#17)

1730: NESW photos taken

1733: install new sensor (#28) using grease on contacts

1734: housing back on

Fan sounds fine during this process

eslope5 site visit

14 Jan: Militzer & Oncley

Purpose: fix rad.mote which died at 8:30 this morning (with Vmote = 12.5V)

(Huge funeral going on)

1415: Mote blinking normally; voltage okay; correct ?ID; not being receieved.  Cycled power, which reset Xbee and all now okay.  Local data store probably is okay. Would base mote reset (initiate with "eio 6 0/1") have solved this problem?

1420: Took NESW photos

1424: wipe off radiometers (just a few drops on uplooking domes); .outs dessicant looked good -- didn't check .ins. (just replaced 2-3 days ago)

kv = 2.0, so just fine.

TRH fan seems happy.

hiland3 site visit

14 Jan: Militer, Oncley

Purpose: radiometers were covered earlier, want to check that all is well; also check ETI

Excellent snowball snow (unconfirmed) -- grass clear of snow for part of walk to station.

1346: ETI gauge all liquid; no fluid needed or added

1348-1350: A few drops of water on Rsw.in and water in Rlw.out shield, but otherwise fine.  Wipe-off radiometers with residual water.  All dessicants nice and blue.

Daily status 14 Jan

Daily status 14 Jan 2011
On site: Militzer, Oncley (back again)

Another snow last night and low overcast in morning.

Summary:
    - Playa1 developed a memory issue yesterday and crashed at 4pm.  Rebooted this morning, but there are no data
       from last night.
    - "flux" computer in base was sluggish -- dsm_server died yesterday about 2pm.  Rebooted and servers restarted.
    - All kryptons read low values last night (must be because Oncley is now here).
    - 3Batteries Stn1,3,5,6;  Insulation added to all.

Vdsm: ok, lowest values still at 7.   Overall 12.2-12.4V.
Vmote.rad: ok.
Vmote.soil(aux): ok. lowest is eslope main

Current Conditions:

P: ok 845 to 885 mb, rising.
T: ok -4 to +1 degC
RH: Currently 60-100%. All >90% last night.
Rainr (3,4,6):  Totals 2.8, 1.8, 1.0mm, respectively, last night
Spd (1,6): ok.
Dir (1,6): ok.

csat diag: ok (slightly surprising, given snow, but playa1 sonic was clear of snow this morning)
samples.sonic: ok
spd: ok, Generally <2 m/s overnight.
dir: ok, most stations west
u'u': ok, max 1.0 m2/s2 higher variations 5&6
v'v': ok, max 1.0 m2/s2
w'w': ok, max 0.4 m2/s2
u*: ok, max 0.5 m/s
sigma_w/u*: 6&7 have the same excursions even below 1 (perhaps real?)

tc: ok, -4 to +2 degC.
tc'tc': ok max 2.2 degC; large values at 5&6
w'tc': ok, -0.06 to 0.06 m/s degC; large overnight negative values at 5&6.

kh2oV:  max 1.6--2.8 yesterday.  All bad overnight.  Lowest now is 0.3V@3
kh2o'kh2o': daytime very small, <.01.
w'kh2o': 0 to 0.010 during day

all radiometers could have snow (except 1 that we cleaned this morning)

Rsw.in: ok, max 350-650 W/m2 yesterday.  Now 100-400.
Rsw.out: ok, max 250-450 W/m2 yesterday.
albedo: ok, 0.7-1.0 yesterday.
Rsw.dfs (1,7): max 200-350 W/m2 yesterday.  River SPN-1 pretty close
Rsw.global (1,7): generally agrees with Rsw.in values

Rlw.in: ok, 230-320 W/m2
Rpile.in: ok, -150 to 0 W/m2
Tcase.in: ok, -5 to +14 degC
Tdome.in (1,2,5): ok, -5  to +14 degC

Rlw.out: ok, 285-320 W/m2
Rpile.out: ok, -80 to +5 W/m2
Tcase.out: ok, -5 to +14 degC
Tdome.out (1,2,5): ok, -5 to +13 degC

soil.aux at E & W Slope (5,6)
Tsoil: ok. concave profile at 2.  Frozen (only) at 2, 7, some 5 and 6
Gsoil: ok.
Qsoil: ok. 2,5,6 partly frozen (consistent with Tsoil)
Cvsoil: ok

playa1 data system hung on about 23z, Jan 13. When John/Steve connected to the console port the next morning it was reporting a lack of memory:

ntpd: page allocation failure. order:0, mode:0x20

This is a rare situation, that we have not seen previously at PCAPS. So far the stations have only died due to low power. river7 has been up continuously for 63 days.

To monitor memory usage on a station, the free command is useful:

ssh isfs1 free -m
             total       used       free     shared    buffers     cached
Mem:            61         59          1          0          7         43
-/+ buffers/cache:          8         52
Swap:            0          0          0

On the first line, the free value of 1 MByte out of a total of 64 looks alarming, but actually linux is using the free memory for buffers and cache, and is supposed to give it up to processes if they need it. The second line shows the free memory if you remove what is being used by buffers and cache, which is a healthy value of 52 MByte in this example.

My wild guess is that in rare circumstances, the kernel exhibits a bug where it doesn't free up the buffer/cache memory for use by processes. Time to upgrade the kernels from 2.6.16...

Or it could be due to a sensor input going bananas. I'll look into that.

Playa1 site visit

14 Jan: Militzer, Oncley

Reason: 3G has been down since yesterday afternoon.

0834: Jiggle router

0835: brush off solar panels

0836: brush off p port and TRH

0838-0840: brush off rads -- not much encrustation, just about a little bit of snow on 30% of uplooking domes/rest clear;  cleaned tops with methanol; dessicant on downlookers fine, uplookers about 1/2.

0840-0905: diagnosing station problem--viper was down since 4pm yesterday and lost data.  Now is working again.

P.S. from Gordon: "It somehow ate all its memory. I'll watch it and see if I can find the culprit."

0916: klean krypton (with new M1 methanol) 0.6V -> 1.9V

Site Visit Riverton7

13Jan11 15:15-17:00 local

conditions: overcast, very light winds

Reason: rlw.in down beginning at ~17z this morning

~21:55z back up

Desiccant shot on both rlw.in and rsw.in.   renewed both.

rlw.in =0x65, sn2

Had to dissect sensor and test w/serial mote in vehicle.  Probable cause may be the tiny 2pin,1mm connectors inside the case, in particular the power one, had some corrosion from water vaper.   After contact renew on i2c main connector and fiddling with those inside it resurrected.   Could also be wire problem with those also.    Keep an eye on this one.

router_check.sh script

Jan 13, 12:26 MST

Installed a new version of the router_check.sh script at eslope5.

It does a dns lookup on the destination hosts that are passed to the
script to be sure that the router's DNS isn't returning an address
of 192.168.0.1 for those hosts.

The hope is that the script will be able catch the problem at eslope5 last night,
and power cycle the router.

I'll watch it for a day before installing it on the other stations.

Later: Jan 13, 17:28 MST, installed new router_check.sh script on stations 2-7.
playa1 has not been accessible since roughly 16:00.

site eslope5 visit

13 Jan 2010 on site: John and Ling

840am - 910

850am: fix the no-realtime-communication problem:

         open the dsm box, router power on, wan on, usb off

         reset usb modem, it works.

         ssh to isfs5, checked ds and lsu

Rsw.in small amount of frost cleaned from dome as sun came over horizon

site playa1 visit

12 Jan 2011 1600-1630 on site John and Ling

1605 installed insulation and added a third battery. One of battery terminal lugs was lose when we arrived.  That may have had some effect on charging but minimal if any.    The TRH fan definitely 'noticed' the voltage swings as we paralleled in a 3rd battery onto the one that had the lose terminal.

Notes:

1> TRH fan is noisy and may need to be replaced.   It did sound more steady after adding the 3rd battery.

2> 2 Batteries' caps were/are missing. 

site wvally4 visit

12 Jan, 2011 on site: John and Ling

1520 to 1540

1525 turn off  wifi

1530 add insulation to keep batteries warm.  All terminal lugs were snug.

Nots:

checked radiometers. all clean

The ETI rain gauge has less fluid than when we were here Sunday so it drained some of its contents.  The slurry appears to have plenty of antifreeze.

site wslope6 visits

12 Jan 2011 on site: John and Ling

1425-1500

1428 turn off wiki

1430 install insulation for batteries, and add third battery

notes:

radiometers are clean.

rain gauge looks good.

site river7 visits

12 Jan. 2012 on site: John and Ling

start at 1325- 50

1230 turn off wiki

1337 install insulation for the batteries

note:

radiometers are checked and clean                                                                                                                 

site abc2 visits

12 Jan, 2011 on site: John and Ling

1210-1240

1210 install insulation for battery

1234: clean up radiometers (methanol first, then distilled water. tryout)

site eslope5 visits

11 Jan , 2011

1450-1605

1459:clean up radiometers,

15:45 change and add desiccants for the lower two radiometers

1545: add third battery, and add insulation

1600: turn off wifi

notes: radiometers:

significant trampling of snow on the south side of the saddle horse. prefer one trail from north to the stand. All others will be outside of the fence.

site hiland3 visit

11 Jan, 2011 On site : John and Ling

1305 clean up radiometers

1338 change the desiccants for the radiometers (2 long and 1 short, the upper right long doesn't need to change)

1356 add the third battery, and add insulation to the battery box to keep them warm (batt-dsm measured 14.44V)

1358 rain gauge, ice blogs

01/10/2010 ~15:00 MST Station-2, ABC, JM, Ling

Sonic Boom Angle check

The direction was shot from the east of the station looking into the CSAT.   DataScope reading = 258.5 with 0-deg. magnetic deviation used.   Handheld compass reading 261.5 with 0-deg magnetic deviation used.   We both tried the datascope and had the same results and agreed upon the location of the tripod.

visit site abc2, jan 10

10 Jan, 2010. 1205-1500 On site John, Ling

1405 wifi is  down

1415 clean up radiometers

1422 mast down

1425 clean up krypton , meausred 2V

1431 mast up

1500 shoot the boom (258.5deg from sonic toward the tower, get owner's permission to enter the field to read)

site playa1 visit

10 Jan, 2011. 1245-1330 On site: John and Ling

1245 turn off wifi.  We turned off the wifi to save what is estimated to be about .6W, roughly 1.2AH/day.   This site has been consistently having low battery swings at night.   Hopefully this will help save enough energy to have a measurable effect upon battery discharge.

1300 clean up raiometers

notes:

vbatt ok

desiccants ok

site playa1 visit

9 Jan 11, John, Steve, and Ling

1543-1550: clean up radiometers and shadow-band.  Rsw.in had a few crystals on the top of the dome.  Shadow-band sensor had bird droppings.  Shadow band looked a bit off, but didn't change.  Other quantum sensor okay.

TRH a bit noisy, but okay.

GPS jumpers were okay.

Well, we didn't get to all 7 sites, but only because ABC was locked.  We'll hit it tomorrow.

9 Jan: Dong, Militzer, Oncley

Continuing our odessy -- will we make it to every site today??

1433-1438: clean radiometers, Rsw.in and Rlw.in were snow encrusted

inspect ETI: okay

inspect TRH: okay

1438: mast down

1443: clean up krypton measuring 2.25V

1449: mast up

wslope6 site visit

9 Jan: Dong, Militzer, Oncley

1359-1402: clean radiometers

Solar panels free of snow

TRH fan a <bit> noisy

1400: inspected precip gauge -- not frozen/looked okay

Inspected GPS power jumpers in mote -- all were in place

river7 site visit

9 Jan: Oncley, Dong, Militzer

~1220: clean radiometers (including SPN1)

1225-1245: plugging/unplugging sensors to determine reason for pyg.in failure.  Second plug/unplug and white-button reset fixed.  Hmmm...

1245: declared "fixed"

9 Jan: Oncley, Dong, Militzer

brush a bit of solar panel

~1125: cleaned radiometers

krypton ok at 2.7 -- no action needed

1135-1138: replaced TRH fan housing since noisy fan.  Back ok

1140 snow flurries starting, though still 6/8 cloud

9 Jan

solar panel cleaning

brush off snow from pressure port and TRH (not plugging inlet)

~1020: radiometer cleaning

1033: mast down

1036: krypton klean 0.7-> 2.1V

1049: mast up

Daily status 9 Jan

Daily status 9 Jan 2011
On site: Dong, Oncley (Militzer expected later today)

IOP5 ended early this morning with a cold front bringing 1-2" of snow.
Weather: low overcast in the valley, clear at higher elevations.

Summary: Everything working as expected.  Some comments:
    - Kryptons have had continual problems with frost/snow.  Most clear themselves during the day.
    - TRH's have plugged up in riming conditions.  Use data with caution.  (It will be challenging to identify these cases.)
    - Props occasionally read lower even in winds greater than stall speed.  <might> be real, but icing also possible.
    - Precip tips overnight are the first real observations we've seen since we've been here (2 Jan)
    - Soils are freezing, making Qsoil measurements (that only measure liquid water) decrease
    - Conditions yesterday had Gsoil nearly constant; the digitizing resolution (0.1 W/m2) is seen in the data at all sites (except 7)

Vdsm: ok, lowest values 12.0-12.3V.  We may brush off panels today.
Vmote.rad: ok. lowest values 12.5-12.9 V
Vmote.soil(aux): ok, lowest values 12.3-12.9.

Current Conditions:

P: ok 835 to 870 mb, with weak minimum last night.  Now rising.
Tbaro:
T: ok -14 to -6 degC now, with up to 7 C drop at 6 overnight (but 2,5,7 show no drop)
RH: 85 to 97% now
Q: ok
Rainr (3,4,6): Tips overnight at all sites are real 1.0--3.1mm total
Spd (1,6): ok, 1,5 m/s now. 
Dir (1,6): ok, westerly now.

csat diag: ok.  Some spiking at 4,6 (and perhaps others) associated with snow.
samples.sonic: drop at 7AM this morning at 2,3,6 presumably snow related
spd: ok, 1--5 m/s now
dir: ok, W or N across network
u'u': ok, 0 to 1 m^2/s^2 dominated by events at 1 and 6 yesterday
v'v': ok, 0 to 1 m^2/s^2 dominated by events at 1 and 5 yesterday
w'w': ok, 0 to 0.2 m^2/s^2 (rather quiet in yesterday's haze)
u*: ok, 0 to 0.4 m/s
sigma_w/u*: 1 and 5 sometimes low

tc: ok, -16 to -6 degC now
tc'tc': ok < 0.7 degC^2; large nighttime values at 1,5,6
w'tc': ok, -0.03 to 0.05 m/s degC (-30 to +50 W/m^2)

kh2oV: all but 6 and 7 went to zero in last nights snow.  1 is recovering itself.  Max values yesterday 0.7--3.1 V.  Time to clean 2,3,4.
kh2o'kh2o': daytime very small, large values as sensors were covered last night
w'kh2o': small values during day; night values larger due to snow

All uplooking radiometers likely have snow on them this morning...

Rsw.in: ok, max only 400 W/m^2 yesterday
Rsw.out: ok, max 300 W/m^2 yesterday (snow cover at all sites)
albedo: ok, 0.7-0.9 at all sites, some with sun angle dependence
Rsw.dfs (1,7): up to 350 W/m^2 at Playa, 200 at River (large values could be real in yesterday's haze);
      SPN-1 agreed with shadowband yesterday, but differs today, likely due to snow.
Rsw.global (1,7): max 250 W/m^2 yesterday

Rlw.in: ok, 200-290 W/m^2
Rpile.in: ok, -90 to 0 W/m^2
Tcase.in: ok, -15 to +1 degC
Tdome.in (1,2,5): ok, -15 to +1 degC

Rlw.out: ok, 220-300 W/m^2
Rpile.out: ok, -50 to +10 W/m^2
Tcase.out: ok, -14 to +1 degC
Tdome.out (1,2,5): ok, -14 to +1 degC

soil.aux at E & W Slope (5,6)
Tsoil: ok, abc2, eslope5aux, river7 now frozen.  Some sites have very little variation (which is ok)
Gsoil: ok, with little surface forcing yesterday, fluxes nearly constant -- see digitizing resolution in data.
Qsoil: 7 (Eslope/River) to 29 (Hiland) %vol; expect values to go to 0 as soils freeze.
Cvsoil: ok, 5e5 (Wvalley, Eslope, River) to 1.8e6 (Playa, Wslope) J/(m^3 degK); strange abc jump yesterday.

site eslope5 visit, jan 8

Jan 8, On site SteveO and Ling D

1212: fix the broken fence around the solar panels, and brush the snow

1214: Clean 1-cm thick snow plug from bottom of TRH -- fan is running.

1216: Verified that GPS power jumpers are in place at all motes.

1228: clean up the radiometers

1230: lower the mast

1232: brush off sonic anemometer and krypton

1233: brush off pressure port

1235: clean up krypton

1238: kv measured 2.9v

1245: mast up

river7 site visit, jan 7

7 Jan: Dong, Oncley

6--14cm of hard-skinned snow; clear but hazy

1545: clean up radiometers

1550: mast down

1553,1557: clean krypton (twice, since only went from 0.5 to 1.1V the first time.  Was 1.8V the second time and 2.1 by the time we left.

1614: mast up

1630: shoot the boom of sonic (80.9deg)

wvally4 visit

7 Jan: Oncley&Dong

12-14cm of snow cover; hazy (near top of haze layer?)

1405-1407: stir ETI gauge (still some uncounted snow); waiting 10min for this (possibly) to be counted,

1417: remove ice from gauge

14:15 clean up radiometer

14:23 add .4 gallon eti fluid to the gauge, (ice plog is 8 inch much bigger than before ), green fluid was frozen, filmed the  side of the gauge.

Daily status 7 Jan

Daily status 7 Jan 2011
(sorry for my typo on the last status header I sent out and for no status messages the last 2 days)

Still in IOP5

Based on previous 48 hour WWW plots, "now" readings from this morning using S+, and other information
On site: Dong, Oncley

Weather: low overcast, light snow overnight and wet(!) roads.  Seems warmer than the -8C reported at most stations.

Summary: All sensor outages addressed, except for Playa1 aux soil site which was deliberately uninstalled
         Sun Photometer installed yesterday at River7
         All kryptons currently frost covered and are reading 0.
         All 10m kryptons have not been cleaned recently, but still have acceptable signal levels.  We would have
             to disrupt flux data during this (long) IOP5 to service these.

Vdsm: ok, lowest values 12.1-12.4V batteries fully charged yesterday
Vmote.rad: ok. lowest values12.6-13.0 V
Vmote.soil(aux): ok, lowest values 12.4-13.0.

Current Conditions:

P: ok 850 to 880 mb now, slightly falling.
Tbaro:
T: ok -12 to -7 degC now.  At least Playa1 get frost during the night
RH: 85 to 97% now
Q: ok
Rainr (3,4,6): Tips at Wslope6 yesterday was servicing.  Wvally4 tips probably not real or current.
Spd (1,6): <1 m/s now.  Seems low at Wslope6 starting 4 Jan ~2200.  Didn't notice anything obvious when shooting
        its boom angle yesterday.
Dir (1,6): ok, variable

csat diag: ok.  A few spikes overnight at Wslope6
samples.sonic: real-time data show dropped samples esp at night, maybe ok in archived data set
spd: ok, ~1 m/s now
dir: ok, variable across the network
u'u': ok, 0 to 0.7 m2/s2 dominated by events at abc2 and river7 yesterday
v'v': ok, 0 to 0.8 m2/s2 "
w'w': ok, 0 to 0.4 m2/s2 "
u*: ok, 0 to 0.4 m/s
sigma_w/u*: ok.

tc: ok, -12 to -8 now degC
tc'tc': ok < 1.8 degC2; huge nighttime values at wslope6, playa1, river7 -- are these real?
w'tc': ok, -0.03 to 0.08 m/s degC

kh2oV: all ~0 now with frost.  During day, 1 and 6 now >2V after recent cleanings.  Others 0.7-1.2V.
kh2o'kh2o': daytime very small
w'kh2o': small values during day; night values larger, likely frost related

Rsw.in: ok, daytime 300-600 W/m2 yesterday
Rsw.out: ok, daytime 250-400 W/m2 (snow cover at all sites)
albedo: ok, 0.6-0.9 at all sites, some with sun angle dependence
Rsw.dfs (1,7): up to 350 W/m2 at Playa, 70 at River (does Playa need to be adjusted?); SPN-1 now at River
     and agrees with shadowband
Rsw.global (1,7): up to 500 W/m2

Rlw.in: ok, 200-280 W/m2
Rpile.in: ok, -150 to 0 W/m2
Tcase.in: ok, -17 to +10 degC
Tdome.in (1,2,5): ok, -17 to +10 degC

Rlw.out: ok, 240-300 W/m2
Rpile.out: ok, -80 to +10 W/m2
Tcase.out: ok, -17 to +10 degC
Tdome.out (1,2,5): ok, -17 to +10 degC

soil.aux at E & W Slope (5,6)
Tsoil: ok, abc2, eslope5aux, river7 now frozen
Gsoil: ok, +4 (most) to 35 W/m2 (River)
Qsoil: 8 (Eslope/River) to 30 (Hiland) %vol; expect values to go to 0 as soils freeze.
Cvsoil: ok, 5e5 (Wvalley, Eslope, River) to 1.8e6 (Playa, Wslope) J/(m3 degK); strange abc jump yesterday.

wslope6 Site Visit

10:16am MST, 7 Jan 2011

Wslope site used for balloon launch observing upslope flow and sidewall radiation processes.

Site inspected as part of this. Riming observed on tower, and  radiometers.

Solar panels were clear and mostly dry.Rain gauge found to have slushy blocks in, but was mostly fluid. Images are available if necessary.

at 10:27am MST radiometers were cleaned with snow by Sebnastian Hoch. Sonic riming was knocked off by tapping the sonic boom. Krypton appeared ok, and data has been coming in for some time, so the krypton was not cleaned.

Rime ice still present on sonic socks, but transducers appear clear. Steve and Ling in the trailer can coinfirm this.

Day appears sunny, so remaining rime ice may melt during the day.

6 Jan Dong&Oncley

5--10cm of hard wind-packed snow

1140 install a new radiometer (twh: SPN1)

1138 clean up radiometers

1140 soil sample

1147 take pictures

boom angles: 86.6 (looking up) 85.6 (to tower near tripod)

TRH fan okay

6 Jan Oncley@Dong

About 10cm of snow on the ground

0953 klean krypton 1.0 -> 2.5V

1003 clean up radiometers

1012 soil samples (top 4cm is frozen, therefore samples represent depths closer to 0-4cm and 4-6cm)

1028 add Tsoil sensors

1030-1034 break ice in the gauge

1038 shoot the booms: sonic 79.0; prop 169.3

1038 take pictures

1046 add about 0.6galloons of ETI fluid 3 to gauge

out of fog

broke out of fog driving south on 172 at about 4000S at 09:17AM.

6 Jan Oncley&Dong

About half as much frost as yesterday -- say 5mm thick everywhere.  Visibility about 0.3mi in fog.

0830 brush off solar panels

0831 blow out snow from TRH (noisy fan)

0833 klean krypton (now 2.4V) and brush off sonic

0836-0844 clean radiometers

prop boom angle 174.2

Because of fog, sensors may ice up again...

Morning status 6 Jan

Status 1/6/2011: Oncley&Dong onsite.

Fog this morning, about 0.2mi visibility.  Frost on van windows at the Rez.

Stn1 krypton frosted up immediately in the evening last night.

Stn3 krypton also went to 0 early this morning

Stn4 krypton appears to be failing just now

Our plan:

Stn1 quick visit to defrost sensors and to shoot prop angle (might not "stick" since lots of fog still present)

Stn6 normal visit

Stn7 normal visit and add SPN-1 and klean krypton if time

Return to campus for 1:00 meeting

router power consumption

This probably is old news, but on the spare dsm (isfs8) with just barometer and gps inputs, the power consumption is:

12.5V @ 0.55A = 6.9W (router on with ssh connection, but no udp data)

12.5V @ 0.31A = 3.9W (router turned off by: viper_dio /dev/viper_dio0 7 0)

for a difference of 3.0W.

Eslope5 site visit Jan 5, 2011

on site 5,  (Jan 5, 1011,   2:45pm )SteveO, LingD, and JoeY

1> take soil samples (2:45PM)

2> decided not to change Tsoil sensor since top 1cm of ground is frozen -- thought I'd do more damage than good. (2:50pm)

3> cleaned radiometer (about 3:00PM)

4> boom angle 251.0 (5:10pm)

krypton status

SteveO and Ling measured through cellullar at playa1 ( Jan 5, 1011, 10:30am):

Site#      krypton-measurements:

1            2.4

2            .9

3            .9

4           1.3

5           1.0

6           .9

7           .6

Playa1 site visit Jan 5, 2011

On the site: SteveO and Ling  (Jan. 5, 1011, 8:40am - 10:10am)

Beautiful frosty conditions.  Half foot of snow, and ice/frost on tree, bushes, and instruments.

1> take pictures ( 8:40am This site is cold, covered with snow, significant different from others, it worthy of pictures taken)

2> clean up sonic and krypton sensors (8:46am)  Both were covered with hoar frost (both arms and over sonic transducers, over frame and krypton tubes).  Also the krypton tube faces had frost.

3> clean up radiometers (9am)

4> clean up TRH (9am)  I'm not positive, but I believe the fan started while we were there (had been stuck?).  I removed the tube and blew bits of snow out.  The SHT itself was relatively clear.

5> reinstalled the krypton electronics on its official bracket, though Joe's cable-tie mount was perfectly fine (reconnected at 9:33am)

6> clean up solar panel (9:35am)

7> take soil samples (9:50)

8> shoot the boom (10am), measured 77.3degrees

Soil sample processing

This will be my template for this round of soil sample processing.

A few notes:

- We forgot to pack the scale, so ours is being sent.  In the meantime, I've borrowed one (generously loaned by John H) with a resolution of 1g.  When our scale arrives, I'll make a few parallel measurements to compare the two scales.

- We couldn't find any good foil cups at the store today to bake the dirt in, so I've made some by bending aluminum foil.  Obviously, they won't tare as consistently as store-bought.

- The coring tool was set up with two 3cm rings on the top (used) followed by two 1cm rings on the bottom (ignored).  Thus, each 3cm sample volume was 3*pi*(5.31/2)^2=66.4cm^3.

Site/Time

Depth

ID

Tare(g)

Wet(g)

Dry(g)

Rho-dry (g/cm^3)

Qsoil (% mass)

Volume fraction (%)

Comments

 

 

 

 

 

 

 

(wet-dry)/wet

(wet-dry)*rho_water/vol

 

1

0-3

2-0

3.5

73

47.9

0.67

36

37

 

5Jan 0950

3-6

2-6

3.7

123

101.9

1.48

18

32

 

2

0-3

3-0

3.3

90

74

1.06

18

24

Soil damp&sticky

4Jan1410

3-6

3-6

3.6

129

121

1.77

6

12

Piece of brick

3

0-2.5

1-0

3.4

89#

68

1.17

25

38

Only ~2.5cm thick

3Jan1500

2.5-5.5

1-6

3.4

148#

124

1.82

17

36

 

4

0-3

2-0

3.5

152#

122

1.78

20

45

 

3Jan1636

3-6

2-6

3.7

137#

115

1.68

17

33

 

5

0-3

1-0

3.4

103

87.2

1.26

16

24

 

5Jan 1445

3-6

1-6

3.4

99

89.9

1.30

10

14

 

6

0-4

1-0

3.4

97.4*

61.4

0.66

38

41

Soil was frozen,

6Jan 1012

4-6

1-6

3.4

72.2

58.0

1.23

21

32

so odd depths (approx)

7

0-3

2-0

3.5

40.9*

23.75

0.30

46

26

 

6Jan 1140

3-6

2-6

3.7

77.4

59.4

0.84

24

27

 

# = weighed wet a day later (but kept cold to reduce water loss)

* = weight after a bit of time in the oven :( [forgot to write down first reading], water content will be a bit low, but dry density will be okay

Here is a quick comparison of the weights of various objects with the borrowed scale and ours.  The borrowed scale read within 1g, so I wouldn't change any of the above readings.

Ours (g)

Borrowed (g)

3.6

3

3.3

3

48.5

49

85.1

85

116.3

117

145.2

146

ROUND TWO:

Warmer temperatures have both melted snow (and there has been rain) to increase moisture and liquified existing moisture to allow the EC-5s to measure it.

Site/Time

Depth

ID

Tare (g)

Wet (g)

Dry (g)

Rho-dry

Qsoil

Volume Fraction

Comments

 

 

 

 

 

 

 

 

 

 

1

0-3

1-0

3.4

135.6

107.3

1.57

21

43

 

17 Jan 0835

3-6

1-6

3.5

163.3

138.4

2.03

16

37

May have been compressed

2

0-3

4-0

3.6

92.0

73.3

1.05

21

28

a bit frozen

18 Jan 0828

3-6

4-6

3.8

96.3

86.8

1.25

10

14

 

3

0-3

2-0

3.6

69.5

38.5

0.53

47

47

grass

17 Jan 1010

3-6

2-6

3.7

138.3

111.1

1.62

20

41

 

4

0-3

1-0

3.5

131.1

108.9

1.59

17

33

 

19 Jan 1140

3-6

1-6

3.5

132.1

111.3

1.62

16

31

 

5 main

0-3

3-0

3.3

113.1

82.8

1.20

28

46

water (frozen)

17 Jan 1120

3-6

3-6

3.6

112.1

89.5

1.29

21

34

 

5 aux

0-3

4-0

3.6

124.8

105.2

1.53

16

30

 

17 Jan 1132

3-6

4-6

3.8

120.1

104.8

1.52

13

23

 

6

0-3

5-0

3.8

110.2

86.8

1.25

21

35

 

19 Jan 1420

3-6

5-6

3.4

131.7

112.0

1.63

15

30

 

6 aux

0-3

3-0

3.4

139.5

113.5

1.66

19

39

 

19 Jan 1425

3-6

3-6

3.6

129.0

104.6

1.52

19

37

 

7

0-3

5-0

3.8

90.9

48.7

0.68

48

64

water (frozen)

17 Jan 1221

3-6

5-6

3.4

112.4

82.6

1.19

27

45

water

ROUND THREE:

John, Tom, and Steve S

We tared the scale prior to weighing the wet soil (net), weighed the dry soil + pan (total), then subtracted the pan wt to get the dry soil (net).

I mistakenly started weighing the soil in ounces, so we continued this in order to make a consistent table.

Ring diameter = 2 3/32" = 2.094" = 5.319 cm; volume = 66.66 cm^3;  1 oz = 28.35 gm

Qsoil = 1 - dry/wet; h2o volume fraction = (wet-dry)/(1 gm per cm^3)/sample-volume

Site

Time

Depth
(cm)

Tray ID

Tare
(oz)

Wet Wt
net (oz)

Dry Wt
total (oz)

Dry Wt
net (oz)

rho-dry
(gm/cm^3

Qsoil
(% mass)

h2o vol fraction
(% volume)

1

Jan 30 10:45

0-3

4-0

0.130

2.845

1.575

1.445

0.615

47

54

 

 

3-6

4-6

0.135

4.700

3.900

3.765

1.601

18

34

2

Jan 29 14:30

0-3

4-0

0.130

3.825

3.440

3.310

1.408

10

16

 

 

3-6

4-6

0.135

4.400

4.185

4.050

1.722

5

   9

3

Jan 28 14:30

0-3

3-0

0.120

3.195

3.120

3.000

1.276

2

   3

 

 

3-6

3-6

0.125

4.810

4.100

3.975

1.691

15

30

4

Jan 28 11:15

0-3

2-0

0.120

2.365

2.045

1.925

0.819

14

14

 

 

3-6

2-6

0.130

4.875

4.335

4.205

1.788

11

23

5 main

Jan 27 14:31

0-3

4-0

0.130

3.060

2.525

2.395

1.019

18

23

 

 

3-6

4-6

0.135

3.790

3.395

3.260

1.386

11

17

5 aux

Jan 27 14:55

0-3

5-0

0.140

3.625

3.130

2.990

1.272

14

21

 

 

3-6

5-6

0.120

2.495

2.390

2.270

0.965

4

   4

6 main

Jan 24 11:45

0-3

4-0

0.135

4.415

3.305

3.170

1.348

26

47

 

 

3-6

4-6

0.125

5.705

4.890

4.765

2.027

15

35

6 aux

Jan 24 12:15

0-3

2-0

0.135

3.420

2.715

2.580

1.097

21

30

 

 

3-6

2-6

0.130

4.775

4.094

3.964

1.686

15

29

7

Jan 27 16:18

0-3

1-0

0.125

3.115

2.200

2.075

0.882

31

39

 

 

3-6

1-6

0.125

3.635

2.785

2.660

1.131

24

36

P.S. (18 Apr 2011): All calculations seemed off in Round 3, reran and edited above entries.

ABC2 site visit Jan 4, 2011

on site: SteveO and Ling Jan 4, 2011

1> The primary purpose of this visit was to fix/replace the Qsoil sensor.  We briefly disconnected the wrong sensor when trying to figure out which cable was Qsoil.  As I started to unbury it from the mote end, the PIC enclosure pulled out of the ground, leaving the sensor and its cable still in place.  I exposed the cable connector, cleaned both ends, and reconnected the PIC.  Readings were then 22.6%, rather than 615% when we first looked at data.  Thus this was a simple connector problem -- the original sensor was never disturbed and the PIC didn't require new calibration coefficients.  I wrapped a bit of tape around the connector before reburying.  I'm really sorry if this was my fault during installation, though I noticed this cable was separate from the others, so perhaps someone else had replaced this sensor earlier?  Also, I think the connector was slightly deformed.  (3:57-4:07pm)

2> clean up the radiometers (4:09-4:15pm) Notice that the sensors are generally dirtier than other sites; recommend to clean up the sensor every visit.

3> take soil samples (4:07-4:15pm), marked as 1:0-3cm and 1:3-6cm (3-6cm sample may not be perfect,since a piece of brick was in the sample tube -- explaining why taking this core took a bit of effort!  I would still use this measurement as typical of the "soil".)  Soil was damp and sticky.

4> shoot the boom , measuring 80 degrees from the back so that it should be 80+180=260.  I simply tried to center mast in the parts of the sonic "claws" that I could see (partially blocked by cables from the sonic electronics box).  (4:25pm)

Hiland3 site visit Jan 4, 2011

On-site SteveO and Ling Jan 4, 2011

1> To test one theory of why Vdsm at this site decreases at a faster rate than the other sites, we replaced the charge controller with one of the new ones sent by Chris.  We turned off the dsm during this process (3:01pm to 3:10pm).  Batteries were at 14.6V and solar input was over 20V.

2> slightly stirred rain gauge, small ice chunks left over from yesterday are still floating in the "antifreeze" (3:11pm)

3> re-seated several connections to the router's DC-DC converter (3:05pm)

4> briefly inspected that radiometers were still clean

see subject.

Big flakes coming down at 7:05, stopped by 7:35, restarted 7:45.

Also at base trailer in the morning.

trailer systems

Joe had noted that the trailer computers appeared to be sick during the break, which he attributes to problems related to computer crashes in Boulder.  When we arrived this morning, we rebooted both "flux" (which had a blank screen that wouldn't react to keystrokes) and "pam" (I don't remember why I rebooted it).  "aster" seemed to be fine and once "flux" was up, S+ worked fine.  Ling did restart dsm_server on "flux" and then was able to run cockpit using localhost (I think) as the server.  We found that printing to "canon" on "flux" and "cannon@pam" on "pam" worked.

"flux" also was just fine when we returned from the field this evening (and I have logged into it from the motel), so we have not yet been able to duplicate the failure.  (If we do, I'll first try ssh from my Mac on the same LAN.)

However, S+ wouldn't work on "pam".  After some sleuthing, I did "exportfs" and then "/etc/init.d/nfs restart" as root on "flux" followed by "mount /net/local" as root on "pam".  (I'm not sure that "exportfs" was necessary.)  I'm closer and "pam" can now find "/usr/local/bin/Splus7" (it couldn't before), but I still get an execution permission error.  Obviously, I can (and did) ssh from pam to flux to run S+...

Wvally4 site visit Jan 4, 2011

On-site SteveO and Ling (Jan 4, 1011, 4:25pm)

1> clean all radiometers. same order and procedure as at hiland3.  Again, all domes looked pretty clean before and after, though I could imagine that there was a tiny frost film on the pyrgeometers. [my best guess from the data is that Rsw.in changed by 1 W/m2 out of 100 W/m2 and Rlw.in not at all.] (4:18PM)

2> replace barometer #2 with #10 (4:26pm), readings 869.56 (4:26pm)

3> take soil samples (4:36pm), marked as 2-0-3cm (full all the way to top) and 2-3-6cm on the sample boxes.  Again, were unable to process so far.

4> The precip gauge had about 10cm of power snow on top of a significant ice plug.  The top of the plug may be at the top of the fluid, but the powder definitely was above that.  I did some poking around to obtain the above information, but otherwise did nothing to the gauge, since it was clear at this site that precip had not yet been counted.  I like the idea of returning to this gauge and heating it, but am not yet convinced that the propane torch suggestion is the best (won't it will destroy the paint on the gauge?) and had not brought it along.  However, I don't yet have a better idea.  We do have extension cords that may be long enough to reach from the van to provide AC power, but the small inverter is only 150W and most sources of heat I can think of would be more than that.  SteveS had suggested talking to JohnH about the ETI gauges, which I have not yet done.  (4:53pm)

5> Shot the boom angle (again with the Datascope) and got 80.8 degrees. 

Hiland3 site visit Jan 3, 2010

On  site-Highland3, SteveO and Ling  (Jan 3, 1011, 2:20-3:40pm)

We have done the following:

1> cleaned all radiometer domes.  Decided simply to breath on the domes and wipe off the condensation.  Order was Rsw.in Rlw.in Rsw.out Rlw.out.  The sun was shining during this period, so the effect of cleaning should be measurable.  ?[PS: have looked at the data.  Haze made the radiation somewhat variable, so my best estimate is that cleaning changed Rsw.in by 2--10 W/m2 out of 380 W/m2 and Rlw.in not at all.  Not <too> bad.]  All domes looked pretty clean before and after.  (2:20pm)

2> As per the task list, disinterred (at a cemetary!) the single Tsoil to check/correct its orientation.  Found that it was nearly horizontal at about 2cm depth.  Reinstalled it at an angle that I eyeballed to put the sensor from 1--6cm depth.  Before doing this it read 0.3 C, after 0.81 C.  Note that these temperatures are slightly above freezing.  I noted a thin (~1cm) crust of frozen soil, but underneath definitely wasn't, which all is consistent.  I should also note that there was about 10cm of snow covering the whole area that I attempted to replace when I finished.  Finally, I note that we first unplugged the TP01 while determining which wire was which.  Thus, its values reset to 0, but later appeared to be okay.  (2:32PM)

3> take soil samples (3pm), marked the two sample boxes as 1-0-3cm (5mm shame of being full)and 1-3-6cm.  Unfortunately, there are no scales at the base to weigh them with!!  We'll have to get one sent out :(

4> shoot the boom angle from across the street (3:23pm), readings are 254.2 (Steve's) 254.7 (Ling's).  We did not touch settings on the DataScope or recalibrate it.  I assumed this had been done earlier for this project.

5> break the ice in the rain-gauge (3:32PM),  and take the ice out of the rain-gauge.  The gauge was on the entire time, so these data will have to be edited.  I judged that, since the top of the ~10cm-thick ice plug was just at the top of the antifreeze/oil, this mass had been recorded by the gauge and that it was okay to remove it from the gauge.  In the process, some of the antifreeze splashed on the side (inside) of the gauge, so I wouldn't be surprised by a tip or after this operation as it (slowly) slides back to rejoin the rest of the fluid.  I should point out that there was absolutely no precip today in the area we could observe, so zeroing out the day should be fine.

6> take pictures (3:37pm)

crontab removed

Following the wisdom of Boulder, I have just commented out the crontab entries that turn the router power off at night on hiland3.  Enjoy the extra data and enjoy looking at the power consumption tonight.

TWH (1/4/11):

Highland Vdsm 1/2/11 at 19:52:30 = 12.80 V (Router turned off overnight)

     lowest Vdsm 1/3/11 at 07:52:30 = 12.11 V

                                             Vdsm loss = 0.69 V

                                 1/3/11 19:52:30 = 12.82 V (Router left on overnight)

     lowest Vdsm 1/4/11 at 07:52:30 = 12.10 V

                                             Vdsm loss = 0.72 V

On Thursday afternoon, JohnM added crontab entries to power this station's router down between 02Z and 14Z to conserve power.  Our intent is to do the same to the other stations if this works.

On Thursday night, the crontab executed, but the router stayed up.  Later that night, I logged into isfs3 and executed the command manually, but the router still stayed up.  We thus concluded that there was a faulty connection or jumper controlling the power.

However, on Friday night the crontab executed and the router went down.  Unfortunately, playa1's rsync took much longer than expected, so the rsync from hiland3 didn't start until after 02Z and thus failed.  Having not seen data coming in, JohnH (and later DaveW) were tasked with rebooting this station to revive it on Saturday morning.

On Saturday afternoon, I changed the crontab to turn off the station at 03Z to give it more time.

On Saturday night, the crontab again executed and rsync worked.

Ling and I have discussed some options.  We have several ideas:

1. rsync should run in parallel on all stations, rather than serially to better define the time window and also allow us to turn off all stations by 01Z.

2. even better would be to have the rsync script (started on porter?) execute the command to turn off the router once the rsync has finished.  The only reason no to do this would be if we wanted staff to be able to log into stations after ~01Z.

3. to fill in the night data, another the rsync script should run in the morning as well.  The evening rsync script would run faster since only half of the data would have to be transferred.

It would be best if Gordon would make these changes, since we think that the rsync script runs under crontab on his account.

Highland battery voltage

From Dec 15 through Dec 19, Vdsm at Highland fell at a rate comparable to other sites.

On the night of Dec 20, Highland fell at a rate slightly faster than other sites.

On the night of Dec 21, Highland fell at a rate comparable to other sites, but cut out at 11.5 V

in the middle of the night.

On the nights of Dec 22 and 23, Highland cut out soon after sunset.

John replaced the Highland batteries on Dec 24 at 7:30 am. 

On the nights of Dec 26-27, Vdsm fell faster than other sites.

On the nights of Dec 28-30, Vdsm fell at a rate comparable to other sites.

On the nights of Jan 1-2 (with the router off), Vdsm fell much faster than at other sites.

Jan 2, ~14:45:  Joe tried cleaning the existing kh2o, SN 1395, with methanol but the voltage never exceeded 2 mv. 

Prior to cleaning, Joe noticed a smudge on the top window and the appearance of rust around the edges.

He replaced SN 1395 with SN 1390, which has an output of 2.7 V.   Unfortunately 1390 did not have the mounting

plate for the electronics in the case, so Joe mounted the electronics temporarily with cable ties.

9:30AM MST 1 Jan 2011

After cleaning in lake effect snow, solar panels were covered in snow, and there was no charging, so the station died. 

2 charged batteries were brought out from the base, and after clearing solar panels, batteries were replaced.

Station was running on departure. Horst was able to ssh into playa1 to confirm it was working.

Solar panels and radiometers cleared of up to 4 inches of accumulated snow. Downward looking SW radiometer had frost on it, the rest were clean after being brushed.

Approximately 9 total inches of snow were now at the site. Subtracting yesterday's 2.5, leaves an additional 6.5 inches of snow had fallen overnight.

1:20pm MST 31 Dec 2010

After battery replacement, hiland3 was not communicating, though it was recording data locally. 

(TWH: I don't think Highland's batteries were replaced.  John Horel recycled power this morning around 7 am,

which perversely stopped the station.)

Horst requested a power cycle of the DSM to restart the cellular modem. 

This was performed and at about 1:25 Horst was able to ssh into the station. 

river7 snow clearing

12:18pm MST 31 Dec 2010

Snow cleared from solar panels at river7. Solar panles had a lot of ice from melting snow falling down. 

Radiometers were clear on arrival. 

Conditions were sunny (scattered clouds) while maintenance was performed. 

river7 rad mote

12:15pm MST 31 Dec 2010

Spare rad mote 8 has been replaced with new rad mote 17 sent from Militzer.

the new mote 17 was able to communicate with radio, and radiation data is now transmitting. Horst confirmed ability to check data via radio.

Mote 8 contains original data on its microSD card, and must be downloaded. The mote is now in the base trailer.

wslope6 snow clearing

11:36am MST 31 Dec 2010

Cleared solar panlels, radiometers were clear on arrival.

Snow depth 4.5 inches with minor drifts around equipment. 

Sun was weakly coming out while mainteanance was performed. This site was just south of the very persistent lake effect band that was hitting playa1 and wvalley4

Playa1 Snow Clearing

10:35am MST 31-12-2010

Playa1 site visited, snowing heavily on arrival. Radiometers were clear on arrival.

Solar panels cleared, no ice was found.

2.5 inches of drifting snow on the ground as of time of visit. TRH at site sounds very rough.

Power cycled Highland at 7:00 AM. 4 inches of snow. Cleared off solar panels and radiometers.

Replaced batteries at West Valley at 8:15. 10 inches of snow.  Cleared off solar panels and radiometers. ETI precip gauge is filled with snow and not melting into antifreeze. Needs to have known amount of antifreeze added. UU has antifreeze at Mtn Met Lab.

Multiple snow depth samples averaged to 8.9 cm at Hiland3 at 1700 UTC 1 Jan 2011.

-Dave & Johanna Whiteman

Power recycled twice at Hiland3 at 1000 and 1005 MST (1700 and 1705 UTC) while on phone with Tom Horst. Recycles were unsuccessful at allowing Tom to ping the site, which had been power-recycled earlier this morning by John Hortel, also to no avail. The station is, nevertheless, running the fan and collecting data on-site. Tom suggests that there may be an intermittency in the control line at this site, which will be handled by the ISFS crew arriving in Salt Lake City tomorrow.

TWH: the power recycle was successful, since I was later able to ssh in and data is being transmitted.

abc2 cleaned 31-Dec 2010

8:23am MST 31 Dec 2010

Solar panels cleaned at abc2. Some ice present on panels

Broke ice U of U scraper on thin ice. 

Radiometer doms were found to be free of snow or ice.

Conditions at the site were various large deep frozen ponds, and 2.5 inches of snow

7:50am MST 31-Dec 2010

Hiland3 batteries replaced after dsm power cutoff at 2:30am MST on 31 December 2010. Replaced with 2 charged gel cell batteries

Old batteries retuned for charging

Cleaning:

Radiation domes found to be clear of ice and snow, 

Solar panels brushed off, snow covered, but no ice formation yet.

3.5 inches of snow was present at the site as of my visit

East slope Dec 31

Cleared off solar panels. Brushed off lightly pyranometer- pyrgeomter clear. 6 inches of snow on ground. Access to site now a long carry unless someone brings a shovel and clears snow berm far back in cemetery.

W Valley back on line

12/30/10 17:00:  John Horel found the West Valley station running.  He recycled power and I (TWH) was able to ping the station successfully.

John also cleaned snow off the solar panels.  The radiometers were still snow-free.

17:40: Vdsm = 11.8 V

          Vmote.rad = 12.5 V

          Vmote.soil = 12.8 V

Status Dec 29, 2010

Status evaluated from 36 hour plots on Dec 29 11:00-13:00
On site: no ISFS staff

Weather: no personal observations

Summary: no Tsoil at W Slope main
         anomolous P excursion at Playa
         * Playa TRH down since 14:00 Dec 28
         Riverton sonic spiking heavily (wet?)
         radiation mote not reporting at Riverton, but recording
         kh2o at Riverton wet(?);
         * Playa kh2o has low V since 12/19 06:20
         ABC Qsoil went walkabout 08:00 12/28 to 08:00 12/29

Vdsm: ok, 12-14.5 V, batteries appear to be fully charged yesterday
Vmote.rad: 12.5-18 V, Riverton not reporting
Vmote.soil(aux): ok, 12.3-14.5 V, Playa and W Valley spiking to 18.5 V

Current Conditions:

P: 820 to 850 mb, falling; anomolous 2 mb, 4 hour excursion at
                                                 Playa this morning
Tbaro: ok, 2 to 7 degC
T: 0 to 2 degC; Playa down since 14:00 Dec 28
RH: 60 to 100%; Playa down
Q: 4 to 5 gm/kg; Playa down
Rainr (3,4,6): ok,  3 to 15 mm/hr showers past 36 hours
Spd (1,6): ok, 12 m/s
Dir (1,6): ok, SSE

csat diag: large spiking at Riverton since 06:00 this morning
samples.sonic: losing samples at some sites since 17:00 Dec 28
spd: ok, 5 to 10 m/s
dir: ok, SSE to SSW
u'u': ok, 2 to 4 m^2/s^2
v'v': ok, 2 to 5 m^2/s^2
w'w': ok, 0.5 to 2 m^2/s^2
u*: ok, 0.4 to 1 m/s
sigma_w/u*: ok, ~1.3 at 1,2,5; 1.5 at 3,4; > 2 at 5

tc: ok, 0 to 5 degC
tc'tc': generally < 0.03 degC^2; E Slope high and variable
w'tc': ok, -0.03 to 0.01 m/s degC

kh2oV: 0.6 to 0.9 V at 2-6; 1 and 7 at a few mV
kh2o'kh2o': ok at 2-6; E Slope high and variable
w'kh2o': 0 to 0.03 m/s gm/m^3 at 2-6; 7 questionable

No radiation data transmission at Riverton

Rsw.in: ok, ~50 W/m^2 at 1-5; 100 W/m^2 at 6
Rsw.out: ok, < 10 W/m^2 at 1-4; 27 at 5; 72 at 6
albedo: ok, ~0.1 at 1-4; 0.5-0.7 at 5&6
Rsw.dfs (1,7): 40 W/m^2 at Playa
Rsw.global (1,7): 50 W/m^2 at Playa

Rlw.in: ok, ~320 W/m^2
Rpile.in: ok, -10 W/m2^
Tcase.in: ok, 0-3 degC
Tdome.in (1,2,5): ok, 2-3 degC

Rlw.out: ok, 315-330 W/m^2
Rpile.out: ok, -10 to 0 W/m^2
Tcase.out: ok, 0-3 degC
Tdome.out (1,2,5): ok, 2-3 degC

soil.aux at E & W Slope (5,6)
Tsoil: ok, -1 to 2 degC, coldest at surface, except ABC today
Gsoil: ok, -10 (abc) to 15 W/m^2 (playa)
Qsoil: 16 (Eslope) to 35 (Hiland) %vol,
       ABC went walkabout 08:00 12/28 to 08:00 12/29
Cvsoil: ok, 7e5 (Wvalley, Eslope) to 3e6 (abc) J/(m^3 degK)

Upon arrival at trailer this morning, pam and flux computers in the trailer were in the state that they were put into on 23 December when an error appears to have caused a failure in numerous parts of the data system.

pam, which is used for s-plus realtime analysis was operating, but flux had become unresponsive. The /usr/local mount on pam did not contain any updated netcdf files from the isfs sites, so realtime analysis was not working. It was advised to restart both pam and flux, starting with pam, and then flux.

This was done, and pam restarted fine, but when remounting, the files did not appear. Then flux was restarted (using the power button due to unresponsiveness) which came back and I was able to start up cockpit and observe that data was flowing properly.

However, when flux was restarted, pam crashed, and became unresponsive. It is apparent that pam mounts flux, and when flux dissapears, pam does not work. The result was a need to restart using the power button. This operation resulted in pam not remounting flux on startup, and therefore does not have access to netcdf data or s-plus. 

For future situations where flux is down, it is advised to shut down pam using the computer controls, then reboot flux, and then restart pam, so that it proceeds in mounting, as well as allows flux to reboot.

pam will need to be addressed in the near future, as it is not particularly useful at this time. 

Restarting flux has allowed Tom Horst to have access to base computers again, and netcdfs have resumed coming into the computer. Realtime analysis can be done on flux for now, and cockpit is working just fine. It is reported that Tom can no longer access flux. This will be addressed in the morning of the 28th.

Spare Battery Status

An update to the status of spare batteries:

There are 6 batteries in the mountain met building. as of 27/12/2010 5:30pm

1 is charged to 12.77 volts, another is charging overnight. The others will be charged tomorrow and wednesday. 

Impending precipitation events and cloudiness mean the ISFS stations will be likely under cloudy conditions from midday tomorrow for up to 84 hours, or more. Spare batteries are being charged to be available for replacement.

The charger that was bought by the Semmer/Stumpy crew indicated that the first battery charged was 'bad' and it is holding a charge of 12.77 volts, so hopefully the charger is incorrect.

12/27/2010 4:10pm MST

Mote power cycling from yesterday did not result in rad mote at river7 functioning.

river7 site was visited to test the sensors/mote and replace the mote if necessary.

Sensors were tested and the mote was found to not be operating. The problem was tentatively identified as a cracked solder on the oscillator for the real-time clock/calendar chip. This results in the 1 second green 'heartbeat' led not lighting. The mote in question is mote 17.

Thus the spare mote: 08 (Channel: B) was connected to the battery/sensors, and turned on, and its LEDs indicated that it was working normally. Mote 8 has been installed, and is locally collecting the data on it's microSD card. It is possible that we can transfer the xbee radio from mote 17 to mote 8, allowing it to communicate with the tower and transmit it's data. But for now, the station radio would have to be reprogramed to receive data from mote 8. Thus it is working in local storage mode only. The transfer of radios may take place tomorrow, and requires disassembly of both motes 17 and 8. 

John Militzer has also indicated that he can send a new mote and pre-programed radio from Boulder so that the radiation stand can be communicated with. This is another option.

It has been discussed that the priority is collecting the rad data, and it is not vital that this data be immediately available to project scientists. Though in the interest of ensuring data quality, it makes sense to have the data accessible.

Status, Dec 27, 2010

Status evaluated from 36 hour plots on Dec 27 15:30 - 17:00 MST
On site: no ISFS staff

Weather: precip, likely snow (T < 0 degC), yesterday afternoon and last night
         partly cloudy today (Rsw.global > Rsw.dfs)

Summary: no Tsoil at W Slope main
         radiation mote down at Riverton
         kh2o at Playa, Highland still wet

Vdsm: ok, 13.5-14.5 V
Vmote.rad: 13-13.5 V, Riverton down
Vmote.soil: ok, 12.7-13.7 V

Current Conditions:

P: ok, 848 to 877 mb, rising
Tbaro: ok, 5 to 7 degC
T: ok, 1 to 3 degC; below freezing last night; playa spiking
RH: ok, 73 to 90%; playa spiking last night and this morning
Q: ok, 3.5 to 4.5 gm/kg; playa spiking
Rainr (3,4,6): ok,  3 to 12 mm/hr; 13:30 MST 12/26 to 04:30 12/27
Spd (1,6): ok, W Slope calm, Playa 2 m/s
Dir (1,6): ok,

csat diag: ok, high spiking at Playa until 10 am today
spd: ok, up to 3 m/s, Riverton high
dir: ok, W-NW at most sites, S at Riverton, ABC, E at W Slope
u'u': ok, up to 1.5 m^2/s^2
v'v': ok, up to 1.5 m^2/s^2
w'w': ok, up to 0.6 m^2/s^2
u*: ok, up to 0.5 m/s
sigma_w/u*: ok, ~1.5

tc: ok, -5 to 5 degC
tc'tc': generally < 0.5 degC^2
w'tc': ok, up to +/- 0.05 m/s degC

kh2oV: 2,4,5,6,7 > 0.5V; 1,3 ~ 0.5 mV
kh2o'kh2o': questionable
w'kh2o': less than 0.01 m/s g/m^3, some questionable

Rsw.in: ok, up to 650 W/m^2 on 12/27, no radiation data at Riverton
Rsw.out: ok, up to 300 W/m^2 on 12/27
albedo: ok, 0.1 to 0.8; E Slope & Highland highest
Rsw.dfs (1,7): ok, up to 300 W/m^2 at Playa
Rsw.global (1,7): ok, up to 600 W/m^2 at Playa

Rlw.in: ok, ~290 W/m^2
Rpile.in: ok, -45 W/m2^
Tcase.in: ok, -8 to 9 deg C
Tdome.in (1,2,5): ok, appears to be some gradients at Playa

Rlw.out: ok, ~320 W/m^2
Rpile.out: ok, -10 W/m^2
Tcase.out: ok, -8  to 10 deg C
Tdome.out (1,2,5): ok, appears to be some gradients at Playa

soil.aux at E & W Slope (5,6)
Tsoil: ok, 0 to 8 C, coldest at surface, mostly above 0 deg C
Qsoil: 14 to 35 %vol, E Slope driest
Gsoil: -20 to 30 W/m^2, mostly > 0
Cvsoil: 6e5 to 2e6 J/(m^3 degK); Playa, ABC, W Slope high, W Valley low

RIVER 7 Rad Mote

26 Dec 2010 6:36-6:45pm MST

Visited Riverton 7 radiation mote after it failed to return values several days ago. Data resumed flowing on 26 Dec between 9Z and 20Z, without any manual input. 

Upon opening the mote, there were no status lights blinking. I power cycled the mote with the on/off switch, and the status lights indicated that it was restarting itself:
all 3 lights flashing rapidly, then off. 

Following restarting the yellow GPS light remained on constantly. I tried recycling again, and clearing off all snow and ice from the GPS antenna, but the result was the same.

The reset button was pressed, and the same outcome. 

Rad battery level was measured at 12.84V, so this does not appear to be a low battery issue. 

The station has been left in this status, with radiation data not reporting. This will be addressed in greater detail during daylight on Monday the 27th.

WSLOPE 6 Rad Mote

6:00 - 6:18pm MST 

Visited WSLOPE 6 to reset radiation stand mote, after data dropped out several days ago.

Upon arrival the mote was opened, and status lights appeared normal (green/red flashing indicating data collection and flow). Rad mote power cycled via on/off switch, lights returned to normal status pattern. Rad data flow appears to have resumed as of power cycle of rad mote. Station radio was not restarted, internally or manually. Mote battery measured at 12.58 volts upon visit. 

Station solar panels were covered with snow and ice, and were cleared. As well, upward facing rads had 1/3 inch of snow and ice built up on the domes. Loose snow removed, ice was not removed due to delicacy of instrument domes, and in order to avoid scratching.

Station reporting normally in online plots.

Highland at around 7:30 AM, East Slope a little after 8 AM.

Highland bare ground; East Slope 2 inches of crusty snow left.

Highland power cycle

Went out at 2 PM and cycled power at Highland site. While that solved the immediate comms problem, it is apparent that the battery voltage is dropping and likely not going to make it through the night. Will check in the morning and swap batteries if necessary. East Slope battery also dropping at a high rate tonight.

John Horel

Status Dec 23, 2010

Status evaluated from 36 hour plots on Dec 23 12:30-15:15
On site: no ISFS staff

Weather: precip last night, Rsw.in = 500 W/m^2 at W Slope

Summary: no Tsoil at W Slope main
         data gap last night from Playa, Highland, W Valley, E & W Slope
         no data from Highland; has not recovered from low power last night
         14:20 MST: John Horel resurrected Highland

Vdsm: ok, 12.9-13.8 V
Vmote.rad: 12.4-13.3V, E Slope lowest
Vmote.soil: 12.1-13.7V, Eslope struggling

Current Conditions:

P: ok, 845 to 875 mb
Tbaro: ok, 5 to 10 degC
T: ok, 1 to 4 degC; above freezing last night
RH: ok, 95 to 100%, E Slope > 100%
Q: ok, 5 to 5.4 gm/kg
Rainr (3,4,6): ok,  3 to 10 mm/hr yesterday evening
Spd (1,6): ok, W Slope calm, Playa 4 m/s
Dir (1,6): ok,

csat diag: ok, some spiking yesterday afternoon and evening
spd: ok, up to 4 m/s
dir: ok, NW at most sites, NE at Riverton
u'u': ok, generally < 0.5 (m/s)2
v'v': ok, generally < 0.7 (m/s)2
w'w': ok, generally < 0.2 (m/s)2
u*: ok, up to 0.4 m/s
sigma_w/u*: ok, ~1.3

tc: ok, 2-5 degC
tc'tc': generally < .2 degC2
w'tc': ok, up to 0.07 m/s degC, E Slope very small

kh2oV: down to a few mV's
kh2o'kh2o': unrealistic
w'kh2o': less than 0.03 m/s g/m^3, but mostly positive

Rsw.in: ok, up to 500 W/m2 on 12/23
Rsw.out: ok, up to 270 W/m2 on 12/22
albedo: ok, 0.1 to 0.4; E & W Slope highest
Rsw.dfs (1,7): ok, up to 270 W/m2 at Playa; 170 W/m^2 at Riverton
Rsw.global (1,7): ok, up to 300 W/m^2 at Playa, 100 W/m^2 at Riverton

Rlw.in: ok, 270 to 330 W/m^2
Rpile.in: ok, -50 to 0 W/m2
Tcase.in: ok, 2 to 7 deg C
Tdome.in (1,2,5): ok, appears to be some gradients at Playa

Rlw.out: ok, 320 to 360 W/m2
Rpile.out: ok, -20 to 20 W/m2
Tcase.out: ok, 2 to 6 deg C
Tdome.out (1,2,5): ok, appears to be some gradients at Playa

soil.aux at E & W Slope (5,6)
Tsoil: ok, 0 to 8 C,
Qsoil: 15 to 36 %vol
Gsoil: -60 to 8 W/m^2, Eslope > 0, others < 0 today
Cvsoil: 8e5 to 3e6 J/(m^3 degK)

STATUS Dec 22, 2010

Status evaluated from 36 hour plots on Dec 22 15:00-16:00 MST
On site: Steve S and Chris

Weather: wet snow/rain this morning

Summary: no Tsoil at W Slope main
station 3 & 5 down this morning, repaired
sonic heights completed today

Vdsm: ok, 11 - 13.7v battery problem at 2 resolved yesterday
Vmote.rad: ok, 12.3-13.4v
Vmote.soil: 12.0-13.4V

P: ok, 836 to 870 mb
Tbaro: ok, 0 to 11 degC
T: ok, -2 to 4 degC
RH: ok, 88 to 100%
Q: ok, 3.6 to 5.8 gm/kg
Rainr (3,4,6): ok,  snow/rain this morning
Spd (1,6): ok, up to 7 m/s
Dir (1,6): ok, currently N(site 1) and S (site 6)

csat diag: ok, some spiking
spd: ok, up to 5 m/s
dir: ok
u'u': ok, generally < 1 (m/s)^2
v'v': ok, generally < 1 (m/s)^2
w'w': ok, generally < 1 (m/s)^2
u*: ok, up to 0.5 m/s
sigma_w/u*: ok, ~1.3

tc: ok, -8 to 10 deg C, spiking today
tc'tc': generally < .5 degC^2, ditto on spikes
w'tc': ok, -0.1 to 0.05 m/s degC

kh2oV: ok, 0 to 1.5, all at zero thsi morning, cleaned 2 this afternoon
kh2o'kh2o': ok, generally < .01, spikes throughout the period
w'kh2o': ok, -.02 to .02, jumps >-.2 to >.2 (m/s g/m^3)

Rsw.in: ok, up to 230 W/m^2 on 12/22
Rsw.out: ok, up to 170 W/m^2 on 12/22, site 2 reading lower than others, this is a water site!
albedo: ok, .1 to 1, snow cover on sensors
Rsw.dfs (1,7): ok, up to 200 W/m^2 on 12/22
Rsw.global (1,7): ok, up to 210 W/m^2 on 12/22

Rlw.in: ok, 270 to 330 W/m^2
Rpile.in: ok, -50 to 0 W/m^2
Tcase.in: ok, -2 to 5 deg C
Tdome.in (1,2,5): ok, -1.5 to 4.8 deg C

Rlw.out: ok, 290 to 340 W/m^2
Rpile.out: ok, -25 to 5 W/m^2
Tcase.out: ok, -2 to 5 deg C
Tdome.out (1,2,5): ok, -1.5 to 4.5 deg C

soil.aux at E & W Slope (5,6)
Tsoil: ok, 0 to 5 C,
Qsoil: 15 to 36 %vol
Gsoil: -35 to 18 W/m^2, low value from site 2 which is turing into a lake!
Cvsoil: 8e5 to 3.4e6

ABC2 Sonic height

14:00 - 14:15 Dec 22

sonic height measurement from pivot point to top of boom - 8.79m

Cleaned krypton

   twh (3/2/11): high-rate data indicate kh2o cleaned at 14:17(!)

                           high-rate T-tc suggest mast down 14:07 - 14:19

                           high-rate u,w do not clearly show mast down, but u close to zero 14:09 - 14:19

13:38 - 13:50

sonic height - 3.26m

sonic boom 79.9 magnetic (boom pointing west)

12:30 - 12:45

Measured sonic height and boom angle.

height - 3.27m

boom - 78.0 (ss), 78.3(cg) magnetic (boom pointing west)

11:00 - 11:30

Station was down this morning. A power recycle started up the station.

Measurement of the sonic height from ground to pivot bolt is  85cm

Hiland3 DMS repiar

10:30 - 10:50

Reprogrammed router at base and installed. Would not work until a full power recycle on DSM.

Able to get into DSM via wifi and cell.

Router and cell units are #3

Hiland3 Visit

8:45 - 9:15am

Visited this site due to no transmission.  Found a fault light on charging system.  Unplugged everything from charging unit...this includes batteries.  It seemed to reset charger and dsm was back up.  The Verizon router was not syncing.  Could not talk to the dsm via wifi or cell modem.  Removed router and bringing back to base trailer to re-install settings.

Hiland3 & Eslope5

Hiland and Eslope off the air early this morning. Low battery voltage may be the reason.

Will take a set of fresh batteries to each site.

13:30 - 14:30, Dec 21

Replaced Tsoil probe.

Lowered mast to clean krypton, clean solar, and clean radiometer

trailer test: brought probe back to trailer and did a visual check.

There was no sign on moisture in the connector. Did a simple ice

bath test and the probe was reading high. Will take back to Boulder.

   twh (3/1/11): high-rate data indicate kh2o cleaned at 14:03

                           high-rate u,w suggest mast down 13:59 - 14:05, but light winds make this ambiguous

          (3/3/11): high-rate pressure data confirm 13:59 - 14:05

ABC2 New router

12:00-12:15

Replace router with a new unit with the correct configuration.

This was the original router at ABC which was removed earlier today

router

ABC2 DSM power issues

10:30 - 11:00

Since power was still low at 2 decided to change out the DSM. Before making the change we use

the meter to verify volatge readings. The result was we found a loose connector on the main

power cable going from the power switch to the power panel board. Everything looks ok now.

We also changed the the router since we could not communicate withe DSM via WIFI or cell.

According to Gordon we still need to make another change in the router firmware.

Snow removal at solar panels and radiometers.

Sonic Heights

Sonic Location

Height from Ground to mast bolt (cm)

Height of sonic from mast bolt (m)

Total height from ground to sonic

playa1

 

 

3.26

abc2

71.7

8.79

9.507

hiland3

68.9

8.79

9.479

wvally4

75

8.79

9.54

eslope5

85

8.79

9.64

wslope6

 

 

3.27

river7

68.6

8.79

9.476

wvally4 Visit

12:05 - 12:30pm

Sonic Angle = 85.8deg

Connected to ETI and noticed the temperature compensation was turned off.  Turned compensation on.

Cleaned off solar panels.

hiland3 Visit

11:16 - 11:36am

Mast height(sonic) = 68.9cm

Sonic Angle ?= 253.5deg.

Looked at Tsoil.  Seems to be reading high.  Noticed ground was snowy except over soil probes.  Everything seemed fine.  Will ask Joe to bury new probe.

Cleaned off solar panels and radiometers.  Covered in snow.

Battery Charging

I've put a battery charger in the Met Shed behind trailer.  The batteries are on the floor by the power panel and charger is on the desk.  There is currently six spare, charged batteries.

ABC2 Battery Swap

8:00 - 9:00am

Removed both batteries at site with new/charged ones.  Will be charging batteries in Met Shed behind trailer.

hiland3 Tsoil

The Tsoil at Hiland jumped ~10C at about 8:00 this morning. We will investigate.

STATUS Dec 20,2010

Status evaluated from 36 hour plots on Dec 20 08:00-9:00 MST
On site: Steve S and Chris

Summary: no Tsoil at W Slope main

Vdsm: ok, 11 - 13.5v, replace batteries at 2 this morning
Vmote.rad: ok, 12.4-13.1v
Vmote.soil: 12.1-13.2V

P: ok, 825 to 865 mb
Tbaro: ok, 1 to 13 degC
T: ok, -1 to 12 degC
RH: ok, 55 to 100%
Q: ok, 3.8 to 7.3 gm/kg
Rainr (3,4,6): ok, getting snow/rain this morning
Spd (1,6): ok, up to 12 m/s
Dir (1,6): ok, currently S

csat diag: ok, some spiking this morning
spd: ok, up to 10 m/s, spikes up to 30 m/s this morning
dir: ok
u'u': ok, generally < 4 (m/s)^2
v'v': ok, generally < 4 (m/s)^2
w'w': ok, generally < 3 (m/s)^2
u*: ok, up to 1 m/s
sigma_w/u*: ok, ~1.3

tc: ok, 0 to 10 deg C, spikes this morning
tc'tc': generally < 1 degC^2, ditto on spikes
w'tc': ok, -0.1 to 0.0 m/s degC

kh2oV: ok, 0 to 1.7, current values 0 due to snow/rain
kh2o'kh2o': ok, generally < 0.03, spikes throughout the period
w'kh2o': ok, -.01 to .03, jumps >-.2 to >.2 (m/s g/m^3)

Rsw.in: ok, up to 180 W/m^2 on 12/19
Rsw.out: ok, up to 30 W/m^2 on 12/19, negative values <-5 during night time
albedo: ok, .1 to .18
Rsw.dfs (1,7): ok, up to 150 W/m^2 on 12/19
Rsw.global (1,7): ok, up to 150 W/m^2 on 12/19

Rlw.in: ok, 260 to 350 W/m^2
Rpile.in: ok, -60 to 10 W/m^2
Tcase.in: ok, -1 to 12 deg C
Tdome.in (1,2,5): ok, -1 to 11.5 deg C

Rlw.out: ok, 310 to 350 W/m^2
Rpile.out: ok, -35 to 5 W/m^2
Tcase.out: ok, -1 to 12 deg C
Tdome.out (1,2,5): ok, -.05 to 12 deg C

soil.aux at E & W Slope (5,6)
Tsoil: ok, -0.0 to 6 C, sensor at 3 is questionable this morning
Qsoil: 15 to 36 %vol
Gsoil: -40 to 20 W/m^2, spiking this morning
Cvsoil: 5e5 to 3.8e6

ABC2 DSM battery

Went by ABC this morning to replace the DSM batteries. We had one fresh one. Chris is going back with another one.

River7 RAD

15:15 - 15:30

After going back to the base to work on the pyg.in, discovered we grabbed the wrong sensor! Spent time in the lab to get the test mote running and are now back at 7.

Tested pyg.in with the test mote. Did not work via I2C.  Did test on sensor with serial cable and it worked. Tried I2C again and it worked. Chris looked

at connector and there appeared to signs of corrosion.

Moved sensor back out to darkhorse and is working ok. The Vmote measurement is now working.

wvally4 Visit

1:00 - 1:20pm

Tried to talk to ETI using SS computer.  Didn't work, was not able to send commands to ETI. Did receive output message.

May need special comms software provided by ETI.

Sonic height = 75cm

Sonic boom angle = 80.3deg(Semmer)

11:30 - 11:50

Rad mote: tried reset button and cycled power, GPS and PYG.IN still bad. Pulled power shunt to GPS.

Mote appears to be using the local RTC now. PYG.IN still down.

Removed PYG.IN and turned on GPS.  Will diagnose PYG in base trlr.  Also not seeing Vrad. 

Measuring the heights of sonics by measuring gnd to mast bolt.  All sonics at the 10m height are mounted at same position and will be added to this measurement.  I measured 68.6cm.

81.6degs for SS CSAT3 angle.

River7 Rad mote

vmote and pyg.in from River7 not coming in. Further investigations show the mote is having trouble with the GPS.

will investigate

STATUS Dec 19,2010

Status evaluated from 36 hour plots on Dec 19 08:00-9:00 MST
On site: Steve S and Chris

Summary: no Tsoil at W Slope main

Vdsm: ok, 11.3-14.5v, battery at 2 still low
Vmote.rad: ok,12.6-14v, battery at 7 gone
Vmote.soil: 12.2-14.3 V

P: ok, 833 to 868 mb
Tbaro: ok, -4 to 11 degC
T: ok, -5 to 9 degC
RH: ok, 55 to 100%
Q: ok, 3 to 6 gm/kg
Rainr (3,4,6): ok, getting rain this morning,
Spd (1,6): ok, up to 12 m/s
Dir (1,6): ok, currently S

csat diag: ok, some spiking at 3,5, and 6
spd: ok, up to 10 m/s
dir: ok
u'u': ok, generally < 2 (m/s)^2
v'v': ok, generally < 3 (m/s)^2
w'w': ok, generally < 2 (m/s)^2
u*: ok, up to 1 m/s
sigma_w/u*: ok, ~1.5

tc: ok, -5 to 10 deg C
tc'tc': generally < 1 degC^2, spiking above 4 degC^2 this morning
w'tc': ok, -0.1 to 0.02 m/s degC

kh2oV: ok, 0 to 2.4 V, current values 0 due to heavy rain
kh2o'kh2o': ok, generally < 0.02, spikes above .2 (g/m^3)^2
w'kh2o': ok, -.01 to .03, jumps from -.1 to >.2 (m/s g/m^3)

Rsw.in: ok, up to 270 W/m^2 on 12/18
Rsw.out: ok, up to 180 W/m^2 on 12/18
albedo: ok, 0.1 to 1.0
Rsw.dfs (1,7): ok, up to 230 W/m^2 on 12/18
Rsw.global (1,7): ok, up to 240 W/m^2 on 12/18

Rlw.in: ok, 220 to 340 W/m^2, down at 7
Rpile.in: ok, -80 to 10 W/m^2
Tcase.in: ok, -6 to 9 deg C
Tdome.in (1,2,5): ok, -6 to 9 deg C

Rlw.out: ok, 275 to 345 W/m^2
Rpile.out: ok, -45 to 5 W/m^2
Tcase.out: ok, -6 to 9 deg C
Tdome.out (1,2,5): ok, -6 to 9 deg C

soil.aux at E & W Slope (5,6)
Tsoil: ok, -0.5 to 4.8 C, missing W Slope main
Qsoil: 12 to 36 %vol
Gsoil: -30 to 20 W/m^2
Cvsoil: 9e5 to 3.5e6

11:00 - 11:30 Dec 18

Cleaned krypton and radiometers. Krypton at 2v.

sonic 76.9 magnetic (boom pointing west)

prop 171.7 magnetic

added power jumper for GPS at soil mote

STATUS Dec 18, 2010

Status evaluated from 36 hour plots on Dec 18 08:00-9:00 MST
On site: Steve S and Chris

Summary: no Tsoil at W Slope main

Vdsm: ok, 11.5-14.8v, battery at 2 still low
Vmote.rad: ok,12.8-15v, peaking to 18v
Vmote.soil: 12.3-14.5 V, peeking to 18v

P: ok, 836 to 871 mb
Tbaro: ok, -6 to 9 degC
T: ok, -12 to 1.5 degC
RH: ok, 45 to 97%
Q: ok, 1.5 to 4.5 gm/kg
Rainr (3,4,6): ok, rain tips from 4 this morning maybe good due snows
Spd (1,6): ok, up to 8 m/s
Dir (1,6): ok, currently S

csat diag: ok
spd: ok, up to 8 m/s
dir: ok
u'u': ok, generally < 1 (m/s)^2
v'v': ok, generally < 1 (m/s)^2
w'w': ok, generally < .3 (m/s)^2, 0.5 this morning at 7
u*: ok, up to 0.8 m/s
sigma_w/u*: ok, ~1.5

tc: ok, -11.8 to 1.8 deg C
tc'tc': generally < 0.1 degC^2
w'tc': ok, -0.1 to 0.08 m/s degC

kh2oV: ok, 0 to 3.0 V, current values 0 to 1.3
kh2o'kh2o': ok, generally < 0.02, spikes up to 0.15 (g/m^3)^2
w'kh2o': ok, -.01 to .02, jumps from -.1 to 1.5 (m/s g/m^3)

Rsw.in: ok, up to 420 W/m^2 on 12/17
Rsw.out: ok, up to 290 W/m^2 on 12/17
albedo: ok, 0.2 to 0.8
Rsw.dfs (1,7): ok, up to 270 W/m^2 on 12/17
Rsw.global (1,7): ok, up to 400 W/m^2 on 12/17

Rlw.in: ok, 190 to 320 W/m^2
Rpile.in: ok, -120 to 0 W/m^2
Tcase.in: ok, -12 to 8 deg C
Tdome.in (1,2,5): ok, -8 to 8 deg C

Rlw.out: ok, 240 to 320 W/m^2
Rpile.out: ok, -45 to 5 W/m^2
Tcase.out: ok, -12 to 8 deg C
Tdome.out (1,2,5): ok, -8 to 8 deg C

soil.aux at E & W Slope (5,6)
Tsoil: ok, -1.1 to 2.8 C, missing W Slope main
Qsoil: 13 to 33 %vol
Gsoil: 6 to 24 W/m^2
Cvsoil: 5e5 to 2.3e6

15:35  -  The rad mote had locked up this morning. Chris checked the leds and power and all looked ok. He then re-cycled power and the mote cmae back to life.

All is well!

precip data, wvally4

Some comments on the precip data and wvally4.

The PIC processor on the ETI gauges generates a tip count as shown below:

data_dump -i 4,80 -A isff_20101216.nc
2010 12 16 05:42:25.8211    5.04       9 ETI8 0\r\n
2010 12 16 05:42:30.8613    5.04       9 ETI8 0\r\n
2010 12 16 05:42:35.9015    5.04       9 ETI8 1\r\n
2010 12 16 05:42:40.9424   5.041       9 ETI8 0\r\n
2010 12 16 05:42:45.9823    5.04       9 ETI8 0\r\n
2010 12 16 05:42:51.0220    5.04       9 ETI8 0\r\n

The above snippet shows what appears to be a false tip at Dec 16 05:42 UTC, (Dec 15 22:42 MST). No other stations reported precip, and the longwave in (Rlw.in) doesn't indicate the presence of clouds.

This station had about 17 false tips over that night of Dec 15/16, and a similar looking set of false tips the next night, after the station visit of Dec 16.

Doing a quick look back through the plots shows similar false-looking tips every night between Nov 23 and Dec 1 with the exception of the nights of Nov 27 and 28. There are a few false-looking tips during the day, but most of them are at night. Typically the tips occurred throughout the night, and were not an isolated incident which one might expect from a prankster.

I think the false tips are correlated with temperature, they seem to occur on cold nights, when the temp is below -6C.

A tip is .01 inch of precip in 5 seconds. In the example above, the ETI at station 4 is reporting every 5.04 seconds. Using a value of 5 seconds is in error by less than 1 part in 100.

NIDAS converts .01 inch / 5 seconds to a rate, called Rainr, in mm/hr by multiplying the tip value by 182.88, so that one tip is a momentary rate of 183 mm/hr. 60 (or 59) of these 5 second points are averaged over 5 minutes. So one tip in a 5 minute period will result in an averaged rate of 182.88/60=3.05 mm/hr or 182.88/59=3.10 mm/hr depending on how many samples were received from the ETI in the 5 minute period, which is why there is a little jitter in the value of one tip on the rain plot.

STATUS Dec 17, 2010

Status evaluated from 36 hour plots on Dec 17 08:00-10:00 MST
On site: Steve S and Chris

Summary: no Tsoil at W Slope main

Vdsm: ok, 11.5-14.8v, battery at 2 ~.8v below others
Vmote.rad: ok,12.3-15v, peaking to 18v
Vmote.soil: 12.2-14.8 V, peeking to 18v

P: ok, 841 to 873 mb
Tbaro: ok, -10 to 17 degC
T: ok, -12 to -.5 degC
RH: ok, 48 to 97%, oscillations at 2,3,5
Q: ok, 1.5 to 3.1 gm/kg
Rainr (3,4,6): ok, false tips at 4 early morning
Spd (1,6): ok, up to 5 m/s, thresholda t 6 looking better
Dir (1,6): ok, currently SE

csat diag: ok; error from 2 when we visited site yesterday
spd: ok, up to 5.5 m/s
dir: ok
u'u': ok, generally < 1 (m/s)^2, high values at 3 this morning (2-3 m/s)
v'v': ok, generally < 1 (m/s)^2, ditto at 3
w'w': ok, generally < .5 (m/s)^2, ditto at 3
u*: ok, up to 0.8 m/s
sigma_w/u*: ok, ~1.5

tc: ok, -11 to -1 deg C
tc'tc': generally < 0.03 degC^2
w'tc': ok, -0.1 to 0.05 m/s degC

kh2oV: ok, 1.6 to 3.0 V, current values
kh2o'kh2o': ok, generally < 0.02, spikes up to 0.19 (g/m^3)^2
w'kh2o': ok, -.01 to .02, jumps from -.03 to 0.5 (m/s g/m^3)

Rsw.in: ok, up to 680 W/m^2 on 12/16
Rsw.out: ok, up to 370 W/m^2 on 12/16
albedo: ok, 0.2 to 1.0 due to frost on the domes?
Rsw.dfs (1,7): ok, up to 100 W/m^2 on 12/16
Rsw.global (1,7): ok, up to 480 W/m^2 on 12/16

Rlw.in: ok, 190 to 260 W/m^2
Rpile.in: ok, -160 to -10 W/m^2
Tcase.in: ok, -12 to 8 deg C
Tdome.in (1,2,5): ok, -12 to 8 deg C

Rlw.out: ok, 240 to 310 W/m^2
Rpile.out: ok, -70 to 5 W/m^2
Tcase.out: ok, -12 to 10 deg C
Tdome.out (1,2,5): ok, -12 to 10 deg C

soil.aux at E & W Slope (5,6)
Tsoil: ok, -1.2 to 3.8 C, missing W Slope main
Qsoil: 13 to 34 %vol
Gsoil: 7 to 31 W/m^2
Cvsoil: 5e5 to 3.2e6

11:50 - 12:05

Cleaned frost from radiometer domes.

sonic angle - 86.7 magnetic (sonic boom pointing west)

Checked rain gauge, solution was slushy. Chris broke through slush so there is bound to be a lot of false readings at this time.

10:45 - 11:10, Dec 16

Cleaned krypton, v = 2.4. Cleaned up looking domes, frost built up on Rsw.in. Rlw.in looked ok.

Since the battery acted up the other day we decided to move the DSM internal power cable from the power panel board to a serial panel board.

If we continue to have problems with power, next step will be to either replace the power cable or the stack power supply.

new angle on boom, 249.5 magnetic (boom pointing east)

   twh (3/1/11): high-rate u,w suggests mast down 10:58 - 11:09

                           high-rate data indicates kh2o cleaned at 11:01

           (3/3/11): high-rate u and p indicate mast down 10:59 - 11:08

STATUS Dec 16, 2010

Status evaluated from 36 hour plots on Dec 16 08:00-10:00 MST
On site: Steve S and Chris

Summary: no Tsoil at W Slope main

Vdsm: ok, 11.8-14.5v
Vmote.rad: ok,12.3-15v, peaking to 18v, site 2 looking good
Vmote.soil: 12.2-14.5 V, peeking to 18v

P: ok, 833 to 875 mb
Tbaro: ok, -10 to 18 degC
T: ok, -10 to 7 degC
RH: ok, 48 to 100%
Q: ok, 2.2 to 6.0 gm/kg
Rainr (3,4,6): ok, false tips at 4 early morning
Spd (1,6): ok, up to 12 m/s, 6 has high threshold
Dir (1,6): ok, currently W (6) and E (1)

csat diag: ok; no errors from yesterday
spd: ok, up to 10 m/s
dir: ok
u'u': ok, generally < 2.5 (m/s)^2
v'v': ok, generally < 2.0 (m/s)^2
w'w': ok, generally < 1.0 (m/s)^2
u*: ok, up to 0.8 m/s
sigma_w/u*: ok, ~1.5

tc: ok, -10 to 7 deg C
tc'tc': generally < 0.05 degC^2, high during snow event
w'tc': ok, -0.07 to 0.05 m/s degC

kh2oV: ok, 0.0 to 3.2 V, most recovered from snow, 2 low at .6
kh2o'kh2o': ok, generally < 0.03, spikes up to 0.2 (g/m^3)^2
w'kh2o': ok, currently up to 0.5 (m/s g/m^3)

Rsw.in: ok, up to 600 W/m^2 on 12/15
Rsw.out: ok, up to 500 W/m^2 on 12/15
albedo: ok, 0.15 to 1.0 due to snow
Rsw.dfs (1,7): ok, up to 300 W/m^2 on 12/15
Rsw.global (1,7): ok, up to 500 W/m^2 on 12/15

Rlw.in: ok, 200 to 330 W/m^2, 1 reading high compareed to others this morning
Rpile.in: ok, -160 to 0 W/m^2
Tcase.in: ok, -12 to 13 deg C
Tdome.in (1,2,5): ok, -12 to 13 deg C

Rlw.out: ok, 300 to 380 W/m^2
Rpile.out: ok, -40 to 8 W/m^2
Tcase.out: ok, -12 to 13 deg C
Tdome.out (1,2,5): ok, -3 to 15 deg C

soil.aux at E & W Slope (5,6)
Tsoil: missing W Slope main
Qsoil: 13 to 33 %vol
Gsoil: -30 to 40 W/m^2
Cvsoil: 5e5 to 4e6

15:15:- 15:30

Cleaned radiometer domes, there was snow accumulation.

sonic angle - 248.7 magnetic (sonic boom pointing east)

Wslope6 Prop issue

It appears the prop at 6 has a threshold problem. Compared to the sonic, the prop halts at speeds around 1 m/s.

Site 1: 026

Site 2: 021

Site 3: 020

Site 4: 019

Site 5: 018

Site 6: 024

Site 7: 017

STATUS Dec 15, 2010

Status evaluated from 36 hour plots on Dec 15 08:30-10:00 MST
On site: Steve S and Chris

Summary: no Tsoil at W Slope main

Vdsm: ok, 10.9-14.3 v, 2 @ 10.9v
Vmote.rad: ok,12.5-14.5v
Vmote.soil: 12.2-13.9 V

P: ok, 833 to 870 mb
Tbaro: ok, -1 to 15 degC
T: ok, -3 to 15 degC
RH: ok, 20 to 100%
Q: ok, 2.2 to 6.3 gm/kg
Rainr (3,5,6): tips coming in since snow event
Spd (1,6): ok, up to 10 m/s, a few missing events at 6
Dir (1,6): ok, currently ~NW

csat diag: ok; some errors this morning, most at 2
spd: ok, up to 10 m/s, 2 intermittant at times
dir: ok
u'u': ok, generally < 2.5 (m/s)^2
v'v': ok, generally < 4.0 (m/s)^2
w'w': ok, generally < 1.0 (m/s)^2
u*: ok, up to 0.8 m/s
sigma_w/u*: ok, ~1.7

tc: ok, 3 to 14 deg C
tc'tc': generally < 0.3 degC^2
w'tc': ok, -0.1 to 0.1 m/s degC, site 2 outages this morning

kh2oV: ok, 0.0 to 3.5 V, snow event shut most down except 6
kh2o'kh2o': ok, generally < 0.03, spikes up to 0.2 (g/m^3)^2
w'kh2o': ok, currently up to 0.1 (m/s g/m^3)

Rsw.in: ok, up to 380 W/m^2 on 12/14
Rsw.out: ok, up to 80 W/m^2 on 12/14, a few negatives at night
albedo: ok, 0.1 t0 0.2, up to .8 this morning with the snow
Rsw.dfs (1,7): ok, up to 200 W/m^2 on 12/14
Rsw.global (1,7): ok, up to 350 W/m^2 on 12/14

Rlw.in: ok, 240 to 340 W/m^2
Rpile.in: ok, -90 to 0 W/m^2
Tcase.in: ok, -3 to 15 deg C
Tdome.in (1,2,5): ok, -3 to 15 deg C

Rlw.out: ok, 300 to 380 W/m^2
Rpile.out: ok, -40 to 8 W/m^2
Tcase.out: ok, -3 to 15 deg C
Tdome.out (1,2,5): ok, -3 to 15 deg C

soil.aux at E & W Slope (5,6)
Tsoil: missing W Slope main
Qsoil: 14 to 33 %vol
Gsoil: -30 to 40 W/m^2
Cvsoil: 5e5 to 2.5e6

Wslope6 Prop Issue

16:10 -  Visit to 6 due to intermittent outages. Prop started working again about 15:45.

Checked connectors inside and outside DSM, all looked ok.

Cleaned krypton since it is at the 3m height.

From http://www.ngdc.noaa.gov/geomagmodels/struts/calcDeclination, the magnetic declinations of the sites varied from 12 11' to 12 17' east. To correct the magnetic azimuths of the sonics and props we'll use a declination value of 12.2 degrees for all sites.

To rotate the wind vectors, NIDAS uses the pointing angle of the sonic +V axis (Vazimuth), which is pointing azimuth of the sonic boom + 90, or the sonic +U axis - 90.

The following are from logbook entries of the sonic azimuths.

Note that all measured azimuths are in degrees magnetic, with the declination set to 0 on the compass or datascope, to reduce untraceable mental mistakes.

site

log date

pointing

sonic boom azimuth, magnetic

boom azimuth, true (mag+12.2)

Vazimuth, true (boom+90)

notes

playa1

12/18

west

76.9 + 180 = 256.9

269.1

359.1

 

playa1

1/5

"

77.3+180 = 257.3

 

 

 

abc2

12/16

east

249.5 - 180 = 69.5

81.7

171.7

 

abc2

1/4

"

80

 

 

shot from the back (centered mast in claws), so hard to see; still thought it was okay

hiland3

12/14

east

249.5 - 180 = 69.5

81.7

171.7

 

hiland3

12/20

"

253.5 - 180 = 73.5

85.7

175.7

 

hiland3

1/3

"

254.2 - 180 = 74.2

 

 

shot from across the street

wvally4

12/16

west

86.7 + 180 = 266.7

278.9

8.9

 

wvally4

12/19

"

80.3 + 180 = 260.3

272.5

2.5

mast was not lowered between 12/16 and 12/19

wvally4

12/20

"

85.8 + 180 = 265.8

278.0

8.0

 

wvally4

1/3

"

80.8 +180 = 260.8

 

 

 

eslope5

12/15

east

248.7 - 180 = 68.7

80.9

170.9

 

eslope5

1/5

"

251.1 - 180 = 71.1

 

 

 

wslope6

12/13

west

77.8 + 180 = 257.8

270

0.0

 

wslope6

1/6

"

79.3

 

 

 

river7

12/13

west

85.5 + 180 = 265.5

277.7

7.7

 

Props

site

log date

pointing

prop azimuth, magnetic

prop az, true (mag+12.2)

notes

playa1

11/20

north

 

1.0

 

playa1

12/18

north

171.7 + 180 = 351.7

3.9

 

playa1

1/6

"

174.2+180=354.2

6.4

 

wslope6

11/20

north

 

1.7

should this have been -1.7, which agrees with 12/13?

wslope6

12/13

north

346.1

358.3

 

wslope6

1/6

"

169.3+180=349.3

1.5

 

Dec 14, 12:15 - 12:50

Cleaned krypton and radiometer domes.

   twh (3/1/11): high-rate kh2o shows kh2o cleaning at 12:27

                           high-rate w suggests mast down 12:25 - 12:30

          (3/3/11): high-rate u, T-tc, p  consistent with 12:25-12:30

sonic angle 249.5 magnetic (sonic pointing east)

fixed battery mount for Tsoil unit

Dec 14, 10:30 - 11:30:

Cleaned the krypton and radiometer domes, added drip loop. Made change to Tsoil mote battery box for easy removal

Sonic boom: 83.4 magnetic (sonic pointing east). Chris had to make a measurement from the west due to the fence east

of the mast. The measurement should be used with caution since it is not possible to see the boom from the west. We

will try to get on the other side of the fence the next time we visit ABC.

   twh (3/1/11): high-rate w suggests mast down 11:08 -  11:13; somewhat ambiguous

                           high-rate kh2oV shows kh2o cleaned at 11:10, which is consistent

          (3/3/11): high-rate data for u, T-tc, and p support this mast down period, but don't clarify it

STATUS DEC 14, 2010

Status evaluated from 36 hour plots on Dec 14 08:00-9:00 MST
On site: Steve S and Chris

Summary:  no Tsoil at W Slope main

Vdsm: ok, 12-14.3 Volts
Vmote.rad: ok,  12.7-15 V
Vmote.soil: ok, 12.3-18 V

P: ok, 840 to 880 mb
Tbaro: ok, 0 to 20 degC
T: ok, -2  to 13 degC
RH: ok, 30 to 100%; E Slope spike low, correlated with Q
Q: ok, 3 to 6 gm/kg; Playa low spike once not correlated with RH
Rainr (3,5,6): ok
Spd (1,6): ok, up to 6 m/s
Dir (1,6): ok, currently ~S

csat diag: ok
spd: ok, up to 9 m/s
dir: ok
u'u': ok, generally < 0.5 (m/s)^2; up to 1.6 m^2/s^2 at 7
v'v': ok, ditto, high spike at 4 at ~Dec 14, 5:00am
w'w': ok, generally < 0.1 (m/s)^2; up to 0.8 m^2/s^2 at 7
u*: ok, up to 0.2 m/s, up to 0.5 m/s at 7
sigma_w/u*: ok, ~2

tc: ok, 3 to 12 deg C
tc'tc': generally < 0.2 degC^2
w'tc': ok, -0.1 to 0.0 m/s degC

kh2oV: ok, 0.5 to 2.5 V
kh2o'kh2o': ok, currently up to 0.001 degC^2
w'kh2o': ok, currently up to 0.01 degC^2

Rsw.in: ok, up to 500 W/m^2 on 12/14
Rsw.out: ok, up to 130 W/m^2 on 12/14
albedo: ok, 0.1 to 0.3
Rsw.dfs (1,7): ok, up to 250 W/m^2 on 12/14
Rsw.global (1,7): ok, up to 500 W/m^2 on 12/14

Rlw.in: ok, up to 330 W/m^2
Rpile.in: ok, -120 to -20 W/m^2
Tcase.in: ok, 0 to 15 deg C
Tdome.in (1,2,5): ok, 0 to 15 deg C

Rlw.out: ok, up to 380 W/m^2
Rpile.out: ok, -20 to 0 W/m^2
Tcase.out: ok, 1 to 12 deg C
Tdome.out (1,2,5): ok, 1 to 13 deg C

soil.aux at E & W Slope (5,6)
Tsoil: missing W Slope main
Qsoil: 14 to 33 %vol
Gsoil: -90 to 40 W/m^2
Cvsoil: 5e5 to 2.7e6

TRHs

We have built four new TRHs for PCAPS.  These are numbered 025, 026, 027, 028.  These have been modified to use RS232 or I2C communications.  We have removed the ferrite beads to the communication lines and added shunts.  The board filler was our 'ear goo' and used epoxy to make it water proof.  Nail polish has been used to coat the contacts on the transducer.  GOR-TEX has been added to transducer to help with protection.

2010/12/12 3:00pm MST Shadowband at riverton adjusted from 39 to 37 degrees.

pyranometer was level upon arrival.

Hoist Conditions

Steve and I took apart the winch for a more in-depth view of gears and barrel condition.  Found the part added to the drum was not needed anymore and we removed it.  Now winch will be able to take a higher load.  Looking closer at the gears we noticed wear on barrel gear than the worm gear.  Worm gear seems to have an odd 'bite' out of the bottom of it.  Nothing too scary, but something to keep on eye on during rest of this winches life.  The barrel  gear is definitely worn.  We will keep an extremely close eye on this gear.  Not that it's a worry for raising mast, but something will be done for the new year.  Kurt has ordered a new winch that is exactly the same so for the new year shift they will have a brand new winch tool.

Fun times!

2010/12/12 2:05pm MST shadowband angle adjusted from 40 degrees to 35 degrees. 

pyronometer was not level upon arrival, axis normal to the bar was not secured tightly, and was off level. Shadowband also not perpindicular to rad stand. Did not have proper equipment to determine offset from 90 degrees. Pyranometer should now be level. Shadow was set so diffuser was in the middle of the shadow band. 

Shadowband at Playa 1 adjusted for the first time since mid November (site installation).

Bird feces observed in both the shadowband and the upward facing longwave radiometer. Both diffusers were cleaned by hand. Precipitation may help with more cleaning.

14:15:   Mote ID locations for soil probes: ID10 is main and ID21 is aux

Sonic angle is 77.8 magnetic (sonic pointing west). Prop angle is 346.1 magnetic (prop pointing south)

cleaned dooms.

1:30 - 1:35 pm

Chris replaced the TRH inlet due to a broken bell and Tom cleaned the radiometer dooms. Drip loops added to downlookers.

The boom angle of the sonic was shot;  85.5 magnetic (sonic looking west)

Status Dec 13, 2010

Status evaluated from 36 hour plots on Dec 13 09:00-10:00 MST
On site: Tom and Gordon; Steve S and Chris

Summary:  no Tsoil at W Slope main

Vdsm: ok, 12-14.5 Volts
Vmote.rad: ok, 13-18 V
Vmote.soil: 13-18 V

P: ok, 845 to 880 mb
Tbaro: ok, 0 to 20 degC
T: ok, 0 to 11 degC
RH: ok, 60 to 100%; E Slope intemittent
Q: ok, 3 to 6 gm/kg; E Slope & Playa intermittent & correlated
Rainr (3,5,6): erroneous 3 mm/hr tip at W Valley
Spd (1,6): ok, up to 7 m/s
Dir (1,6): ok, currently ~S

csat diag: ok; minor errors last night
spd: ok, up to 8 m/s
dir: ok
u'u': ok, generally < 0.5 (m/s)^2; up to 1.5 m^2/s^2 at 3,4,7
v'v': ok, ditto
w'w': ok, generally < 0.1 (m/s)^2; up to 0.8 m^2/s^2 at 3,4,7p to 1 m^2/s^2
u*: ok, up to 0.7 m/s
sigma_w/u*: ok, ~1.5

tc: ok, 0 to 14 deg C
tc'tc': generally < 0.4 degC^2
w'tc': ok, -0.1 to 0.1 m/s degC

kh2oV: ok, 0.5 to 3 V
kh2o'kh2o': ok, currently up to 0.002 degC^2
w'kh2o': ok, currently up to 0.01 degC^2

Rsw.in: ok, up to 400 W/m^2 on 12/13
Rsw.out: ok, up to 150 W/m^2 on 12/13
albedo: ok, 0.1 t0 0.25
Rsw.dfs (1,7): ok, up to 500 W/m^2 on 12/13
Rsw.global (1,7): ok, up to 500 W/m^2 on 12/13

Rlw.in: ok, up to 340 W/m^2
Rpile.in: ok, -120 to 0 W/m^2
Tcase.in: ok, -2 to 18 deg C
Tdome.in (1,2,5): ok, -2 to 18 deg C

Rlw.out: ok, up to 400 W/m^2
Rpile.out: ok, -20 to 20 W/m^2
Tcase.out: ok, -2 to 18 deg C
Tdome.out (1,2,5): ok, -2 to 18 deg C

soil.aux at E & W Slope (5,6)
Tsoil: missing W Slope main
Qsoil: 13 to 28 %vol
Gsoil: -100 to 50 W/m^2
Cvsoil: 5e5 to 2.5e6

playa1 TRH

12/12/10  6:00 pm  changed SHT09 toSHT26. SHT09 was installed yesterday by TOm and Gordon. It came from Boulder in the care package.

SHT026 is a new sensor from Boulder.

hiland3 soil

Dec 11, 6:15 pm, Tom, Gordon

Installed old-style single tsoil probe, labeled 021, removing 4 probe pickle fork which failed several days ago.

eslope5 krypton, soil

Dec 11, 5:10 pm, Tom, Gordon

Tom installed old-style single soil probe at main soil site, unit 23

mast down 5:15, cleaned krypton, mast up 5:27. Sonic wicks ok.

   twh (3/1/11): high-rate w suggests mast down 17:17 - 17:26

          (3/2/11): high-rate u, P indicate mast down 17:16 - 17:27

                           high-rate kh2oV indicates kh2o cleaned at 17:20

Inserted GPS jumpers in main soil mote.

river7 soil, krypton

Dec 11, 03:54 pm

Dug up tsoil 8. Replaced with unit 5 which we received from Boulder this morning. This is a 4 level pickle fork.

mast down 4:04 pm

krypton had liquid water on lens. cleaned.

  twh (3/1/11): high-rate kh2oV indicates cleaned at 16:07

Replaced one missing sonic wick on bottom transducer

mast up 4:22

   twh (3/1/11): high-rate w suggests mast down 16:05 - 16:17

     (3/2/11): u, T-tc and P data indicate mast down 16:06 - 16:17

                           logbook times may include prep and completion for lowering mast

Flange on the TRH radiation shield has a chunk missing.

wvally4 krypton

Dec 11, 3:00 pm, Tom, Gordon

Mast down 3:05, cleaned krypton, which had liquid water on lens

   twh (3/1/11): high-rate kh2oV shows cleaned at 15:08

Mast up 3:22 pm

   twh (3/1/11): high-rate w suggests mast down 15:07 - 15:15;

          (3/2/11): high-rate u,P, support these times 

                          time in logbook may reflect prep and finish times for lowering mast

Dec 11, 1:08 pm, Tom, Gordon

Replaced TRH transducer 013 with 009

Cleaned krypton from ladder, did not drop mast.

Installed higher gain antenna on cellular modem. This turned out to be a hassle.

Prior to the antenna change, the router modem page reported signal 65%, -78 dBm for the modem, which is not bad, but I figured we could use this as an opportunity to test the higher gain antenna. If it ain't broke, fix it anyway...

I tried hot-swapping the antenna on the modem, which then apparently caused problems on the router. At first the WLAN would not come up. Connected via the ethernet, the wifi was disabled. After re-enabling the wifi, the router would not stay up. The router WAN and WLAN LEDs would come on, and then the USB LEN, and then after a second all would go out again, as if the router was resetting. Connection to the ethernet or wifi would only last a second or two.

Reset router to factory defaults by holding in reset button while powering up. Then downloaded the configuration and tweaked it for station isfs1. Router would then stay up.

After the change the modem signal level was 70%, -77 dBm. This is no real improvement from the smaller antenna.

Status Dec 11, 2010

Status evaluated from 36 hour plots on Dec 11 09:00-10:00 MST
On site: Tom and Gordon

Summary: RH at Playa out
         kh2oV low at all sites
         soil mote out at Riverton, presumable Tsoil took it down
         no Tsoil at Highland, Riverton, E&W Slope main

Vdsm: ok, 12-14.5 Volts
Vmote.rad: ok, 13-18 V
Vmote.soil: 12-14 V, missing Riverton

P: ok, 840 to 880 mb
Tbaro: ok, 2 to 12 degC
T: ok, 0 to 10 degC
RH: ok, 60 to 100 %; Playa bad
Q: ok, 4 to 6 gm/kg; Playa bad
Rainr (3,5,6): up to 10 mm/hr at all sites
Spd (1,6): ok, up to 10 m/s
Dir (1,6): ok, currently E at Playa, NW at W Slope

csat diag: ok now; high last night, likely rain
spd: ok, generally up to 9 m/s
dir: ok
u'u': ok, up to 4 m^2/s^2
v'v': ok, up to 4 m^2/s^2
w'w': ok, up to 1 m^2/s^2
u*: ok, up to 0.7 m/s
sigma_w/u*: ok, ~1.5

tc: ok, 2 to 10 deg C
tc'tc': generally < 0.2 degC^2
w'tc': ok, -0.05 to 0 m/s degC

kh2oV: ok, minimal now, 0.4 to 1.7 V prior to rain
kh2o'kh2o': ??
w'kh2o': ??

Rsw.in: ok, up to 400 W/m^2 on 12/9
Rsw.out: ok, up to 80 W/m^2 on 12/9
albedo: ok, 0.1 t0 0.2, wslope still wet?
Rsw.dfs (1,7): ok, up to 300 W/m^2 on 12/9
Rsw.global (1,7): ok, up to 300 W/m^2 on 12/9
Note: on 12/9 Rsw.dfs = Rsw.global = Rsw.in

Rlw.in: ok, up to 340 W/m^2
Rpile.in: ok, -60 to 0 W/m^2
Tcase.in: ok, 0 to 9 deg C
Tdome.in (1,2,5): ok, 0 to 9 deg C

Rlw.out: ok, up to 370 W/m^2
Rpile.out: ok, -20 to 15 W/m^2
Tcase.out: ok, 0 to 9 deg C
Tdome.out (1,2,5): ok, 0 to 9 deg C

soil.aux at E & W Slope (5,6)
Tsoil: good at 1,2,4,5aux,6aux;
       missing Highland, E Slope main, W Slope main, Riverton
Qsoil: 25 to 35 %vol; missing Riverton
Gsoil: -50 to 30 W/m^2; missing Riverton
Cvsoil: 5e5 to 3e6; missing Riverton

Sonic QC

I created and edited csat_isfs#.dat cal_files in boom_normal directory

I set cal_files to NA when diag exceeded 0.01 to 0.02.

Cal files edited through Dec 1 12:00 and copied to instrument directory.

Dec 9, 08:44 am, Tom, Gordon

data from wvally4 stopped overnight. At the site, the data system up and taking data, yesterday's files had been copied.

/var/log/isfs/kernel had many messages about eth0 down and up.

Power, WAN and WLAN LEDs were on, but not the USB. Modem LED was on solid. Could not ping the router, 192.168.0.1.

Power cycled router with

vio 7 0
vio 7 1

Router came up and the modem connected.

Apparently need to modify the network check scripts. If the viper cannot ping the router, then the router is not power cycled. It is cycled only if the viper can ping the router, but cannot ping google or EOL. This was done so that the modem would not do reconnects if the problem was in the ethernet wiring of the box. Will change the crontab, so that once an hour if the router cannot be pinged, the power will be cycled.

Pulled a drip loop in pyrgeometer data cable. Drip loop on pyranometer is not as good.

Left 8:44 am.

Updated the crontabs on all stations, so that once an hour do a router_check.sh without a net_check.sh:

# Only run router_check.sh if net_check.sh succeeds to avoid power cycling modem
# if the problem is with the ethernet connection
15,30,45 * * * * net_check.sh eth0 192.168.0 192.168.0.1 && router_check.sh 7 www.eol.uca
# once an hour skip net_check.sh                                                         
0 * * * * router_check.sh 7 www.eol.ucar.edu www.google.com
playa1 krypton

Dec 08, 4:23 pm, Tom, Gordon

Cleaned krypton with water from ladder, didn't lower mast. 4:2? pm

   twh (3/1/11): high-rate data shows cleaned at 16:27

Sonic wicks ok, though one extended about 1cm past surface of transducer.
Pulled it back, so that it extended just a little past surface.

kh2oV = 1.5 V after cleaning

abc2 krypton

Dec 8, 3:40 pm, Tom, Gordon

kh2oV = 0.001 V

Mast down 3:49

cleaned krypton with water

   twh (3/1/11): high-rate data shows cleaned at 15:53

kh2oV=1.6 V

sonic wicks OK

mast up 3:56 pm

   twh (3/1/11): high-rate w suggests mast down 15:51 - 15:56

           (3/2/11): high-rate T-tc supports these times

wvally4 krypton

Dec 8, 2:33 pm, Tom, Gordon

kh2oV = 0.06 V

Lowered mast 2:42 pm

Cleaned krypton with water. kh2oV=1.3 V

  twh (3/1/11): high-rate data shows cleaning at 14:44

Sonic wicks OK

Raised mast, 2:49 pm.

  twh (3/1/11): high-rate u,w data shows mast down 14:42 - 14:47;

                          14:49 may reflect time taken to complete guy wire attachment

         (3/2/11): high-rate pressure data agrees with u,w data

hiland3 krypton, soils

Dec 8, 1:42 pm MST, Tom, Gordon

kh2oV = 0.27 V

Lowered mast at 1:48 pm. Cleaned krypton with water. kh2oV = 1.5 V

Raised mast 1:59.

   twh (3/1/11) high-rate kh2oV and w suggest these times for lowering & raising the mast are okay.

          (3/2/11): kh20V data shows cleaning occurred at 13:52;

                          high-rate pressure data clearly shows mast down 13:50 - 15:58;

                          longer period (13:48 - 13:59) may account for time to begin and complete mast lowering

Soil mote has not been reporting at this site since last night. Tsoil owed spikes last night, then all soil data stopped.

mote_dump showed it was continually resetting. Yellow light on solid.

Disconnected tsoil 018, restarted mote. This site now has no Tsoil.

Left 2:03

Daily status Dec 8, 2010

Status evaluated Dec 8 10:00-12:00 MST
(36 hour plots; missing Dec 1700 to Dec 8 0900)
On site: Tom and Gordon

Summary: RH at Playa out
         kh2oV low at sites 1-4
         soil mote out at Highland, presumably Tsoil took it down.

Vdsm: ok, 12-14.5 Volts
Vmote.rad: ok, 13-18 V
Vmote.soil: ok, 12-18 V

P: ok
Tbaro: ok, 0 to 20 degC
T: ok, -3 to 8 degC
RH: ok, 60 to 100 %; Playa bad
Q: ok, 3.5 to 6 gm/kg, drying; Playa bad
Rainr (3,5,6): 3-9 mm/hr events(?) at Highland, W Valley
Spd (1,6): ok, up to 5 m/s
Dir (1,6): ok, currently W at Playa, S at W Slope

csat diag: ok now, high afternoon of Dec 6, likely precip/fog
spd: ok, generally up to 3 m/s; 6-7 m/s at Riverton
dir: ok, currently W at Playa, others ~SE
u'u': ok, up to 1.5 m^2/s^2 at Riverton; others generally < 0.5 m/s
v'v': ok, ditto
w'w': ok, up to 0.7 m^2/s^2 at Riverton; other generally < 0.2 m/s
u*: ok, up to 0.6 m/s at Riverton; others generally < 0.3 m/s
sigma_w/u*: ok, 1-3

tc: ok, -2 to 10 deg C
tc'tc': generally < 0.2 degC^2; currently < 0.04 degC^2
w'tc': ok, fluxes > 0 today, ~ 0.03 m/s degC; highest at Riverton

kh2oV: ok, 0 to 1.4 V, Playa and ABC < 20 mv; W Valley 150 mv; Highland 270 mv.
kh2o'kh2o': generally < 0.01 (gm/m^3)^2; Playa and ABC too high
w'kh2o': ok, generally < 0.02 m/s gm/m^3

Rsw.in: ok, up to 300 W/m^2 on 12/8
Rsw.out: ok, up to 120 W/m^2 on 12/8
albedo: ok, up to 0.4; ABC lowest, W Valley highest
Rsw.dfs (1,7): ok, up to 200 W/m^2 on 12/8
Rsw.global (1,7): ok, up to 200 W/m^2 on 12/8

Rlw.in: ok, up to 320 W/m^2 on 12/8
Rpile.in: ok, -100 to 0 W/m^2 on 12/8
Tcase.in: ok, -3 to 7 deg C on 12/8
Tdome.in (1,2,5): ok, -3 to 7 deg C on 12/8

Rlw.out: ok, up to 340 W/m^2 on 12/8
Rpile.out: ok, -35 to 5 W/m^2 on 12/8
Tcase.out: ok, -3 to 7 deg C on 12/8
Tdome.out (1,2,5): ok, -3 to 7 deg C on 12/8

soil.aux at Playa E&W Slope (1,5,6)
Tsoil: good at 1,2,4,5aux,6aux;
       missing Playa aux, E Slope main, W Slope main,
       RIP at Highland sometime after 1800 MST Dec 7
       3 cm at River looks ok, but spiking
Qsoil: 15 to 34 %vol; missing Highland
Gsoil: -100 to 20 W/m^2; missing Highland
Cvsoil: 5e5 to 3e6; missing Highland

baked TRHs

Per Steve Semmer's suggestion, left a bad TRH (015 from abc2) in the soil oven at between 35 and 40C for a few hours yesterday. This unit was giving a RH of -3, with a raw value of 0. After the baking it was generating believable T and Rh in the trailer, but upon blowing on it, the RH would go up to 50% and then cut out, back to -3%. Same thing happened when putting it outside.

Left 015 and 023 (from wslope6) TRH transducers in the soil oven at about 40C overnight. Later the next afternoon tested 015 again. Data "looked" ok and couldn't get it to fail by breathing or putting it outside.

Tom doesn't trust it, since the calibration may have changed, so we'll send these back to Boulder, along with some Tsoils.

Dec 6, 15:20 pm

Removed TRH023 which has been reporting raw values of 0 for T and RH since yesterday. Saw no obvious problems with the transducer.

Installed TRH24 at 15:30

Sonic has wicks.

Pulled a drip loop in downlooking PYG cable. PSP had a nice drip loop.

Dec 6, 14:20 Tom, Gordon

Installed Epply PSP 26224 downlooker. Made sure there were drip loops in the data cables for the downlooking radiometers.

PSP 16714 had been removed from this site yesterday due to a water leak.

26224 was a spare from the trailer, which needed an I2C address change. Previously it was address 85 (0x55), and I changed it to 91 (0x5b).

Sonic at this site has all wicks. Tom used datascope as a binocular.

At the site, noticed that main soil mote (ID12) was not reporting. Power cycled it.

Left at 14:40.

Dec 6, 10:20 am, Tom, Gordon

Fog, light rain, which stopped.

Installed TP01 at aux soil (mote ID20). This is the TP01 which was removed from site 1 this morning. Didn't check the serial number. There is no serial number on the connector. Tom thinks it might have been unit 013.

Left 10:42 am.

abc2 Qsoil, TRH

Dec 6, 09:20 am, Tom, Gordon

Light rain.

RH has been reading -3 % here since last night. Removed unit 015. Goretex sock was intact, a damp black film present on the goretex - probably from dust. This unit is covered with a brown resin where the transducer is soldered.

Brought 015 back to car to let it dry out a bit. Inside of din socket was damp. Sprayed it with some canned air. Put back in unit, but RH still bad. Replaced with 021.

Tom removed Qsoil, ECHO003, which had been reporting intermittently. Replaced with unit 020 which we had just removed from aux playa1. Values look good.

Left 09:53

Dec 6, 08:25 am, Tom, Gordon

Removed all aux soils, sensors, mote, and post.

Removed dead ts013 at main site and buried ts017 inserting it into the holes of ts013.

ts017 had been hanging in air since last night.

Left site at 08:40.

TRH at wslope6 failed

A little after 15:00 MST Dec 5, the TRH at wslope6 failed:

TRH23 -40.21 -3.99 0 0\r\n
TRH23 -40.21 -3.99 0 0\r\n
TRH23 -40.21 -3.99 0 0\r\n
TRH23 -40.21 -3.99 0 0\r\n

Note that the raw values are 0.

I've tried:

eio 5 0
eio 5 1

but it didn't help.

Fog was prevalent all over the valley yesterday, including at 6. We visited the site at 16:15 but didn't notice that the TRH was out.

Status December 5, 2010

Status (36 hour plots): Dec 3 12:00 to 13:20

On site: Tom and Gordon

Summary:  bad Rsw.out removed at E Slope
         Mote at W Slope main down, likely caused by bad Tsoil
         Missing Tsoil at Playa (both), E Slope main and W slope main
         Missing Cvsoil at Eslope aux

         Qsoil intermittent at ABC

Vdsm: ok, 12-14.5 Volts
Vmote.rad: ok, 13-18 V
Vmote.soil: ok, 12-18 V; missing Playa main, W Slope main

P: ok
Tbaro: ok, 0 to 20 degC
T: ok, -4 to 6 degC
RH: ok, 70 to 106 (Playa) %RH
Q: ok, 3.5 to 5.3 gm/kg
Rainr (3,5,6): 3mm/hr event at Highland, 2 at W Valley
Spd (1,6): ok, up to 4 m/s
spd: ok, generally up to 3 m/s; 6-7 m/s at Riverton
Dir (1,6): ok, currently N at Playa, S at W Slope
dir: ok, N at Playa, W Valley; others ~S

u'u': ok, up to 2 m^2/s^2 at Riverton; others generally < 0.5 m/s
v'v': ok, ditto
w'w': ok, up to 0.7 m^2/s^2 at Riverton; other generally < 0.2 m/s
u*: ok, up to 0.5 m/s at Riverton; others generally < 0.3 m/s
sigma_w/u*: ok, 1 to 2.5 , including Riverton

kh2oV: ok, 0 to 1.5 V
kh2o'kh2o': generally < 0.1 (gm/m^3)^2
w'kh2o': generally < 0.02 m/s gm/m^3

tc: ok, -2 to 6 deg C
tc'tc': generally < 0.2 degC^2; eslope, wslope and river have higher variance
w'tc': ok, fluxes positive today, ~ 0.01 m/s degC; highest at playa and w valley
diag: ok, a few events to 0.01 to 0.04

Rsw.in: ok, up to 500 W/m^2 on 12/5
Rsw.out: ok, up to 350 W/m^2 on 12/5; removed sensor at E Slope ~1830 MST, Dec 4
Rsw.dfs (1,7): ok, up to 400 W/m^2 on 12/5
Rsw.global (1,7): ok, up to 500 W/m^2 on 12/5

Rlw.in: ok, up to 320 W/m^2 on 12/5
Rpile.in: ok, -140 to 0 W/m^2 on 12/5
Tcase.in: ok, -3 to 15 deg C on 12/5
Tdome.in (1,2,5): ok, -3 to 13 deg C on 12/5

Rlw.out: ok, up to 320 W/m^2 on 12/5 (spurious spike at site 5, 1800 MST, Dec 4)
Rpile.out: ok, -40 to 10 W/m^2 on 12/5
Tcase.out: ok, -3 to 15 deg C on 12/5
Tdome.out (1,2,5): ok, -3 to 13 deg C on 12/5

soil.aux at Playa E&W Slope (1,5,6)
Tsoil: good at 2,3,4,5aux,6aux;
       missing Playa (both), E Slope main, W Slope main,
       3 cm at river(7) is increasingly high also erratic
Qsoil: 12 to 34 %vol; missing W Slope main, abc (intermittent),
Gsoil: 5 to 10 W/m^2; missing W Slope main
Cvsoil: 1-3e7; missing W slope main, E Slope (aux)

Dec  5, 1158 MST: Joe Young disconnected Tsoil at Playa main soil site and pressed the white reset button on the mote.

Gordon looked at the data and it is reporting Gsoil, Qsoil, and TP01.

Dec 4, 6:15 pm, Tom, Gordon

visited eslope5 to diagnose problem with Rsw.out data, which has been looking bad for a while.

The inner dome of downlooking Epply PSP 16714 was 1/3 filled with water. Removed.

12/4 16:15 pm, Tom, Gordon

The mote at this site (ID3) has been removed since 12/3 when we couldn't get it to report data.

Replaced this mote with ID19, swapping radios. All soil sensors were reporting, including Tsoil. Before we left noticed that the 4th Tsoil value went bad.

Dec 4, 09:15 am

At the playa site yesterday, we noticed that the soil mote at the rad stand was ID3 and the auxilary mote was ID8. The XML configuration had them reversed. Hence, the web plots, netcdf files and cockpit are all labelling the soil variables incorrectly. I will correct the XML and re-run the netcdf covars and remake the plots.

playa1 auxiliary soil

12/4, 16:15 pm, Tom, Gordon

Opened door of aux mote (ID8) and checked voltage at white power connector: 11.5 V.

Sequentially pulled and replaced one sensor at a time. Voltage recovered to 11.6 when Qsoil, Gsoil, TP01 removed, to 11.7 when Tsoil removed.

The mote was running all during this time. It is supposed to shut down when the voltage falls below 11.9. It appears that the mote voltage measurement was erroneously high, see plot and
discussion below.

When Tsoil was unplugged, Gsoil and TP01 were reporting, no Qsoil.

When Tsoil was plugged in, Gsoil abruptly changed from 9.4 to -1.

Replaced battery. Voltage 12.46 without Tsoil. Now receiving Q,G and TP01. When Tsoil was plugged in, it caused step change in G and Qsoil. Left Tsoil unplugged. Note that Qsoil started reporting after the battery was replaced.

Attached is a plot of soil variables from aux at playa1. When the Tsoil goes bad it effects the measurement of Vmote and Gsoil. When we arrived at the site and disconnected Tsoil then Vmote dropped from 17 V to a more correct value of 12. The Vmote values of 12.5 V after 17:00 MST are with the replaced battery.

Status December 3, 2010

Status (36 hour plots): Dec 3 11:00-12:00

Summary: kh2o deployed Nov 30-Dec 1
         Resurrected Site 1 aux sensors Dec 3
         Replaced Site 3 Tsoil Dec 3

To do: Soil mote bad at Playa #1
          Tsoil (3 cm) bad at River
          Qsoil intermittent at ABC

          All soil data spiking at W Slope #1

          Cvsoil missing at E Slope #2

          Rainr spikes at W Valley

Vdsm: ok, 12-14.5 Volts
Vmote.rad: ok, 13-18 V
Vmote.soil: ok, 12-14 V; missing Playa #1

P: ok
Tbaro: ok, -10 to 20 degC
T: ok, -12 to 10 degC
RH: ok, 35 to 103 (Playa) %RH
Rainr (3,5,6): spurious 3 mm/hr spike at W Valley
Spd (1,6): ok, up to 10 (W Slope) m/s
spd: ok, up to 9 m/s; W Slope & Riverton are highest
Dir (1,6): ok, currently W; have prop directions been set?
dir: ok, have sonic directions been set?

u'u': ok, up to 1.5 m^2/s^2, Riverton high
v'v': ok, up to 2 m^2/s^2, Riverton high
w'w': ok, up to 0.7 m^2/s^2, Riverton high
u*: ok, up to 0.6 m^2/s^2, Riverton high
sigma_w/u*: ok, ~1.5, including Riverton

kh2oV: ok, 0.75 to 2.5 V
kh2o'kh2o': died around 09:00 Dec 2
w'kh2o': fluxes > 0 after 09:00 Dec 2,
         all except Playa and Highland -> 0 ~17:00 Dec 2

tc: ok, increasing from -10 to 5 deg C
tc'tc': eslope, wslope and river have higher variance
w'tc': ok, fluxes negative today, highest at eslope, wslope, and river
diag: ok, playa bad 0700-1000, Dec 2, probably water on transducers

Rsw.in: ok, up to 600 W/m^2 on 12/2
Rsw.out: ok, up to 450 W/m^2 on 12/2
Rsw.dfs (1,7): ok, up to 450 W/m^2 on 12/2
Rsw.global (1,7): ok, up to 600 W/m^2 on 12/2

Rlw.in: ok, up to 320 W/m^2
Rpile.in: ok, -140 to 0 W/m^2
Tcase.in: ok, -10 to 15 deg C
Tdome.in (1,2,5): ok, -10 to 15 deg C

Rlw.out: ok, up to 320 W/m^2
Rpile.out: ok, -80 to 0 W/m^2
Tcase.out: ok, -10 to 15 deg C
Tdome.out (1,2,5): ok, -10 to 15 deg C

Dec 3 17:30
soil.aux at Playa E&W Slope (1,5,6)
Tsoil*: good at 1b,2,3,4,5,5a,6a;
       missing Playa (#1), E Slope (#1), W Slope(#1),
       3 cm at river(7) is increasingly high also erratic
Qsoil*: 10 to 40 %vol; missing playa(#1), abc (intermittent),
        spikes in W Slope #1
Gsoil*: 6 to 16 W/m^2; missing playa(#1), spikes in W Slope #1
Cvsoil: 1-3e7; missing playa(#1), E slope(#2); spikes W slope(#1)

Dec 3, Playa motes at main site (near radiometers) and aux site (north of main) and Highland not reporting any data

Playa, 14:16 MST

Mote at main soil site is ID3: battery voltage = 12.5 V; reporting time information, but no soil data.

Mote at aux soil site is ID8: battery voltage = 11.4 V; disconnected gps, recycled power and data good.

However ID8 soon quit because battery voltage below 11.9 V (per JM).

Replaced aux battery w/ battery from main; now aux battery voltage = 13.5 V (perhaps 12.5 V was a typo).

Brought Mote ID3 and aux battery back to base

Highland, 15:54 MST

Soil mote battery voltage = 12.7 V.

Looking at data from motes:

mote_dump

showed continual reboot-like messages from the soil mote (ID22): "watchdog timer" and "sampling_mode=0", etc. No soil data. Yellow light was on solid.

Disconnected soil sensors and added them back sequentially. Data good until we connected Tsoil.

Replaced Tsoil w/ #0018 from Boulder (GM coated with fingernail polish in base).

Brought bad Tsoil back to base.

Soil motes at E Slope

When Gordon and Tom were at E Slope on Dec 1, the main soil mote was not reporting, but the aux mote was reporting.

Then around 15:52 (while we were there), the aux mote stopped and, around 16:02 the main mote started.

On Dec 2 around 05:37, the main mote restarted.

wifi changes

During the site visits of Nov 30, Dec 1 I changed all the wifi configurations at the sites so that the SSIDs are visible. According to a Cisco web site I read, a hidden SSID offers no real added security.

I couldn't get the KDE network manager on Fedora to connect to hidden SSIDs (so its a linux thing...)

river7 winds okay

It was obvious yesterday that river7's winds were completely different from the other sites.  Our internal checks showed that diag and sigma_w/u* from this site were okay and reasonable.  I show corroborating evidence below from the MesoWest network that gap flow does occur at this site.  The time series of the MesoWest Riverton site (hopefully not our data!) shows exactly the same behavior in wind speed that we observe.


Qsoil and Cvsoil

So far, I haven't noticed much correlation between Qsoil and the TP01-derived heat capacity, except for one event at hiland3.  Presumably, the soil is still "healing" from the probe installation.  I'm pretty sure that I didn't pack the soil around the probes to the density of the undisturbed soil.  However, there has been a fair amount of precip, both liquid and frozen, since installation, so they should have healed a bit.

Unfortunately, temperatures are now cold, so the soils are close to freezing.  Tsoils are now in the range 0--2, though none have gone to below 0 yet.  When they do, we should start seeing Qsoil values drop as the water becomes invisible to the EC-5 sensors.  I suggest using the Qsoil values just prior to this drop as the actual soil moisture amount for the rest of the experiment, assuming that water in the soil is immobile once frozen.  This would ignore any melted snow that might penetrate the soil.

TRH fan noisy at Playa

November 30: TRH fan noisy at Playa.

November 30, 2010

Site 2 (ABC): upper radiometers snow/ice covered; lower pyrgeometer has a little frost on it; cleaned

Site 3 (Highland): upper radiometers partially snow/ice covered; cleaned @ 11:20

Site 1 (Playa): Radiometers bare of snow

Site 7 (Riverton): Radiometers bare of snow

Site 5 (E slope): cleaned uplooking radiometers @ 14:53

Cleaned solar panels as needed at all these sites; mostly bare or a little snow/ice near the bottom

kh2o installation

Nov 30, 2010.  Visited several sites to install recalibrated kh2o's.

Site 2 (ABC) 1389 head bad; output ~30 mv.  Installed 1393 about 10:30; output ~2 V.

     twh (3/1/11): dat("w") shows mast down ~ 09:15 - 10:35; high-rate w suggests similar period

             (3/2/11): high-rate pressure data more clearly show mast down 09:16 - 10:36

Site 3 (Highland) Arrived ~ 11:15

                          1333 head  bad; output ~30 mv

                          1258 head bad; output ~20 mv

                          1390 head bad; output ~20 mv

                          1525 good!; output 2.2 V

Attached 1133 head to 1525 electronics and warmed head; still bad.

Reattached 1525 head; output 2.3 V.

Reattached 1525 head and left.

   twh (3/1/11): dat("w") suggests mast down ~ 11:25 -  12:35; high-rate w suggests similar period

          (3/2/11): high-rate pressure data clearly shows 11:25 - 12:36

Site 1 (Playa) Installed 1395 around 14:30.  Also tested 1394, which was good.

   twh (3/1/11): dat("w") suggests mast down ~ 14:10 - 14:35; high-rate data suggests 14:07 - 14:34

      (3/2/11): high-rate pressure data clearly show mast down 14:07 - 14:34

Site 7 (Riverton) Installed 1394 around 15:55; output ~2.3 V.

   twh (3/1/11): dat("w") suggests mast down ~ 15:45 - 16:05; high-rate w shows 15:47 - 16:03

           (3/2/11): high-rate data of T-tc also support 15:47 - 16:03; pressure is not clear

Dec 1, 2010:  Completed kh2o installation.

Cleaned 'bad' kh2o's by scrubbing with qtips and distilled water. 

1133 went from 30 mv to 2 V

1258 went from 14 mv to 2 V

1389 went from 23 mv to 2 V

1390 went from 11 mv to 1.7 V.  Designated as spare.

Site 4 (West Valley): 12:10 installed kh2o 1258; 2 V output

    twh (3/1/11): dat("w") suggests mast down ?~ 11:35 - 12:15; high-rate w shows 11:41 -  12:15

           (3/2/11): high-rate pressure data clearly indicate mast down 11:42 - 12:16

Site 6 (West Slope): 14:30 installed kh2o 1389; 2 V output

    twh (3/1/11): mast does not appear to have been lowered

Site 5 (East Slope): 15:50 installed kh2o 1133; 2 V output

   twh (3/1/11): dat("w") suggests mast down ?~ 15:30 - 15:50; high-rate w shows 15:31 - 15:55

          (3/2/11): high-rate data for T-tc and pressure support 15:31 - 15:55

Playa 1 Snow Removal

Solar Panels at Site 1 Playa cleared at 1pm MST. Snow covered bottom panel on older set, and 1/3 of lower panel on the newer set. Melting resulted in ice from melted snow forming on lower panels.

Panels at rad sawhorse, and both soil sensor panels were clear at visit.

Approximately 10-12 inches of snow observed at site. Snow damming observed at south facing solar panels, leading to deeper snowdrift at base of panel. Built up snow cleared away from base of panels. Snow is not expected before NCAR ISFS crew arrives.

Snow removal occurred concurrently with installation of UU Mini-SODAR approximately 200m away.

site 1

visited site one to determine why the  aux soil mote was down. found that Tsoil was dragging down the mote, so I disconnected Tsoil, and mote now seems to be OK although Qsoil is not reporting.

so...  Tsoil.aux and both Qsoils are no longer reporting at site 1 although both motes are OK

checked radiometer domes and all are clean

site 2 trh

I have noticed that at site 2 on two separate occasions, when power to DSM is recycled, the trh data output is in hexadecimal until the sensor itself is rebooted.

moved Viper power cable to another power port on the main power board.

the prop vane azimuths were shot using the datascope earlier this week with the following results

site 1 (magnetic declination set to 12.3 degs per NOAA website, and datascope field calibrated by Chris Golubieski)   1.0 degs

site 6    1.7 degs

neither site has had this boom direction entered into the RMY propvane program as yet.

site 2 power

visited site 2 again today from about 1:30 till 3:00 monitoring power (voltage dropped again this morning to about 11.2 v)

swapped out viper power cable.

wiggling the cable brings the voltage back up to 12.7 v, (where it should be), but that's what was happening yesterday as well, and I repaired the original cable at that time. since the voltage dropped again this morning, right about the time the wind started getting brisk, and shaking the tower, i suspect a cold solder joint in the power distribution board. I had an extra cable, so I decided to try that first.

will monitor situation, and if voltage drops again, i will replace either power panel, or more likely, entire DSM so that the offending board/box can be scrutinized in a more friendly environment

Site 1 Playa notes

Friday, Nov 19th 11:00 AM MST to approx. 2:00 PM

 I cleaned incoming and outgoing radiometer domes - I noticed during site visit to improve cell connection that the domes (mostly the incoming) had a fine coating of soot. This is a particularly windy site, and the Kennecot copper mine smelter is directly across the highway, about 1-2 miles away. The mine regularly sprays water on the substantial slag heaps to keep the dust down, but it looks to be ineffective. This site may need cleaning more often.

while at the site I had a visit from Randy Finch, an engineer from KSL Radio who regularly visits their building / tower which is about 1/4 mile to the east of our ISFS station. He pointed out to me where the Verizon cell towers (2) are located. Neither are visible to me but their locations are to the NW and SW of our tower.

I positioned site 1's  cellular antenna to minimize blockage.

KSL's tower is a AM radio transmitter around 1.5 MHz

They have a secure building with land line internet connection as well as power, and he offered both to us if we so desired.

Contact info:

Randy Finch

KSL radio

801 556-7565 (cell)

801 575-7637 (transmitter building)

Sat Nov 20

visited site 4 (not reporting as of 7:00 PM last night)

10:30 AM MST cycled power to dsm

station now reporting

PS by Gordon: discovered that this site had a typo in the crontab
entry and the net_check.sh and router_check.sh scripts weren't being
run. Fixed the typo.

/var/log/isfs/messages* and a listing of data files (lsu) indicate that
the DSM was up and taking data, but the router/modem couldn't connect.

If router_check.sh script can't ping google or eol, it power cycles
the router&modem which should have fixed the issue.

reset TRH at abc2

Gordon, Nov 19 18:30 MST

At 22:39 UTC (15:39 MST) this TRH started generating the hex gibberish:

2010 11 19 22:35:30.1623  0.9771      29 TRH15 14.10 23.92 5413 732\r\n
2010 11 19 22:38:07.1683     157      29 TRH15 14.07 24.13 5410 738\r\n
2010 11 19 22:38:08.1425  0.9742      29 TRH15 14.05 24.19 5408 740\r\n
2010 11 19 22:39:53.3761   105.2      47 TRH\xff\xff1\xff\xff5 \xff1\xff4.\xff\xff\xff03\xff 24.\xff4\xff8 \xff5\xff40\xff\xff6 74\xff\xff8\r\n
2010 11 19 22:47:55.4524   482.1      42 TRH\xff\xff15 13\xff\xff.99 \xff2\xff4.26\xff \xff540\xff\xff\xff2 74\xff2\xff\r\n

I noticed the gravestones in cockpit after I had installed the new nidas code.

Powering it down and up with eio got it working again at 00:39 UTC Nov 20 (17:39 MST)

eio 5 0
eio 5 1

Perhaps we should do the power cycle of the TRH (port 5) every so often (10 minutes) at this site?

Kurt and Ling were at this site today, Kurt was working on the power inside the box, which had the low voltage problem.

I'll wait to see if the problem returns before doing the watchdog power reset on the TRH.

Nov 19 afternoon.

Kurt and Ling visited playa1 and abc2 today.

Per phone calls:

At playa1, tried to improve the cellular connection. Swapped in a new modem (modem-8 as I recall). Fiddled with the antenna. Sometimes it appeared that the signal was better without an antenna, but later the signal went bad and an antenna helped. So left the site with modem-8 and the external antenna connected. This site still has a typical signal of 28%, -93 dBm, and that seems to be the best we can get with the current setup. It is acceptable, but it would be nice to try a higher gain antenna here.

At abc2, investigated why the system voltage is low. Kurt found that the cable to the VIPER (or DSM) did not have a good connection at the panel, which he fixed. During this time the router reverted to factory defaults. (sad) Ling re-installed the router configuration. The system voltage recovered to 12.8 V. This is the second or third time we've seen the CTR350s revert to factory defaults, when they've had a power glitch.

Ling added: The cable from the power board to the VIPER is still questionable. The cable's connector was losing which Kurt fixed, but we still get lower battery voltage sometimes. We suggest to replace the cable completely. Gordon, please bring one of the kind the power cable with you and replace it, when you go to Utah after Thanksgiving. If you have question regards the cable,  please talk to Kurt or me.

Found why inadyn had exited on some systems. If the modem is down, a DNS lookup to the router can return an address of 192.168.0.1 for the dyndns.com server. inadyn then gets an incorrect response back from 192.168.0.1 and exits. Modified the code to sleep and continue.

Also, in a similar way, nidas could get a bad address for eol-rt-data. When it sends a UDP packet to the wrong address it gets a "connection refused", but continues to try that same address. New code will do a new DNS lookup.

Installed the new inadyn and nidas on all systems between 17:30 and 18:15 MST Nov 19.

Status 19 Nov

Today's status:

In general, pretty good by now so John and Ling return today.  (Kurt remains alone for a bit longer.)  Kryptons are being recalibrated in Boulder with shorter pathlengths and will be tested through serializers retrieved (today) from Manitou.  They will be installed by Tom and Gordon just prior to operations 29/30 Nov.  Fixes to the SPN1 and TP01 should also be straightforward.  Problems with Tsoils seem to be caused by inadequate conformal coating of the boards.  We'll probably just apply nail polish to fix.

Site

Battery
&Solar

Sonic

Krypton

Baro

TRH

Rad

Soil

Soil.aux

Precip

Shadowband

Prop

1

ok

ok

not installed

ok

ok

ok

ok

some Tsoil bad

not planned

SPN1 in Boulder
(A/D found bad)
Licor ok

ok

2

lowest

ok

not installed

ok

ok

ok

ok

not planned

not planned

not planned

not planned

3

ok

ok

not installed

ok

ok

ok

ok

not planned

ok

not planned

not planned

ok

ok

not installed

ok

ok

ok

ok

not planned

ok

not planned

not planned

5

ok

ok

not installed

ok

ok

ok

ok

Tsoil ok
Qsoil erratic
Gsoil ok 
TP01in Boulder
(will be reprogrammed)

not planned

not planned

not planned

6

ok

ok

not installed

ok

ok

ok

Tsoil profile kinked

ok

ok

not planned

ok

7

ok

ok

not installed

ok

ok

ok

ok

not planned

not planned

ok

not planned

pinging stations

Here are results from a short sample of about 10 pings to each station from Boulder, with a default ping packet of 56 bytes:

station

ping times (ms)

playa1, isfs1.dyndns.org

390-1160

abc2, isfs2.dyndns.org

200-480

hiland3,isfs3.dyndns.org

180-280

wvally4, isfs4.dyndns.org

150-300

eslope5,isfs5.dyndns.org

150-250

wslope6, isfs6.dyndns.org

180-200

river7, isfs7.dyndns.org

150-210

.
Station 1 is the obvious outlier, and it shows the lowest signal level as shown inhttps://wiki.ucar.edu/x/A4giB

Note that starting at 00:00 UTC (17:00 MST) every day, the previous days files are downloaded from each station. This process takes about 1/2 on each station (longer on isfs1). During the downloads the ping times will be longer.

hiland3 site visit

10:00 John just called from the site.  He tested the ETI gauge with about 1/2 liter of water and saw 26 tips.  27 tips would have been exact.  Thus, this gauge appears to be working.

site names

Field crew site names:

1 - none

2 - unprintable

3 - happy gravediggers

4 - dog poo

5 - don't touch my dog

6 - cemetary

7 - sex on a stone

wvally4 visit

Per phone conversation, Dec 17 15:48 MST

The modem at this site was doing many reconnects and registrations with dyndns.org. Ling taped the modem into the router, and saw a good signal level.

Tested the rain gauge, ETI8. Saw no tips from "data_dump -i 4,80 -A" after pouring 1 liter of water in the gauge. John connected a pc to debug, and power cycled the gauge. After the power cycle it registered tips. Poured 1/2 liter (from graduated water bottle) at 22:51:52, and saw a total of 27 tips, as follows, after a delay of 1 minute 35 seconds:

2010 11 17 22:53:12.1234    5.01       9 ETI8 0\r\n
2010 11 17 22:53:17.1336    5.01       9 ETI8 0\r\n
2010 11 17 22:53:22.1438    5.01       9 ETI8 0\r\n
2010 11 17 22:53:27.1535    5.01       9 ETI8 2\r\n
2010 11 17 22:53:32.1734    5.02       9 ETI8 7\r\n
2010 11 17 22:53:37.1836    5.01       9 ETI8 7\r\n
2010 11 17 22:53:42.1938    5.01       9 ETI8 4\r\n
2010 11 17 22:53:47.2039    5.01       9 ETI8 0\r\n
2010 11 17 22:53:52.2149   5.011       9 ETI8 0\r\n
2010 11 17 22:53:57.2251    5.01       9 ETI8 0\r\n
2010 11 17 22:54:02.2445   5.019       9 ETI8 0\r\n
2010 11 17 22:54:07.2547    5.01       9 ETI8 0\r\n
2010 11 17 22:54:12.2649    5.01       9 ETI8 0\r\n
2010 11 17 22:54:17.2751    5.01       9 ETI8 0\r\n
2010 11 17 22:54:22.2853    5.01       9 ETI8 0\r\n
2010 11 17 22:54:27.2954    5.01       9 ETI8 2\r\n
2010 11 17 22:54:32.3064   5.011       9 ETI8 5\r\n
2010 11 17 22:54:37.3262    5.02       9 ETI8 0\r\n
2010 11 17 22:54:42.3360    5.01       9 ETI8 0\r\n
2010 11 17 22:54:47.3462    5.01       9 ETI8 0\r\n

PS: Gordon, Nov 18, 9:30 MST. The frequent reconnects from this site have stopped. At this time the last dyndns registration from site 4 was at 4pm yesterday. Seems that securing the modem with tape solved the problem.

wslope6 visit

Per phone conversation, Dec 17, pm.

It appears that the CTR350 router had somehow lost its configuration and had reverted to factory defaults. Ling re-configured the CTR350 router at this site, with the configuration file at http://svn.eol.ucar.edu/svn/projects/PCAPS/ISFF/config/ctr350.gws. Ling sees good cellular signal level at this site.

John poured roughly 1 liter of water in the rain gauge at about 21:07:37 UTC. After roughly two minutes it reported tips. Counted a total of 66 tips.  [p.s. 0.01"/tip in a 12" gauge should produce 54 tips for 1 liter.  Thus, this works out to 1.22 liters.  John thinks that he "double poured" when he initially didn't see tips, so 1.22 liters is possible...SPO]

Sensor status 17 Nov

Today's status:

Site

Battery
&Solar

Sonic

Krypton

Baro

TRH

Rad

Soil

Soil.aux

Precip

Shadowband

Prop

1

ok

ok

not installed

ok

ok

ok

ok

ok

not planned

SPN1 removed
Licor ok

ok

2

lowest

ok

not installed

ok

not installed

ok

Tsoil removed

not planned

not planned

not planned

not planned

3

also low

ok

not installed

ok

not installed

ok

ok

not planned

ok?

not planned

not planned

ok

ok

not installed

ok

not installed

ok

Tsoil0.6/3.1/4.4 bad

not planned

ok?

not planned

not planned

5

ok

ok

not installed

ok

ok

ok

ok

Tsoil removed
Qsoil erratic
Gsoil ok 
TP01removed

not planned

not planned

not planned

6

ok

ok

not installed

ok

ok

ok

Tsoil dead

ok

ok?

not planned

ok

7

ok

ok

not installed

ok

not installed

ok

ok

not planned

not planned

ok

not planned

Nov 16, around 15:30 MST

Chris installed the sonic head at playa1

hiland3 site visit

Nov 16, 15:50 MST

Phone conversation with John.

Fixed charging issue, one solar panel wasn't connected.

Powered up soil probes. Soil battery was at 12.6 V. Qsoil didn't report initially, cable wasn't fully plugged in.

Cell signal is about 64% of full signal.

No TRH at this site yet.

abc2 site visit

Nov 16, 14:30 MST

Per phone conversation with John.

On site Ling, John, Kurt.

Communications to this site went out last night, which appears to be due to the USB modem not being fully plugged into the router. USB light was off on the router. After pushing the modem in, comms were re-established.

Updated coefficients in down-looking pyranometer.

Tsoil died at this site at around the same time as the modem went out last night. The Tsoil probe was removed for repair.

No TRH (or krypton) at this site at this time. Everything else looks OK. Cellular levels look good here: around 60% of full signal.

TRH dropout at eslope5

Nov 16

TRH at 5 dropped out for a little over an hour this morning, from 10:11 to 11:22 MST.
I don't think anyone was at the site.

Here's the raw dump:

data_dump -i 5,50 -A isfs_20101116_160000.dat.bz2 

2010 11 16 17:11:41.2109    0.98      29 TRH16 9.65 46.96 4946 1418\r\n
2010 11 16 17:11:42.1917  0.9808      29 TRH16 9.64 46.80 4945 1413\r\n
2010 11 16 17:11:43.1618    0.97      29 TRH16 9.65 46.71 4946 1410\r\n
2010 11 16 18:22:03.8285    4221      21 Ha2 = -1.382833E-6\r\n
2010 11 16 18:22:03.8585 0.03001      21 Ha3 =  3.354407E-2\r\n
2010 11 16 18:22:03.8895 0.03105      21 Ha4 =  3.666422E-5\r\n
2010 11 16 18:22:04.2285   0.339      29 TRH16 8.22 51.75 4804 1573\r\n
2010 11 16 18:22:05.1985    0.97      29 TRH16 8.22 51.72 4804 1572\r\n
2010 11 16 18:22:06.1794  0.9808      29 TRH16 8.22 51.72 4804 1572\r\n
playa1 visit

11:20 am MST, Nov 16

From phone conversation with John and Ling:

On site: John, Kurt, Ling

When they removed the SPN1, then the aux soils started coming in. Apparently a faulty I2C device can hang the mote. Mote ID 3 serves the aux soils and spn1 at this site.

They corrected the coefficients on the the pyranometers. Rsw.in and Rsw.out look good.

prep -D Rsw.in#1,Rsw.out#1 sock:porter
2010 11 16 18:26:12.4286         95       15.6
2010 11 16 18:26:17.4287       94.2       15.4
2010 11 16 18:26:27.4291       92.6       15.1
2010 11 16 18:26:32.4293       91.8       14.9

They forgot to bring the sonic head.

TRH013 is installed.

Shadow band radiometer is now working.

Cell modem signal levels: 28-30%, -93 dB

Updated sensor status

Here is a revised table, now that some things have been fixed (and others have died):

Site

Battery
&Solar

Sonic

Krypton

Baro

TRH

Rad

Soil

Soil.aux

Precip

Shadowband

Prop

1

ok

head off

not installed

ok

not installed

Rsw.in/Rsw.out need new coefficients
Rlw ok

ok

mote died last night

not planned

SPN1 reported  ~1.0/0.3 for a day, then died
Licor not working

ok

2 [network died last night]

ok

ok

not installed

ok

not installed

Rsw.out needs new coef
Rsw.in/Rlw ok

ok

not planned

not planned

not planned

not planned

3

still lowest

ok

not installed

ok

not installed

ok

turned off?

not planned

no fluid

not planned

not planned

4 ?[network resetting]

ok

ok

not installed

ok

not installed

ok

Tsoil 0.6/3.1/4.4 all bad

not planned

ok?

not planned

not planned

5

ok

ok

not installed

ok

ok (reprogrammed sensor installed)

4-comp flipped yesterday
Rsw.in needs new coef
Rsw.out/Rlw ok

ok

Tsoil.3.1/4.4cm bad 
Qsoil erratic
Gsoil ok 
TP01 not reporting

not planned

not planned

not planned

6

ok

ok

not installed

ok

ok

ok

Tsoil dead

ok

no fluid

not planned

ok

7

ok

ok

not installed

ok

not installed

ok

ok

not planned

not planned

ok

not planned

Sites 2,3 and 4 showed dropouts last night around 09:15 UTC (02:15 MST). Data from 3 and 4 resumed about 1/2 hour later, site 2 is still out.

As of this morning we're still not getting data from 2. A host at the IP address returned for isfs2.dyndns.org responds to pings, but reports "connection refused" when ssh-ing to the special port we use. It probably isn't our system at that address.

From the system log (/var/log/isfs/messages) at site 3, the router_check.sh script detected at 09:32 UTC that it couldn't ping eol or google, and successfully power cycled the router and modem. This happened just once. Things worked like they should!

Site 4 is more complicated. The cellular connection is frequently going up and down - several times an hour, which has been going on for the last serveral days. The router detects the connection problem itself and resets the modem, the router_check.sh script is not powering things down.

I think we should try an external antenna at site 4.

Over the weekend, crontab entries were added to all stations, except 6, to power cycle the base motes on /dev/ttyS6:

From crontab -l:

0 */3 * * * /usr/local/isff/scripts/power_cycle_port 6

This will then happen at 00:00, 03:00, ... UTC.

cron logs to /var/log/isfs/messages when it runs, which can be checked to verify the power cycle
script is being run.

Once we have 2-way comms to site 6 we can update its crontab.

Just spoke with John.  He is pretty sure that the radiometers were inadvertently mounted upside down.  This explains the pyrgeometers reporting reversed and Rsw.out being large.  It doesn't explain Rsw.in being VERY large.  Their schedule doesn't allow him to flip them over today, but it is on the list.

While at the site, he also unplugged the offending TP01.aux -- we'll see if data from the other sensors improve.  (So far, not for Tsoil.4.4cm, at least.)

   Site

Modem Signal Level in % and dBm

Taped USB-Modem

Playa1

28-30%   -93 dBm   (Antenna)

 

ABC2

64%        -78 dBm  

       yes

Hiland3

64%        -78 dBm

 

WValley4

61%        -80 dBm

       yes

ESlope5

75%        -74 dBm   (Antenna)

 

WSlope6

86%        -70 dBm   (Antenna)

       yes

River7

75%        -74 dBm

       yes

 

 

 

As of noon Nov 14, all 7 stations are set up and reporting.

Cellular comms are excellent at all sites, except for some weirdness at wslope6.

When wslope6 was setup, ChrisG reported that the LEDs on the router and modem indicated that it was having problem establishing a connection. I believe the LEDs looked similar to what he saw with the spare isfs8 system when powering it up in the base trailer. Putting an antenna in the window solved the problem at the trailer.

From http://www.eol.ucar.edu/deployment/field-deployments/field-projects/pcaps/ISFS/FieldNetwork
the LED on the USB760 modem (a little round LED which is about impossible to see in the sunlight) indicates the following:

USB760 LED

status

solid green

powered but not connected

blinking green

connecting or is connected. What you want to see

solid red

service not found, searching for service

solid orange

error has occurred. Reset the device. If this does not resolve the issue, it must be replaced.

The logs at dyndns.com show that wslope6 (isfs6.dyndns.org) registered 17 times with dyndns.org between Nov 12 11:40 and 12:01 MST. Each registration was for a different IP address. The last address that it registered with was 75.220.246.241, which is what dyndns is still reporting on Nov 14.

dyndns can blacklist a host if it is registering too often. I believe they send an email to the addresses in the account profile, which is currently my address, but I have not received any reports.

Logging into eol-rt-data and doing a tcpdump indicates that station 6 is reporting from 75.220.122.209. This address was not listed in the logs at dyndns.com. One cannot ping or ssh to either of the two addresses: 75.220.122.209 or 75.220.246.241.

Note that we are getting UDP data from this site from address 75.220.122.209 but cannot login.

My guess is that the signal is marginal at this site. One could bring a PC with wifi to get on the network at this site and point a browser at 192.168.0.1. After logging into the router's admin account, under Status->Modem (as I remember) one can see the signal level. Or, if someone has a Verizon 3G phone, we could get an idea of the signal level at this site.

Perhaps one of the little 6" whip antennas would help at this site, if placed as high as possible on the mast. If we still believe the signal is low, we may need a higher gain antenna.

Update on station 6 mystery. Sun Nov 14, 8:40 pm
~maclean/tcpdump.sh on eol-rt-data now indicates that station 6 is now reporting from 69.98.142.231. That IP is now pingable, but ssh fails with a timeout. dyndns.com still has the same IP address, 75.220.246.241, registered on Nov 12 12:01 pm

sensor issues

Here is a list of what I know about sensors at each site:

Site

Battery
&Solar

Sonic

Krypton

Baro

TRH

Rad

Soil

Soil.aux

Precip

Shadowband

Prop

1

ok

head off

not installed

ok

not installed

Rsw.in a bit strange, but went
to 0 when covered

ok

ok

not planned

SPN1 reporting 0
Licor not working

ok

2

ok

ok

not installed

ok

not installed

Rsw.in<Rsw.out

ok

not planned

not planned

not planned

not planned

3

charging?

ok

<5mV

ok

not installed

ok

turned off?

not planned

no fluid

not planned

not planned

4

ok

ok

not installed

ok

not installed

ok

Qsoil bit high

not planned

reporting all 0

not planned

not planned

5

ok

ok

<5mV

ok

problem -1<T<0

Rsw.in high by x2 until 11/14
Rsw.out high by x4 
Rlw.in/Rlw.out swapped

Qsoil started being
erratic on 11/13

Tsoil.4.4cm bad 
Qsoil erratic
Gsoil had spike 11/13, otherwise ok 
TP01 not reporting

not planned

not planned

not planned

6

ok

ok

<5mV

ok

not installed

turned off?

turned off?

turned off?

no fluid

not planned

ok

7

ok

ok

<5mV

ok

problem -1<T<0

ok

ok

not planned

not planned

ok

not planned

Well...we already know about I2C problems with the TP01 on site5.aux.  I note now that Qsoil is erratic at this site and even Gsoil had a spike yesterday that wasn't seen at any other site.  Tsoils are consistently okay, though.  At the very least, it might be a good idea to reseat these connections in the mote box.

P.S. As of ~1700 on 13 Nov, we're losing Tsoil.4cm as well.  I wonder if gophers are getting to these sensors???  I sure hope not.

I also note that all Qsoils show a small spike when the TP01 is heating.  The amplitude is small enough to be ignored (about 3 A/D counts), but we should be aware of this.

Humitter problem

We have found a bug in the humitter code where the output format is incorrect for temperatures between 0 and -1 C.  This causes both T and RH values to be reported as missing.  Examples are:

2010 11 12 12:28:44.0730  0.9708      29 TRH20 0.00 66.96 4004 2164\r\n
2010 11 12 12:28:45.0522  0.9793      30 TRH20- 0.02 66.95 4001 2164\r\n

2010 11 12 13:53:16.4885    0.97      30 TRH20- 0.99 74.28 3906 2430\r\n
2010 11 12 13:53:17.4685    0.98      30 TRH20 -1.01 74.27 3904 2430\r\n

The humitters thus will have to be reprogrammed.  Current thinking is to send all of them back to Boulder for Steve to fix.

This also is an issue for one (TRH11@7m) of the Manitou sensors.

Tbaro changes

I note that since we are for the first time mounting the barometer outside of the DSM box, Tbaro changes  much faster than before.  In particular, in the past 30minutes this morning, Tbaro has changed at the east sites from being nearly equal to T to being almost 10degrees higher.  I assume this is due to heating by shortwave radiation as the station comes out of the shadow of the mountains. 

The barometer pressure is compensated for changes in its temperature and is still well within its calibrated range, but it is possible that this rate of change could induce temperature gradients within the sensor that makes the Tbaro measurement (and thus any compensation) less valid.  It would be interesting to examine these pressure data in detail during these times to see if an error in pressure can be detected.  (Though it isn't obvious what the reference would be.)

Dynamic DNS

Immediately after the cellular modem connects to the Verizon ISP, the station registers itself with the dynamic name service, dyndns.com.

So, a good remote check on whether the cellular modem and router are working is to login to our account at http://dyndns.com and check the times of the host registrations.

Ask Gordon, Ling or Gary for our account name and password at dyndns.com.

Once you're logged in, click on the My Hosts link. You will see the 7 isfs stations, and two spares, along with the iss systems. The time (MST) that a station last registered with dyndns is shown. Click on Host Update Logs to see a listing of all recent registrations.

ping

Each of the stations should also be reachable via ping:

ping isfs1.dyndns.org

At site about 1400-1500.  Lots of standing water.  Need a hard hat and should check in at office trailer.

Tower site placed near E fence in NE corner on gravel.  A bit of a notch between buildings for N winds and relatively open to S.  Told not to worry about E winds, so plan on tower dropping to W.  Rad stand placed out more in the gravel to give better exposure.  Would be nice to set this stand as low as possible to minimize vegetation in field of view.

Soil was wet, sticky, black, relatively large granuals with a few stones (and worms!).  Actually not too bad to work in.  Placed near solar panel location.  Again, couldn't test sensors.

At site, perhaps 1145-1300.  Rain less, but turned to cold fog.

Chose to move from last night's location since Tom and I couldn't figure out how to interpret data over a combo of sagebrush and tilled field.  Dave contacted landowner and told anywhere in the stubble field was okay.

New tower site chosen in middle of wheat stubble field to minimize effect of 3m deep x 30m wide gulley to south and high mound to north.  Rad stand still to SSW of tower.  Despite discussions at the previous site6 location concerning the sonic height, IT WILL BE 3m AS ORIGINALLY PLANNED AT THIS SITE.

Soil black, sticky, globular (like site 4).  Again, couldn't (re)test these sensors.

At site about 1045-1130.  Still raining -- used canopy.

Tower site picked to avoid building for NNW winds.  Rad stand put on east side in case of west winds.  Should be able to drive quite close on track from parking lot.

Soil installed.  Sticky, globular black clay.  Quite wet.  Lots of dog droppings in this area (including the soil site).  Did not check sensor function.

At site ~0830-1000 with Jeff Lachowski from Kennecott.  Driving rain from the west.  White spots in Google had standing water.

Combo locks on two gates.  (silver lock on first gate).  Should inform Jeff when entering/exiting site.  Hardhats and googles should be worn.

Decided on site on the southern boundary to minimize effect of berm to the north.  South and West winds should be great.

Soils installed, 1 near rad stand (to west to minimize cross section to radiometers), 2 NW of tower.  Soil was dry on bottom, clayey in layers, but not obviously saline.  Gel-cell battery was down to 6.7V -- was able to see sensor IDs, but no data.

[THIS IS OLD -- SITE RELOCATED]

At site from about 1645-1745 (left in dark).

Dave says that he drove to site location (perhaps following the path drawn) in a 4WD vehicle.  Walking through the sagebrush was obnoxious.  Set-up crew should have Dave shown them the correct path.

Took a while to decide on site Dave had selected.  In the end, decided that an acceptable sonic height could be found, though perhaps not the 3m that was originally thought.  The issue is that the local sagebrush is significantly rougher than surrounding fields, but not extensive enough to dominate a flux footprint.

Sited rad stand to W by ~50m  This could be upwind for local downslope winds, but affect judged to be minor (compared to affect on radiometer view of large rocks and/or sagebrush elsewhere).  Two soil sites were installed, again one near rad stand and one near tower.  Soil appeared to be dark, fluffy, loam in both cases with just a few rocks.  TP01 "cap" lifted out like lasagne.  Sensors tested okay (Gsoil positive). 

[removed image, since site relocated next day]

Visited site from about 1545-1630.

Chose tower site at top of (very minor) hill.  Although soil at the tower was quite dry, chose more representative lower ground for rad stand and soil.  Rad has view of minor trees (we avoided being next to slightly larger ones).  Soil solar panel might be in field of view of rads, but judged negligible.

Soil was clayey, but slightly drier on top than site 3.  Also rocky.  Still moist below.  Didn't try to make "cap" over TP01 since layer crumbled.  Sensors tested good (Gsoil just slightly positive).

I think from ~1400-1500 flagged and installed soils at East Slope.

Chose tower location on ridge top with reasonably good exposure north and south.  East exposure (from Big Cottonwood canyon drainage) is directly over houses, though tried to avoid being downwind of the <biggest> trees.  Also tried to center between 2 footpaths, both of which were used by locals while we were there.  Chatted with local homeowner who said he'd keep an eye on the site.

Soil very crumbly with gopher holes.  Hopefully we won't have chew problems with our soil sensors!  All sensors in first replicate (near rad stand) tested okay.  TP01 in second replicate (near tower) didn't show up (via I2C) after several power cycles and white button pushes.  Other sensors were fine (Gsoil still negative).

In below image (from memory), locations of the tower, rad stand, and soil sites 1 and 2 are marked.  Also drawn is the access route we used.

From about 1:00-1:45 flagged tower and rad locations and installed soil sensors.

Soil rather clayey and held together reasonably well.  Sensors tested okay with John's setup (Gsoil negative).

Small site, so rad stand is reasonably close to tower.  Tower placed close to road site to avoid potential footpaths nearby.  We noted that the townhomes to the east don't have windows that will look out on this site.

A start    at a site visit    task list (with    what I remember    from Kurt's lesson):

- lower the tower:
  - grab charged drill battery
  - set mast support
  - attach winch (rotated at angle)
  - attach pulley (just outside bolts)
  - loosen winch, attach carabiner to mast cable, tighten winch
  - release front guy
  - remove plate
  - lower with drill about 1/3 trigger on lowest power (1)
  - watch front footpad while lowering
  - ease onto mast support

- clean krypton
- inspect sonic/prop/etc.

- raise the tower:
  - inspect welds at all tower junctions
  - inspect integrity of bottom pivot block
  - inspect battons for being perpendicular to mast
  - inspect side guy turnbuckles
  - ensure that front guy won't catch
  - raise in one smooth motion, full trigger on power (1)
  - slow trigger power near end
  - second person pushes in
  - secure plate
  - secure front guy and cable tie
  - loosen winch tension
  - detach carabiner
  - detach pulley
  - remove winch

- listen for T/RH fan

- inspect pressure port

- take gravimetric soil sample using corer -- both 0-3 and 3-6cm layers.  Process entire sample for bulk density.

- clean all radiometers with Kim-Wipes and distilled water (alcohol [or methanol?] if freezing??)

- adjust shadowband on radiometers (done by Utah?)

- connect to data system via wifi
  - check sensors okay (run cockpit?)
  - check USB disk status
  - check 3G network connection
  - check network time drift

Sensor

1
PLAYA

2
ABC

3
HIGHLAND

4
WEST VALLEY

5
EAST
SLOPE

6
WEST
SLOPE

7
RIVERTON

CSAT3

0537

0800

0855

0538?? 853

0671

1119

0539

Krypton

1395 / 1390 (1/2/11)

1393

1525

1258

1133

1389

1394

Barometer

B8
U4110003

B7
U4110001

B3
NCAR003

B2
NCAR002

B4
S0610001

B5
S0610002

B9
U411004

TRH








Propvane

00318

----------------

----------------

---------------------

--------------------

000314

-------------------

Rad. IN (pyn/pyg)

16945 /
31980

17414 /
31975

940185 /
050824

970380 /
030675

16107 /
31979

970379 /
100225

970377 /
050823

Rad. OUT (pyn/pyg)

17877 /
31976

15108 /
31978

970378 /
040740

940181 /
040741

16714 /
31981

940187 /
100226

940186 /
030676

LiCor

PY7121(DIR)
PY99291(DIFF)

----------------

----------------

---------------------

-------------------

---------------

PY10835(DIR)
PY10836(DIFF)

Tsoil

TS007
TS001

TS009

TS003

TS002

TS004
TS010

TS003
TS011

TS008

Qsoil

ECHO007(017)
ECHO010(020)

ECHO003(013)

ECHO005(015)

ECHO009(019)

ECHO004(014)
ECHO006(016)

ECHO008(018)
ECHO001(011)

ECHO002(012)

Gsoil

H993563(007)
H943149(001)

H013390(009)

H983211(005)

H023009(002)

H013391(004)
87034(010)

H993561(003)
H993560(006)

H983213(008)

TPO1

TP008(200242)

TP006(200592)

TP005(200239)

TP007(200241)

TP009(200243)

TP010(200234)

TP001(200589)

Rain Gauge

---------------------

---------------------

???????????

????????????

-------------------

????????????

--------------------

Motes

B,8,3,7

F,4,5

D,22,6

15,15,13

 

11,10,9

17,16,

Vbatt

31

32

38

34

 

36

37

Viper

16

1

14

12

 

6

3

Note:  These were the original sensors during testing and calibrating.

TP01 and EC5 lab testing

Here I'm dumping my work over the past month, to remind myself at the end of the project what I've done.

After consulting with Dave Gochis, I came up with about a 35-gallon plastic keg (borrowed from Steve Semmer) that I have half-filled with 150 lbs of play sand.  I can weigh this entire keg and add water to obtain a bulk gravimetric soil moisture content.  I roll/shake the entire keg to ensure that the water is thoroughly mixed with the sand and vibrate it (hit the sides with a pipe) to ensure that the bulk density remains similar (as determined by level marks inside the keg).  With the vigorous mixing, it is necessary to remove and reinstall the probes under test each reading.

Multiple Qsoil measurements show that the mixing process works reasonably well, though differences of up to 2% soil moisture still remain.  However, the Csoil measurements range by as much as a factor of 2, showing little consistency between readings with different probes and little (though some) consistency with Qsoil.  I have to conclude that this is a result of the installation process and thus have tried repeatability tests at constant Qsoil.  There still is about 25% variability between the same probe installed several different times in the same media.  (It is true that most of these tests were done near field capacity and should be redone at a lower Qsoil value.)  I have carefully dug up the probe each time and found that the probes are not being deformed significantly by my latest installation procedure.  There is little variation with the same probe installation as a function of time.  (We set it to make a reading every 3 hours and I run it for a day or more.)  I also get about 25% variability between different probes in the same media.  With all of this variability, I obviously do not see the expected linear variation of Csoil with Qsoil.  Clearly, this lack of repeatability seriously compromises our confidence in the data from these probes.

To understand this problem, I have also looked at asoil and lambdasoil values individually.  asoil (derived from Tau63) varies by as much as 30% from the mean of 5 probes tested, where the Hukseflux spec is 20%.  lambdasoil (derived mostly from Vpile_on) varies by 20% with a spec of only 5%.  Thus, either there is a problem with the Vpile measurement (that could be installation related, or Semmer wants to check the A/D gain programming), or the sensor gain factor E is incorrect.  We probably can check the E values in Agar gel, but won't have time for this prior to PCAPS.  This is a TODO item.  We might as well check the resistance R as well.

I attach the plot that I made of these lab data.  Note that the symbols 1--5 refer to TP01 sensors 1,2,4,10,11 respectively.

P.S. (Apr 2011):  These files are on my PC/projects/pcaps/soilkeg.qq.  Adjusting the values using the readings in Agar taken in Sept 2010 reduces the scatter (to generally within the 20%/5% specification), especially in Lambda, though there still is more variation in Csoil than I'd like.  Still haven't measured Rs.

P.P.S. See new logbook entry Apr 2011 about post-project testing.

  • No labels